The middle leader job role is being used when listing vacancies and is working well when being used. However, we are seeing a number of false negatives of vacancies where we would expect a vacancy to be categorised under middle leader, but is instead being categorised otherwise.
Note that the figures of this analysis could be subject to change as more data is collected.
# Key findings
- ~10% of listed vacancies are being categorised as middle_leader
- Of the remaining 90% not middle_leader ~12/90, so ~12% total should fall under our definition of middle_leader and are considered miscategorised. Meaning we’d expect around 22% total vacancies listed as middle_leader and 78% not middle_leader (teacher, leadership, etc)
# Hypotheses
The following hypotheses on miscategorisation based on observing the data have been explored :
- More likely to misclassify middle leader in primary schools as roles are bigger proportionally and may “up” vacancy to leadership
- Leadership is too vague a definition and leads to people selecting over middle leader (journey is single select so cannot combine middle_leader and teacher or leadership for example)
- Pay affects categorisation (using TLR in job_role and salary descriptions to determine)
1. More likely to misclassify middle leader in primary schools as roles are bigger proportionally and may “up” vacancy to leadership
- 12% miscategorisation in general in secondary schools
- 7% miscategorisation in general in primary schools
- 15.5% miscategorisation in general in all through schools
Likely reject this hypothesis as miscategorisation in primary schools is below our total miscategorisation average. We are seeing more in secondary schools, but the numbers are still inkeeping with our overall average. Data suggests that primary schools are better at correctly categorising a vacancy correctly when it is middle leader.
2. Leadership is too vague a definition and leads to people selecting over middle leader (journey is single select so cannot combine middle_leader and teacher or leadership for example)
- We deem 23% of those categorised as leadership should really be classified as middle_leader
- Likely good evidence to suggest leadership is too vague
- Teacher categorisation is also misclassified ~11%, we think these should be middle_leader (as teacher has a much higher count than the other job roles this is a larger issue by quantity)
3. Pay affects categorisation (using TLR in job_role and salary descriptions as an indicator
- If contains TLR in salary description or job title - 42% miscategorisation
- If doesn’t contain TLR in salary description or job title - 9% miscategorisation
Other notable mentions (please note that some sample sizes are small when drawing conclusions):
- Secondary + TLR + leadership - 91% miscategorisation
- All through/secondary/primary + TLR + teacher - 89.5/70/62.5% miscategorisation respectively
- Secondary + leadership + no TLR - ~24% (moderate sample size)
CTR observations
- 1.4% searches middle_leader filter used with 12% unique searches leading to a vacancy view
- Comparing to leadership/teacher filters they have much higher usage 40/10% as well as almost double the views from unique searches at around 24%. It’s worth noting here that middle_leader is slightly under half it’s true “population size” due to miscategorisation. Proper categorisation would mean a greater volume of middle_leader vacancies which could increase views from searches, however, we can’t be sure.
- When a combination of filters including middle_leader is used e.g Teacher + middle_leader, views from unique searches increases to >20% + in most combinations.
We are seeing that the middle_leader filter is yielding fewer views on vacancies when used in isolation compared to other job role filters. This difference isn’t observed if middle_leader filter is used in conjunction with a combination of job role filters, if it is paired with another filter views are consistent to our service averages. It’s also important to note that the volume of vacancies being labelled as middle_leader are not representative of the vacancies that appear to be middle_leader based on job title descriptions. If they were to be correctly categorised this could lead to a change in the view percentage. It also appears middle_leader is more likely to be searched in conjuction with teacher + leadership than on it’s own, could suggest that users don’t exclusively search for middle_leader exclusively in the same way they do teacher or leadership.
# Summary
- Middle leader job role categorisation is being used and when used is used well
- Leadership job roles that we deem should be middle leader are miscategorised the most proportionally
- Teacher job roles that we deem should be middle leader are miscategorised less than leadership proportionally but as it is our largest volume account for a lot of the misclassification. Some of this seems to be from split responsibility roles. This could be due to the inability to select teacher and middle_leader together or they could be choosing teacher > middle_leader for other reasons.
- Vacancies with TLR in the description leads to high miscategorisation of middle_leader, instead being categorised as a mix of leadership and teacher
- Middle_leader filter is leading to views, however, it is currently lower than our other filters (teacher, leadership) proportionately
# Next steps
Noting the percieved confusion of when to use leadership or not we have recommended to change the wording of “leadership” to “senior leadership” to help our users discern between selecting “leadership” or “middle leader” for job roles.
We have also recommended to investigate allowing for users to select both “middle leader” and “teacher” as job roles when creating a vacancy. As not all vacancies seemed to be able to be categorised definitively as either or, leading to miscategoisation in our results.
Accounting for these two changes we hypothesise to see an improvement in usage of our middle leader job role category by having fewer false negative categorisations. Which im turn should improve search for users browsing vacancies.