Our Results

WLZ is evidence-led: we believe it is possible to adequately evaluate the effectiveness of our partners’ programmes and the progress of the children and young people we work with.

Jump down to Our Methodology



Our results so far



READ HERE THE SUMMARY OF THE PILOT IMPLEMENTATION STUDY



Who did WLZ work with?
At the beginning of the pilot, our intention was to work with 120 children in 3 anchors – a children’s centre, primary academy and secondary school. We originally recruited 128 students, however, throughout the year, five students from Randolph Beresford, and five students from Ark Swift, left the cohort as families moved away from the area or changed schools, or, in one case, withdrew from the cohort. The final cohort was 118.





How did the cohort change?

We initially identified children for inclusion in the cohort by conducting a ‘risk factor’ analysis using academic, behavioural and attendance data, information about their communities and families, and a large dose of teacher knowledge.

We then used the same data at the end of the year to determine whether the children had progressed. Data quality issues meant that we were unable to track changes in attendance across the year.

Through this analysis, we determined that overall, 36 students exhibited fewer indicators of risk of poor future outcomes at the end of the pilot compared to the start, 55 stayed the same and 20 declined (please note, we were unable to track changes for some students in the cohort due to missing or incomplete pre and/or post data). In Phoenix High School, 14 out of 30 (46%) students reduced their risk profile. In Ark Swift Primary Academy, 9 (27%) students out of 33 reduced their risk profile, whilst in Randolph Beresford, the number was 13 out of 49 students (26%).



Measuring progress: data on outcomes

The major areas of change identified through the ‘risk factor’ analysis were attainment in Phoenix High School and behaviour in Ark Swift Primary Academy. At Randolph Beresford Early Years Centre a high proportion of our cohort scored low in the Early Years Foundation Stage Profile.

At Phoenix High School, in total, 87% of the cohort for whom we have data showed improvement in English (controlling for age). Of those ‘at risk’ in the bottom quintile nationally in English, 80% lifted themselves out of the bottom quintile. The support offered by WLZ focused more on literacy than numeracy, and the results bear this out with only 31% of the total cohort showing improvement in maths and only 20% of those ‘at risk’ in maths lifting themselves out of the bottom quintile.

executive-summary-of-wlz-implementation-study

Post-pilot improvement in English amongst Phoenix High School children, classified ‘at risk’ at start of pilot:

pie-chart-phoenix

At Ark Swift Primary Academy, progress in academic attainment was difficult to measure because the data was provided as teacher assessed levels. There are only five levels which does not allow for much movement or nuance. In addition, WLZ did not provide any academic support, as mentioned above. There were very modest improvements in English writing and although there was no recorded improvement in English reading and Maths according to the data, no students declined between the start and the end of the pilot.

On behaviour, at Ark Swift Primary Academy, for the 33 children for whom we had behaviour data, at the start of the pilot, 13 of these (39% of the cohort) had 0 behaviour incidences over the first 31 days of the school year, a number that rose to 22 by the end (67% of the cohort). Those who had 0-4 behaviour incidents also declined. And most importantly, of those 8 children considered ‘at risk’ with 5 or more instances, all improved as shown in the graph below (though 2 children who started with 1-4 incidents declined to more than 5 incidents).

Number of students  with behaviour incidents at Ark Swift in the first and last 31 days of the pilot: 

ark-swift-behaviour
 
The progress measurement at Randolph Beresford was positive with the numbers of ‘below baseline’ students declining in all three of the Prime Areas of the Early Years Foundation Stage (EYFS). For communication and language, the number of children below baseline shrank from 22 to 14, for physical development from 15 to 7, and for personal, social and emotional development from 20 to 10.

Numbers of students in the ‘below baseline’ groups for selected outcomes in EYFS, before and after pilot, Randolph Beresford Early Years Centre:

 rb-findings



Relationship between outcomes, Link Worker engagement rates and attendance at support sessions

Phoenix High School students who improved their overall risk profile (that is, the number of risks they presented with) by the end of the pilot year had on average 24.1 Link Worker (LW) engagements across the year, compared to 18.2 for those with no change and 12.5 for 2 students whose risk profile worsened.

Average attendance at partner support sessions was 70% for Phoenix High School, a figure which must be raised in future. In terms of partner support sessions, those students with 1 or 2 risk factors at the end of the pilot recorded notably higher attendance than those with 2 or more risk factors. Students who improved their risk profile attended more partner support on average, when compared with their peers whose risk profile declined or stayed the same.



Therefore, although the data set is very small, the results in Phoenix High School point to a positive relationship between higher attendance at support sessions, higher LW engagement and a reduction in risk factors.

At Ark Swift Primary Academy, LW engagement was spread evenly across the children, which was the engagement framework agreed with the school and partner services, and reflected the children’s age and capability to engage. In future, the intensity will be more varied according to need.

Average attendance at partner support sessions was 62% for Ark Swift students, a figure which must be raised in future. There was no clear relationship between attendance at partner support sessions and risk factors, although those students who improved their risk profile by the end of the pilot had on average higher attendance and those who still had a high number of risk factors (3 or 4) by the end of the pilot tended to have lower engagement with delivery partner support overall. However, risk factors at Ark Swift included 3 academic factors and 1 behaviour factor so the improvement in risk factors is heavily influenced by academic performance which WLZ did not focus on with this group of students. We are currently working on addressing some of these methodological limitations in our data collection and analysis that were evident in the pilot, to ensure that these outcome findings will be even more robust for Year 1.



Our Methodology

WLZ’s evidence-led approach has three components. Click the links below to find out more about the measures we use to measure progress (our ‘outcomes framework’), how we identify the right children and young people to work with, and the means of gathering data and analysing it.

  • Outcomes Framework

  • Identification

  • Input & Analysis