Over on BlueSky, which suddenly become the main place where education discussion happens, a chance discussion has shown how powerful AI can be in analysing and interpreting data.
Ben Newmark started a discussion about the need to abandon all use of predicted grades.
In all the target grades chat there's a fair few sensible people saying it's wrong to use them before KS4. Respectfully the stage makes no difference. The way they are formed isn't valid so it's really no better to use them when a kid is 15 than it is when they are 11.
— Ben Newmark (@bennewmark.bsky.social) Aug 18, 2024 at 7:52
After much fascinating discussion, Richard Selfridge shared this fascinating but dense set ofdata:
Here's the transition matrix for GCSE Maths in 2022/23. explore-education-statistics.service.gov.uk/data-tables/… Every GCSE grade has at least 1 student who achieved each KS2 Scale score.
— Richard Selfridge (@richardselfridge.bsky.social) Aug 19, 2024 at 12:22
[image or embed]
But it’s not immediately obvious what this means. So I went over to trusty Claude.AI, pasted in the image of the table and asked
Could you analyse this table and tell me the implications for how we try to predict students’ likely GCSE grades from their KS2 SATs?
Here’s what Claude said:
This table provides valuable insights into the relationship between Key Stage 2 (KS2) performance and GCSE outcomes. Let’s analyze the implications for predicting GCSE grades based on KS2 SATs:
- Wide range of outcomes:
The table shows that students with similar KS2 scores (represented by the columns) can achieve a wide range of GCSE grades. This suggests that while KS2 performance is correlated with GCSE outcomes, it’s not a perfect predictor. - Imperfect correlation:
Even students with high KS2 scores (e.g., 110-113) can achieve lower GCSE grades, and some students with lower KS2 scores can achieve high GCSE grades. This implies that factors beyond KS2 performance significantly influence GCSE outcomes. - General trend:
There is a general upward trend where higher KS2 scores are associated with higher GCSE grades. However, the relationship is not linear or deterministic. - Overlapping distributions:
Each KS2 score range shows a distribution of GCSE grades, with significant overlap between adjacent KS2 score ranges. This makes precise predictions challenging. - Ceiling and floor effects:
At the highest KS2 scores (117-120), there’s a concentration of high GCSE grades, suggesting a ceiling effect. Similarly, at the lowest KS2 scores, there’s a higher proportion of lower GCSE grades, indicating a floor effect.
Implications for predicting GCSE grades:
- Use probabilistic approaches:
Instead of predicting a single grade, consider using probability distributions or ranges of likely outcomes based on KS2 scores. - Consider multiple factors:
KS2 scores alone are insufficient for accurate predictions. Incorporate other factors such as student engagement, subject-specific aptitude, and ongoing assessments. - Regular reassessment:
Given the imperfect correlation, regularly reassess predictions throughout the GCSE course as more current data becomes available. - Avoid overly narrow predictions:
Be cautious about making very specific grade predictions, especially early in the GCSE course. - Differentiated support:
Use KS2 data to identify students who might need additional support or extension, but remain open to individual variations in performance. - Growth mindset:
Emphasize to students that their KS2 performance doesn’t determine their GCSE outcomes, encouraging effort and improvement. - Tailored interventions:
Design interventions that target specific areas of weakness indicated by KS2 performance, but remain flexible based on ongoing GCSE coursework performance. - Data-driven decision making:
Use this transition matrix data to inform school-wide strategies, resource allocation, and teacher training to support students across the performance spectrum.
In conclusion, while KS2 SATs provide a useful starting point for predicting GCSE performance, they should be used as part of a more comprehensive, dynamic, and individualized approach to student assessment and support throughout the GCSE course.
Not bad, huh?
Funnily enough, one of the first ever blogs I wrote, back in 2011 was about this ‘probabilistic approaches’ idea, when I was a newbie teacher to Twitter and also a school data manager who was coaching/supporting other schools and data managers,
Follow me on BlueSky at https://bsky.app/profile/informededu.bsky.social