Students across Northern Ireland woke up last week for what could be the most important day in their lives to date: A-Level results day. Having spent two years studying subjects they love and applying for University courses in their chosen fields, yesterday was supposed to mark the day that they found out if all their hard work had paid off.
For many students across the UK, it was a day of devastating and confusing news as they received results that didn't match their levels of academic achievement. The coronavirus pandemic came at just the wrong time for students, making it unsafe to sit exams and forcing the education boards to come up with alternative ways to grade GCSE and A-Level students.
Boards across the UK used algorithmic and statistical approaches to calculate a student's expected grade from previous known data, and in the process have once again shown the power of algorithms to completely miss the point. While it's now been announced that students will get their teacher's predicted grade if it was higher than the one they were assigned, it's clear that A-Level students have been severely let down.
The students whose lives have been disrupted
When the actual A-level results came out, it was immediately obvious that something was wrong. Stories have been popping up all over social media of extreme discrepancies in results, with students predicted to get straight A grades somehow receiving E's and D's. Official figures showed that over 40% of students in England had their submitted scores downgraded, with 37% in Northern Ireland.
It's natural for some percentage of scores to be downgraded compared to teacher expected grades, as teacher projections are often higher than the final attained grades, but some schools saw reductions across the board compared to previous years. There have also been cases of students who performed well in coursework, AS levels, and even mock exams but were awarded much lower scores without explanation.
The statistical breakdowns of results compared to previous years have given many cause for alarm, as it appears the difference in attainment was split along economic and geographical boundaries. Students enrolled in private schools with smaller classes were more likely to attain highly as their teacher's predicted grades were given more weight, while teacher's predictions were given no weight in larger testing centres in typically more deprived areas.
The decision was made on Monday to give students the higher of either the algorithm score or their teacher's prediction, but this may be too late for some. Many students who already had conditional offers at university are now left in limbo as they've been rejected and their place might have gone to someone else already. Some students may end up having to defer to next year to take their place, or give up on their first choices.
Statistical analysis sacrifices the individual
CCEA announced in April that it would be using a kind of predicted grade system to assess students in Northern Ireland. Schools would be asked to generate assessment grades for each student in each subject based on previous assessed work, which would then be sent off for evaluation. Students would then be ranked in order of assessed score, and a statistical method would be applied to assign each person a final grade on a bell curve.
Statistical modelling can be a great tool for predicting overall trends or planning education interventions, but it doesn't preserve the fine details. The model might spit out similar grade distributions to previous years but it treats everyone as one homogenous group, leaving no room for exceptional individual improvement and ignoring that rates of achievement can vary significantly between schools.
There are always students who get poor A2 grades and spend the entire year studying hard to make up the ground; I was one of them! I got A, B, D, D at A2 level and was predicted to stay there, but managed to drag one the D's up to a B in order to qualify for university. I graduated with a first class Masters degree in Computer Science, but I would have failed to even get into the course if assessed under this year's rules even if awarded the teacher's predictions.
What happens now?
I get the feeling that we're yet to see the full fallout from this year's cancelled A-levels. Even with the U-turn on predicted grades, universities may be unable to offer enough places to cover the offers on the table this year. There may be knock-on effects on university drop-out rate in the coming years too, and teaching resources could be stretched thin if classes get too big.
No matter what the outcome for students, this incident should serve as a warning on what can go wrong when we put an algorithm in charge of predicting a complex system like academic achievement.
