Can we assess education without the bell curve?

This summer the government introduced new measures for tracking educational achievement at GCSE level in British Schools - Attainment 8 and Progress 8 - which have been hailed by some as a great improvement in how we judge exam results. Michael Reidy FRSA suggests that while some elements are welcome, others contribute nothing to understanding achievement.

Adults have more problems with education than children. For the most part, children accept school for what it is, and by and large, do what they’re asked. Some do it exceptionally well; others fail to engage after a certain level, but will attend for social reasons; the odd subject or activity they enjoy, or sadly, because it’s better than being at home.

Adults, on the other hand, are conflicted about education because they are looking at it with their own selfish squints. To them education is about one or more of the following:

• Identifying the best of the best
• Preparing children for 'life' and employment
• Creating a more equal society
• Sharing cultural values and protecting national values
• Achieving value for money
• Enriching children with sport, art, music, dance

. . . and so on.

Adults also expect the above to be meaningfully measured with results that can be plotted on a bell curve. The problem is that using a bell curve means that 40-50 per cent of children 'fail' automatically. Actual achievement doesn’t objectively matter, the curve will nonetheless deem them unworthy.

While easy to understand, the curve is not a good measure of learning, nor is it good as a criterion for forming educational policy. Compounding the problem of the bell curve is that no politician will ever admit that 50 per cent of children will always be below average.

Attainment 8 goes a step too far

The recent introduction of 'Attainment 8' and 'Progress 8' attempts to address a number of these problems with measurements of educational progress and achievement.

Measuring the best eight results of each child provides a real step forward in attempting to give an accurate picture of achievement. It creates a more even playing field and rewards progress in a wide variety of subjects.

However, double-weighting English and maths is less easily justifiable. Command of the language is important, but is it twice as important as other subjects? The same is true of maths. While I certainly advocate mastery of addition, subtraction, multiplication, division and being able to work out percentages, what else is really needed by non-academics?

My main objection to the Attainment 8 and Progress 8 approach is that it tries to create data to measure the success of schools by averaging their scores across all subjects. To me this is negating the value of A8 and P8 as they measure individual performance on specific subjects.

There is little statistical justification for combining them, and because the number of students taking any given subject is not considered in the average, the result is suspect. For example, a +0.5 history result and a -0.5 physics result will yield a 0.0 average, even if 100 students took history and only 10 physics. Averaging subject grades produces a mush that actually removes information from the numbers and leaves only data. Very little can be learned about a school whose A8 or P8 scores are +0.5. Clearly it is doing some things right, but it could be abysmal in others. As a snapshot it’s very blurred.

As an alternative, looking at subjects individually can provide a great deal of information for students, parents and employers. One can get a local and national picture about how good one is in history, for example. Parents can select schools on the basis of their achievement in certain subjects. Individual subject performance is also helpful for college and university admissions, and for employers. This measure provides a great deal of contextual information about the individual and school.

Perhaps the only benefit of averaging overall scores is that it lends itself to the bell curve model, producing the same unhelpful picture that curves always did. Yet, the curves endure; Gaussian distributions are preserved, and generalisations and policies made on their supposed validity.

Such curves were originally used (outside academic study) to plot risk, then extended to predict performance, and make decisions on matters never originally intended. When used in exam results, the height of the Y axis (achievement) makes little difference; it’s the grade allocation made on the X axis that is important, and infinitely flexible. Like predicting the future by extrapolating the direction of line, bell curves always continue in the same direction, depending which way the line is going at the starting point.

Bell curves blur the picture of both achievement and progress and circumvent the need to read the real results and think about what they mean.

Preserving information

More useful are bar graphs that show the national, school and individual levels of progress and achievement by subject. This provides information, not mere data, and is a far better basis for assessing programmes, schools and policies.

Results presented this way are far more comprehensible. Students can clearly see where they are; parents choosing schools and admissions departments choosing students can make informed decisions, and more information is available to administrators and policy makers, without the hindrance of a bell curve.