Boris Johnson has been kicking up a stink over being brought to task on his misuse of statistics. The issue in hand is to do with the statistics which supposedly demonstrate the reoffending rate of prisoners leaving the Heron Unit at Feltham Young Offenders Institution, which is funded by the London mayor. When Johnson cheerfully announced to a Commons Committee that only 19% of prisoners leaving the unit went on to reoffend, compared with the 78% national figure, it didn’t bother him that the figures related to people who had only been out of prison for a few weeks. Not only that, but the comparisons made were inappropriate because they do not control for differences in the characteristics of the different populations.
These discrepancies were summarised by Sir Michael Scholar, who is chair of the UK Statistics Authority, and wrote to the chair of the home affairs committee to tell him that the figures do not stand up to scrutiny and should not be used.
When questioned about the issue during Mayor’s question time yesterday, Johnson was blasé, and has been quoted as having said, “There’s this guy called Scholar who writes me letters, who appears to be some sort of Labour stooge.” Making such a slur on a highly respected, impartial champion of trustworthy statistical evidence shows an unwarranted lack of respect. The question the mayor was being asked was whether, in light of this latest misuse of statistics, he would now sign up to the Code of Practice for Official Statistics. His response to that question was “I’m not minded to.” This casual dismissiveness is troubling and draws attention to what Ben Goldacre has called the “contempt with which politicians hold evidence and statistics”.
Making such a slur on a highly respected, impartial champion of trustworthy statistical evidence shows an unwarranted lack of respect.
Goldacre points out that what politicians want to do with statistics is to use them as narrative devices, adding weight to whatever story they’re trying to tell. In defending his misuse of the statistics, Johnson said “I used the statistics I had in my head about the success of the Heron Unit, because I want to promote that unit and I want to see more work done like that. And if it means advertising the success, I am absolutely determined to do it.” This basically amounts to saying that he believes the unit is working, irrespective of statistical evidence, and he’s going to do whatever is necessary to convince people of its success, regardless of whether the statistics he has in his head are valid or reliable.
This complete lack of concern for accuracy represents a deeper problem with the way in which politicians, journalists and the public treat statistical evidence.
This complete lack of concern for accuracy represents a deeper problem with the way in which politicians, journalists and the public treat statistical evidence. If we believe something works well, whether it’s a young offenders unit, a new way of teaching maths, or any kind of social intervention, it’s easy to assume that once it’s collected, the evidence is bound to demonstrate it works well, so therefore it’s fine to grab the nearest statistic that corroborates your view and use it to gain support.
There’s plenty of examples of how this nonchalant attitude to statistical evidence leads to policy mistakes and wasted resources. The Social and Emotional Aspects of Learning (SEAL) initiative is a case in point. Based on the instincts of well-meaning academics and policymakers, a national programme was rolled out, for which there was only flimsy evidence of effectiveness. Only when it had been set up and delivered nationally was a robust evaluation conducted, which demonstrated that it had virtually no effect on the domains it set out to influence.
Instead of dismissing expert advice, Boris Johnson should take seriously the need to gather and analyse evidence properly, and I think he really ought to apologise to Michael Scholar.