If we compare thee favourably to a summer’s day, you might accept it as a compliment, but not the basis for a league table. People and weather are categorically different, obviously. Such comparisons are impossible – with apologies to Shakespeare – without a good deal of definitional fudge. In a sonnet, we approve, and call it metaphor; but in government . . .

Politics has grown to love league tables and their raw material, the performance assessment: how one compares with another, who’s up, who’s down, who’s top, who’s bottom, who’s good, bad or “bog standard”, who shows us “best practice”.

Comparison has become the supreme language of government. It is now, in many ways, the crux of British public policy, in the ubiquitous name of offering informed choice.

But it is more common than that. Measured comparison is the staple of political argument. Almost every “this is better than that” attempts it. The motif to keep in mind is one that everyone already knows, but has grown stale with overuse. It is as true as ever, however disguised in league tables or performance indicators, and it is this: is the comparison of like with like?

In September 2003 a teenager, Peter Williams, attacked Victor Bates with a crowbar in the family jeweller’s shop while his accomplice shot and killed Mr Bates’s wife, Marian, as she shielded her daughter. Twenty days earlier Williams had been released from a young offenders’ institution. Twenty months later he was convicted as an accomplice to murder. He was supposedly under a curfew order and electronically tagged at the time of the murder but, in the short time since his release, had breached the order repeatedly.

In autumn 2006 it was revealed in reports by the National Audit Office and the House of Commons Public Accounts Committee that, since 1999, convicts on tag, as they are commonly called, had committed 1,000 violent crimes and killed five people. Tagging was denounced in parts of the media as an ineffective, insecure alternative to prison, putting the public at risk and used only, it was alleged, because in comparison to prison it was cheap.

The defence of tagging was brisk, assertive – and dependent entirely on spurious comparison. It was argued that of the 130,000 people who had been through the scheme, only 4 per cent had committed offences while tagged, compared with a recidivism rate for newly released untagged prisoners of around 67 per cent. Tagging was thus a beacon of success. Comparison is a fundamental tool of measurement and thence judgment. If we want to know the quality of A, we compare it with B. In the case of criminal justice, the comparison is generally between the statistical consequences of the alternatives: what would have happened, how much more or less crime would there have been if, instead of B, we had tried C?

But comparison runs into all manner of booby traps, accidental and deliberate. The principal problem is well known: any comparison, to be legitimate, must be of like with like. And this particular comparison – of reoffending by convicts on tag with reoffending by others – was a model of what to watch out for, of how not to do it, or at least how not to do it honestly; an object lesson in the many ways of bogus comparison.

The comparison was bogus for three reasons:

1 It did not compare like with like. The people who were tagged were judged suitable for tagging because prison governors decided they were least likely to offend. The released prisoners, since they served out their sentences, had clearly been judged more likely to offend. So the result of the comparison was a foregone conclusion. The claim of success for the tagging scheme when the people differed so much, and in fact were chosen precisely because they differed, was, to put it politely, lacking rigour.

2 The time period was different. The maximum period of tagging during which offences were committed was four and a half months. But offences by released prisoners were counted for two years. Obviously, there will be more.

3 The wrong people were compared. Tagging should have been compared with its alternative. The alternative to tagging is not to be at liberty: it is to be in prison. Either you are let out early wearing a tag, or you are not let out at all. These are the two groups that should have been compared: and obviously, since people in prison do not commit crimes against the public, tagging would have come out badly.

The comparison was an open and shut statistical felony, and even prompted an unusually forthright attack from the normally reserved Royal Statistical Society.

Explore Books

• Book Extracts

• Book Reviews

• Books Group

• Audio Books

Times Recommends

• The 100 Best Books of the Decade
• The Times picks the 50 best paperbacks of 2009
• The conversation: James Ellroy