The trouble with evidence-based policy

I’ve had a go at a critique of ‘evidence-based policy’ in another place (as they say), and have not necessarily won the argument.  What I’m trying to say is that criticising the specific ideology called ‘evidence-based policy’ does not mean that somehow we should stop demanding evidence.

This emerged from a fascinating blog by the Liberal blogger Jonathan Calder, criticising the new political class.  This is what he says:

“These days mainstream politicians are overwhelmingly likely to come from the same wealthy middle-class families, to have been to the same limited range of schools and universities, to have worked as special advisers (and perhaps in a more lucrative career and then to have been selected to fight winnable seats. The are all light on ideology and tend to buy in their policies from charities and think tanks. Their shared enthusiasm for ‘evidence-based policy’ disguises a tacit, unexamined agreement about the nature of the problems we face. Where is the evidence-based policy for reducing income inequality, for instance?”

Jonathan is quite right to link the distrust of ideology with the strange growth of platitudes around ‘evidence-based policy’.  Nobody is in favour of evidence-free policy, of course. The problem is that ‘evidence-based policy’ is not the opposite. As currently understood, it is a discrete branch of utilitarianism.

Underlying evidence-based policy is the assumption that, before change can happen, the right kind of numerical evidence must appear. Often this kind of evidence is quite impossible, one way or the other – and evidence-based policy thus becomes an argument for inaction, dressed up as hard-headed objectivity.

It isn’t objective, of course.  Numbers look hard-headed but they are actually chained to definitions, which are endlessly malleable.  Numbers can take you by surprise, yes, but you can’t rely on numbers alone.

Evidence-based policy is at the same time a deep and abiding scepticism about change of any kind, except tiny changes, linked to a deep and abiding reliance on a particular kind of statistics – an implicit belief that, despite indications to the contrary, that the target figures (any figures actually) must be true and accurate.

It isn’t really about evidence at all. It is Gradgrindism dressed up as open-mindedness, the idea that all you need somehow is to measure your way to an answer.

“He had an open mind for so long,” somebody said of Bertrand Russell, “that he couldn’t get the damn thing shut.” That’s evidence-based policy.

It leads to inaction, as I said, but it also leads to a great deal of absolute tosh. Because the Prime Minister wants evidence-based policy, then he must be given graphs. The figures may be based on the tiniest samples – they may be wholly inaccurate – but it is evidence.

Then there is the worst of it.  Evidence-based policy, as currently understood, can only compute evidence from institutions as they currently exist. That is the only possible source of this kind of evidence.

It rules out consideration that those institutions might be very different. Why? Because not only is there no evidence in the approved shape, but there can’t be – the world will have shifted and the definitions changed.

I’ve talked to medical statisticians about how flimsy the evidence used by NICE is. I know from my own experience in government departments how graphs get created out of the flimsiest stuff. But it’s evidence-based policy – how can anyone possibly be against it?

Jonathan is quite right that this tends to go hand in hand with a scepticism about ideology.

Ideological approaches certainly need to be subjected to common sense, and real evidence – the stuff you get by looking at things and talking to people.

I can think of so many ways in which ideology has undermined policy. But there is another way of looking at these things: bringing ideology to bear on problems is to look at them sideways, from the point of view of a philosophy and a tradition.

Often that is the only way you can break free of intractable problems. You can’t do it regardless of evidence, of course. But don’t let’s allow ‘evidence-based policy’ to stop us, because it isn’t what it seems.

It looks like a hard-headed guide for common sense. It actually conceals an unreliable lens that will tend to support the status quo.