04 Aug 2013

Teacher Proof: Tom Bennett's dose of healthy scepticism

So, when was the last time I read any book, laughing out loud? Can't remember. But I sat by the river Dee on a sunny Saturday morning and read Tom Bennett's forensic piece of work on educational research. I had to keep wiping my eyes.

But this book is serious, deadly serious about some of the less fragrant ideas to have wafted their way into our classrooms. They have arrived in schools via universities, the DfE, LAs and commercial courses. Bennett takes his cue from Ben Goldacre's 'Bad Science' which, among other things, debunked 'Brain Gym'. This book sets out to show why research in education doesn't always mean what it claims. And what we can do about it. Yes, and you too.

Bennett's background in philosophy, religious studies and Soho night club management mean that he brings both rigourous epistemology and tough egginess to the process. With a lightness of touch which belies significant scholarship, he shows how research in science differs fundamentally from research in the social sciences. More importantly, he shows how bad research in science can happen through sloppiness, vanity and cargo cult science. Am not going to expand on that - read the book. A beautiful example is the story he tells against himself. One of his assignments as a student in the sixth form was to conduct  experiments on extracting caffeine from coffee. All carried out with meticulous care. Only to realise, after he finished, that he'd forgotten to measure the original amount before he started. That's what's so clever about this book. He takes his subject, but not himself, seriously. So, for example, if we want to spot when a piece of research is from the cult of cargo, it will say somewhere 'sponsored by...', which is not necessarily the kiss of death, but should make us alert to some of the biases behind a research project. Such research might only refer to a very small sample, there might not be any kind of control and the control might not be double-blind. All in all, failing the fundamental tests of research.

Now, his section on confirmation bias is very interesting. I personally love the views of people who share my prejudices, so this made me think. Apparently, confirmation bias is a long-observed effect in research. It is the tendency of people to favour information that confirms their hypothesis and discard that which does not. And that's because science is conducted, as Bennett says, by human beings, not robots. So, Dan Willingham's recent tweet to follow those we don't necessarily agree with, is healthy. That's what I took from this section, anyway.

And a wonderful section on correlation versus causation. Ooh, how often have I assumed that because something happens at about the same time as something else, I can spot a link. Big mistake. Pause for thought. And a priceless injunction at the end of this chapter 'If you make a factual claim about the world, I expect you to provide factual evidence. Otherwise it's flatulence.' Only an RS professional could come up with that. More mopping of the tear ducts.

He has nothing against social science per se - its intention is to comment on and understand humanity. But it is much harder to draw solid conclusions about human behaviour. This because the subject matter, human beings, unlike a pile of bricks, is made up of unique individuals. They have many attributes which militate against solid conditions for research. For one thing, they move, for another they are likely to change their minds. Oh, and none of whom is going to respond in an identical way.

I hadn't come across the Tooley report before. 'Educational Research', commissioned by Ofsted, was written in response to Professor David Hargreaves' criticism of educational research being 'poor value for money, remote from educational practice and of indifferent quality. Another tear wiping moment when I read Bennett's observation on this:  'As you can imagine, this went down like surf and turf at Gandhi's funeral'. The Tooley report itself had taken care to follow the principles of good research in its breadth and randomisation of the sample. It confirmed what Hargreaves had claimed. Namely, that there is a 'considerable amount of research that is frankly second-rate, which is irrelevant to practice and which clutters up academic journals that virtually nobody reads.' Oh dear.

Perhaps most sobering of all is this. That neither Tom Bennett, nor any students teachers, including myself at the time, were given any guidance in how to conduct a trial, write a research paper, or approach the matter in anything approaching a scientific manner. And in a field where there has been so much quasi science, this is more important than in most professions.

So, enough of the negative. Bennett does say that there is good research being done. That there are useful, credible studies that can assist the understanding of education, how children learn and ways in which we can ensure they are safe, well taught and tended in our care. But that even more important is the 'great collective ocean' of experience and understanding which resides within the profession itself.

Bennett proceeds to demolish multiple intelligences, NLP (should be mothballed forever), Brain Gym, group work, the thoughtless use of new technologies, the three-part lesson, learning styles and six hat thinking. Now, at some point I am going to write 500 words on the place of group work and to be fair, Bennett does provide examples of how and when he uses group work in practice. This is after an eye watering account of the guerrilla tactics he used when at school himself to survive the onslaught of the gorillas he regularly had to 'work' with in a group.

I hope Bennett follows up this substantive piece of work when he turns his attention to some of the stuff that does work.

In the meantime, a great call to scepticism. And that's all of us.

comments powered by Disqus