Tom Bennett

Home » Bad Science

Category Archives: Bad Science

Thinking in the right direction; just don’t put all your faith in RCTs. Ben Goldacre’s vision for evidence based learning.

But is it outstanding?

Bethnal Green Academy, a soy latte’s throw from both hipster Columbia Road and the surrounding estates, was the venue for the launch of Ben Goldacres’s new advocacy project: Building Evidence into Education. Nice looking school; it’s got BSF written all over every plane and pane. The livery outside the school shouted every second sentence of the latest Oftsed report. Most pleasingly it was styled in the Star Wars font (or as my kids call it, ‘That old film’). 

Hosted by Teach First, the Royal Marines of the teacher profession, we were first treated to an introduction by Dame Gove himself in what was, I thought, a remarkably short set. It was like booking Geoff Capes and asking him to open a jam jar. Goldacre followed;  a passionate and determined thinker and speaker, whose Bad Science series shook me, like Hume did for Kant, from my dogmatic slumber.
It’s a familiar saw for him: the need for Randomised Controlled Trials (RCTs) to become more prevalent in the social sphere (here represented by education), as they are in the medical profession. He made an interesting comparison: medical practitioners at first resisted RCTs because they were convinced of their own intuition, until it turned out that intuition often led us down the wrong path; eg using steroids for brain inflammation did more harm than help- the reverse of orthodoxy. Now, so are teachers (resisting RCTs, not harming the brain. Oh wait.)

Goldacre’s point is refreshing and disarming: we run trials like this for every pill and potion; why not policies, which in the galactic scale of things, dwarf even pharma for dollar tonnage. And as usual, there is a lot of practical wisdom in what he says. I am delighted that the DfE is looking into ways of conducting research that is robust and real, rather than fanciful and driven by dogma.
So why not have RCTs in the social sciences? One possible objection, the ethical, can be easily dismissed: issues of consent, and issues of knowingly withholding a potentially useful intervention from the control group. But we already introduce dozens of interventions in schools every single day. It isn’t as if we aren’t already drowning our kids in optimistic hoodoo. Why NOT randomise it once in a while? When was the last time we asked a kid’s consent before we popped a Thinking Hat on them, or told them to rub their brain buttons? 
I think there are, however, some serious problems with the use of RCTs in education.
 RCTs aren’t the answer to the question ‘What really works in schools?’ because differences in context can never satisfactorily be ironed out. The causal density of humanity is too high; there are too many factors to establish a reliable protocol that could hope to encompass the variables of the human mind. There are as many factors to juggle as there are grains of sand on the beach. Lazy research is sodden in bias and assumptions, wooly over interpretation and optimism. That said, there is room for quantitative research , some of which will be amenable to RCTs. I’ve heard social scientists say that the problem is merely one of design; I say the problem is the methodology itself. If you’re making quantifiable predictions about the physical world you need to provide unambiguous methods of establishing initial conditions as well as outcomes, possible causal mechanisms, and demonstrate reproducability. That isn’t easy.  

Also, the problems of cognitive bias, the Hawthorne Effect, the John Henry Effect, the Pygmalion Effect and so on are enormous obstacles to appreciating when an intervention is effective and when it is not.

This is definitely thinking in the right direction. Goldacres’s attempt to lasso the spoon bending of social science and pin it down to some kind of methodological rigour, is exactly what I, and many teachers, want to hear, and I absolutely support his attempts to reinvigorate the debate. But there are many obstacles to this, even before considering motivation.
One is time. I work pretty hard. I don’t expect a biscuit, because I bet you do too. I’m also pretty good with my time. And I barely have time to do, well…this kind of thing, and the only reason I do is because I cash in hours of sleep like chips in a casino. Doing research into education is going to be a niche pursuit until workloads go down. Like archaeology in the 19th century, it will remain the hobby of gentlemen.
And let’s be honest, most teachers aren’t scientists. They’re as prone to their pet prejudices and tender perversities as most people. Read the Tooley report if you want to see how badly education research can be conducted even by people who should know better. Now multiply that by 100 and you get, I would say, most of us. Many teachers would love to get involved with what Goldacre sensibly described as driving research.But would research benefit? Not until teachers learn what the scientific method is.

Dating for Nerds
So, what next? From the presentations, and the discussion one thing became apparent: this isn’t the launch of anything concrete yet. At this point, the program is advocacy. There was no funding committed, no projects starting tomorrow. From one perspective, it’s hard to see many education secretaries getting too worked up about RCTs- the average tenure of the Headmaster-in-chief is about 18 months. RCTs can take years; to be efficient, to be meaningful, they have to be as large and long as possible (as the inspector said to the janitor). Why invest in something that a) won’t bear fruit until after you’ve been moved to The Department of Silly Walks and b) Might disagree with your own pet projects? It’s always safer to simply pick research that appears to validate your own objectives.
And yet, and yet. Gove was, at least, there. The Dfe’s involvement with Goldacre shows, at least, a symbolic commitment to better research in education. And let’s not forget the £125 million DfE funding for the Education Endowment Fund (represented here by Dr Kevan Collins) a Sutton Trust start-up devoted to research, some of which involves RCTs. There might be some will there, but it isn’t very full-throated. 
So much in education is an abstract, an artefact of art as much as engineering. We can barely agree as to what educated means. Or learning. Or thinking. Or engagement. Or creativity. These aren’t amenable to metrification.How do we study what we cannot catch?
But to sound a more positive note: anything which seeks to firm up the wiliest of educational research is an asset.  Goldacres’s profile should help turn a search light upon the relationship between research community and Chalkface warriors. He’s absolutely right when he says that there needs to be more communication between the practitioner and the research communities. Teachers make bad researchers, and often, researchers make bad teachers, and the assumptions are apparent in many articles of research I read- where assumptions and biases that would have you laughed out of the physics club are common.
One thing’s clear: it’s a mess. I’m not so sure, as Mark Keary and Ben Goldacre implied, that we’re on the cusp of a Golden Age of research bounty in education. They’ve been saying that about the social sciences since they were invented, and microscopes and abacuses, which had been so generous in the natural sciences, were turned to the human sphere. We’re still waiting for the Industrial Revolution in social science.
One thing teachers need to do, exactly as Goldacre said, is to familiarise themselves with the the principles of science; to arm themselves with at least a basic understanding of what it means to say an intervention is true, or probably true, or probably not. God knows we need to, given the deluge of garbage that we’ve endured, justified by silly hat research.
So I wrote a book about it. It’s called Teacher Proof, and it’s out this June. 

Uri Geller bent my classroom: a parable of bad science in education

‘Guess where this has been?’
I met Uri Geller once. You may know him as the Israeli illusionist and magic man who claims, among other things, that aliens visited him when he was a boy and gave him the world-changing ability to…bend spoons. No, I don’t know what they aliens were playing at either. Bending spoons doesn’t stand high on my wish list of superpowers, but I expect it’s jolly important to the fate of humanity. Anyway. Uri Geller makes a living bending spoons, and telling people he can bend spoons, which means he must hang out with a lot of fairly stupid people,. He was a chum of Michael Jackson, you know. Isn’t that nice? The King of Crawl and Magneto, Master of Cutlery and condiments. What a pair. Mankind’s only hope.
So I met him. I was running a restaurant (TGI Fridays, for my sins) in London, Piccadilly Circus, and Mr Geller was in town, on a tour bending spoons. (Jesus Christ, I feel like I’m writing this about the Victorian Music Halls- ‘Mr Geller the Israelite will bend metal paraphernalia to his will using electromagnetic mesmerism’ etc) He was dining right in the centre of an already indiscreet restaurant, and we had clocked him the moment he walked in. There aren’t many spoon benders, to be fair. 
At the end of the meal, he jumped up (I am not making this up- he leapt up like I was in a cutlery set. I feared for my belt –buckle) and grabbed me by the wrist.
‘Do you know who I am?’ he said. He was staring at me like I had murdered his family. He had presence, I’ll give him that.
‘Of course, Mr Geller,’ I said, big smoothie. ‘You’re very well known.’ It’s odd speaking to a famous person who demands to be recognised. More from pity than anything else I wanted to put him at his celebrity ease. Plus he frightened me.
‘Do you know what I do?’
‘What apart from lie to people and hide bent spoons up your arse?’ is what I wanted to say. But because I wanted to enjoy a continued career in the upper end of the casual dining market, I demurred and flattered his withered ego.
‘I believe I do,’ I compromised.
‘Would you like me to show your staff something amazing?’ he said. In truth, I would have. Instead, I was worried it was going to involve a spoon. But I nodded.
‘Gather them round,’ he said, ‘And get me a fork.’ Curve ball.
As my staff were all carnies and out-of-work showbiz types, the chance of a bit of star f*cking and spectacle was irresistible. Try and get them to sing happy birthday to a child and they vanished like rats in a spotlight. Now, they clumped like iron filings around a magnet. Now the next bit is important. I went to the kitchen and fetched a fresh fork from the dishwasher, which is to say I had to clean it again before I took it out. I got the fork. One of our forks. That bit is important.
I brought the fork to him and we crowded round like mobsters round Brando in Guys and Dolls. Geller took my fork in one hand, and lightly placed two fingers in a benediction on to its throat. He rubbed them back and forth like a cautious DJ, everyone’s eyes trained on the unremarkable utensil, inches away.
And the fork started to bend. 
Slowly, but visibly. And it- at least it seemed– to carry on bending after he took his fingers from it. Reader, he bent the fork. Looking pleased, he smiled a smile that nature never made, and passed the fork to me. Curved, the neck was as stiff as steel, not hot. I held it like it was made of Kryptonite, or moonbeams.
‘Thank you!’ he said. ‘A souvenir!’ He left. I gave the fork to a waiter, who wore it on his braces for years afterwards.
Of course you may not be amazed to learn that Geller was doing shows in London that week. Who better to amaze than waiters in one of the West End’s most fertile gardens of pre-theatre-goers and the easily entertained? I knew a cunning nightclub manager who would ride in taxis, talk up the club, tip big, and give free passes to the cabbie. These people are key advertisers, and I suppose so were we. I imagine many customers heard that Big Chief Uri had parked his wagon in town with much big medicine.
This story illustrates how we can be misled from reason and the experience of our senses. I do not for one second believe that aliens from space travelled light years across the cosmos to impart a lonely Israeli boy with absolute mastery over the architecture of the tines of tableware. I don’t believe that the fork bent, or appeared to bend in any way other than the perfectly rational. I am well aware that he is a shuffling huckster, albeit a harmless one. But in that one odd moment, the unavailability of a sensible explanation left one with the palpable sense that something mysterious and inexplicable had occurred. In short, in the absence of a concrete explanation, the mind raced to alternatives more fantastic.
Which is exactly how I feel whenever I read educational research that suggests something idiotic, like all children learn better in horseshoes or hedgehogs or whatever. If someone tells you they can bend spoons in the classroom, ask them what other gifts the aliens gave them.

Son of Brain Gym: Dancing to Nursery Rhymes Boosts A-Levels or something.

In other news: thinking burns calories.

Remember Brain Gym? It was a now-discredited theory that pressing your brain buttons and doing warm ups would somehow improve the cognitive development of your learning conversation, or something similarly moronic. It would be laughable, except that a sizeable purse of public money was spent promoting this ridiculous snake-oil. You know, money that could go to orphans and homeless people and that. I should know, because I was one of the recipients- I was a cultish recruit on the now-defunct Fast Track program (motto: Be the inspiration- from the classroom to the staffroom, which should give you some kind of idea how much we were hated), sort of a predecessor to Teach First. They threw money at us, really chucked it as hard as they could. One of the now-unimaginable training bonuses was a three day residential where we learned NLP (another dubious bag of serpents and spanners) where we were taught the uncertain joys of Brain Gym, and had it recommended for dissemination in the real world.

Fortunately the stake has been fairly firmly planted in the heart of the Brain Gym vampire, especially after Ben Goldacre’s famous assault on it in Bad Science. But not before thousands of schools had wasted their time, and most importantly that of the students, on pointless, pointy-headed miracle crystal exercises that made extraordinary claims to efficacy but without concomitant extraordinary evidence. Any efforts accrued from Brain Gym could be replicated from giving your pupils a break every now and then and getting them to stretch their legs a bit. Which, you know, people do anyway, unless you treat your students like laboratory beagles (and even they get very long fag breaks).

And THAT’S what it’s all about.

But the dead do not stay in their graves; they rise, they reek, and roam the edu-sphere, looking for new necks, fresh blood and brains. Fans of feeling sad and slightly intellectually superior weren’t disappointed this week if they read the news that ‘Moving to rhyme may boost pupil results.’ What the research appears to be telling us this week is that doing exercises set to nursery rhymes helps children to develop. I AM NOT MAKING THIS UP I READ IT ON THE BBC WEBSITE THEY DO NOT F*CK AROUND:

The Primary Movement project involves getting nine-year-olds to do set exercises to nursery rhymes and will be tested in 40 schools in north-east England. The exercises mimic the earliest reflexes made by babies and foetuses. The theory is that children can be held back if such reflexes persist. Trisha Saul from the Primary Movement project said: “Some of the songs and the nursery rhymes will be familiar, it’s the movements that are different. “These are designed to replicate movements the foetus makes in the womb and the baby makes in the first six months of their life.”

 Now, why on earth would anyone think that this was a thing that actually existed, rather than simply being a whimsical daydream of a theory? Because the ubiquitous research suggested it, in a study by the Queen’s University Belfast in 2000. Blimey, they’ve taken their time to get round to doing anything, haven’t they? Or maybe it’s a testimony to the distance back anyone had to dig in order to get any academic support for this latest foray into the desperate world of educational fairy tale research. Maybe not.

 The small scale little known research project found that children who carried out systematic physical exercises for a year gained 15-20 months progress in reading compared to a control group which did not do the exercises.

‘Small scale’. ‘Little Known’. It’s not looking good, is it? They could have said ‘obscure’, but I think the Beeb draw a line somewhere. Trisha Saul, from the Primary Movement Project, said this:

“It’s a bit like a caterpillar turning into a butterfly, but the butterfly still has bits of the caterpillar attached to it.”

That’s EXACTLY what it’s like. What?

‘I am a confident, independent learner.’

I had a look a Primary Movement’s website (the body coordinating the project, which is taking place in 40 schools in England and Wales). It’s far from illuminating, although it links to a sole credit- a press release from Queen’s University, Belfast (oddly enough from 2006, and the article published by the no-doubt beyond beyond reproach Journal of Research in Special Educational Needs– available online only, of course- was in the November 2005 issue, so I don’t know where 2000 comes from).

What it DOES offer is a number of courses that you can apply to take. There isn’t a price list on it. I’m guessing it isn’t free, for either the Foundation or Advanced Level certification. The website advises that as a parent, you should check if your local teacher is trained properly in the method, and suddenly it’s getting a bit mystical and Alexander techniquey, and only the elect are chosen etc.

There is an interesting name that comes up again: Dr Martin McPhillips, who developed the Primary Movement program, and also appears as the author of five out of the seven published papers supporting the work of the PMP so boy, he’s busy. As far as I can see, the research produced by the PMP seems to focus on children with SEN. I might be wrong, of course.

‘The Primary Movement programme developed at Queen’s University, Belfast has been shown to have a significant impact on reducing reflex persistence. It has been evaluated in a number of formal studies that have been published in peer-reviewed scientific journals.’

In a school-based study of children in their first year at primary school, it was found that the Primary Movement programme had a significant effect on the development of fine motor control(9). In another large, school-based study, involving more than one thousand children, it was found that the Primary Movement programme had a significant effect on ATNR persistence. This led to improved academic attainments in reading, spelling and mathematics

‘I endorse brain gym. And anything else.’

That last sentence interests me, because that’s where programs like this intersect with my work as a teacher. The PMP is, I’m sure, beyond reproach, has impeccable academic credentials, and works solely to promote the well being of children. Its authors and board are undoubtedly motivated by nothing but the noblest of motives. That last sentence is quite a claim. I wonder how they reached that conclusion?

My concern is that it is far from clear that instigating a program, however well endorsed, of physical exercises has anything like a substantial effect on a child’s learning ability, and if it does, can be replicated on anything more than laboratory conditions, on all, or even merely most children. And that it is far from clear that such a program has any significant difference from any other program of simple physical exercises. That the suggested increases in learning can be accounted for solely by reference to the exercise program, and can’t be accounted for by other means, such as the children and the teachers feeling that there should be some kind of benefit. Maybe, maybe, maybe. That’s the problem with this kind of research.

The problem remains with ALL forms of educational research; controls aren’t real controls; exact conditions can’t be replicated and tested against. High causal density in human interactions means that causal relationships can rarely, if ever, be inferred from any pool of data, and researcher bias becomes overwhelming in both the design, execution and interpretation of any such project.

Meanwhile, in austerity UK, trials like this receive funding.

Social science. It’s not a real science, is it?

Teacher Voice.

I believe that children are the future. So; no change, then. Brain Gym, Bad Science, and the generational story of stupid.

Excellent article in today’s Guardian from the excellent Ben Goldacre, who writes (again) about the car-crash of stupid that is Brain Gym, and the fact that, despite its reliance on the kind of science normally found in the average Green Lantern comic, it persists in being adopted by schools that really should know better. There’s nothing more I can add about Brain Gym, other than to say that anyone who still believes it has any utility beyond keeping credulous writers in employment deserves to be placed in  a corner wearing a conical hat.

But there’s a point that deserves examination: Goldacre’s optimism that, despite the persistence of- often adult- participants in the kind of moonshine and snake oil that so entrances educational experts, the future holds a possible golden age of intelligence. Why? Because as he puts it:

‘Information is more easily accessible now than ever before, and smart, motivated people can sidestep traditional routes to obtain knowledge and disseminate it’

Bouchard: You BASTARD.

Information certainly is a lot more available these days, oh boy it is. I remember a GCSE (or equivalent thereof) homework where we were invited to either fill in a work sheet, or research a biography of Pierre François Bouchard, the engineer who discovered the Rosetta Stone. Being somewhat friendless and socially gauche, I of course chose the research project, intuitively apprehending its innate complexity, challenge, and geek points. The family picture encyclopaedia was sadly not up to the task; a trip to my local library revealed nothing except for the painful lack of diversity in the Scottish public lending establishment. A further venture to the grand Mitchell Library in the city centre was more helpful. I handed the report in with pride, only to discover that I had carefully recited the life history of another Pierre Bouchard who, unhelpfully, was a contemporary in time and space. Some stories don’t have happy endings.

The point is, that with a few stylish finger flourishes, Dame Google can furnish me with more information about the intrepid French tomb raider than I could possibly digest in a month. Instant, endless raw data is now an axiomatic principle of our daily lives. Pub arguments about statements of fact now last only as long as the time it takes to wave an Iphone. Pub Quiz Masters have to emit EMP pulses just to level the playing field; that, or allow everyone an identical phone and speed recite the questions, testing participants on digital dexterity and search-term efficiency instead of scrappy long-term memory and retention of trivia.

The homework task I described would now perfectly exemplify the IT revolution we have experienced, and surely guarantees Mr Goldacre’s claim that even children- or perhaps especially so- will be the inheritors of a new age of intellectual rigour and penetration; that armed with the facts, reason and empirical scepticism, New Age rhetoric, Religious cant, and superstition will evaporate like fog in a November night, replaced by precision, degrees of certainty and cool, unexcitable reason. In fact, children should surely be the drivers of this intellectual revolution, given the alleged proficiency and taste they show for digital manipulation, and the familiarity they evidence with the online adventure.

Brilliant! You look like a moron!

But not so, not so; if only it were. This reporter is at Ground Zero on Planet Earth, bringing you the latest, and from where I’m standing it isn’t pretty. It pains me to say that, while many children do conform to this description as brave and rigorous denizens of an information superspace, many do not. In fact, you could even say that many of them are just as naive as their slightly older counterparts. They hold beliefs, prejudices and fancies that are just as fanciful, unsubstantiated and unsupported as anything that their generational predecessors could conceive.

For example: t a recent conference for students at a school I was visiting, I showed a group of approximately 100 sixth formers a picture of the 1969 Moon Landing, above which I had written ‘Are we being fooled?’ Now bearing in mind that I had already announced to the audience that this session would be about conspiracy theories, and how most of them were totally unfounded, you might expect a high level of scepticism. Not a bit of it. When I asked the audience to put their hands up if they thought that Neil Armstrong and his fellow possessors of the Right Stuff had really made it to Earth’s little sister, only around a third of the group did so. OK, I reasoned; participant anxiety. So I rephrased the question in the negative: who thought they hadnt really gone, and that the Moon Landings were faked?

Around two-thirds of the group launched their own Apollos into the air, give or take. In my mind, I could hear the Ancient Gods of mysticism laughing at the pretender Reason. When I asked them why they believed such an apparently transparent piece of guano, the usual answers came out: the shadows from the lander; the motion of the flag, and so on and so on. All the usual, cranky suspects. I can’t be bothered to retread the well worn path of dispelling this crap, but the answers are out there- you know, on the Internet. I spent around half an hour blowing each conspiracy to pieces, showing them sources, experts, references.

By the end of the session, I asked them to repeat the show of hands. With a trembling lip I saw that the ratio was now an even 50:50. A victory for rationality? I didn’t think so. Even when I questioned them later, the nays still reverted to their default prejudices and positions: they ‘didn’t trust’ the official story; there was ‘something in’ the conspiracies. I was confused. Had I not explained the perfectly simple refutations of these quack ideas? No; they certainly understood.

I realised what was happening. The conspiracy version of events suited their emotional and cultural appetites; the idea that there is ‘something going on’ behind the scenes of history; that authority narratives are unreliable and manipulative; that doubting the official record enables the powerless to perceive themselves as something powerful; prophets of the truth, discerners of deception, and holders of wisdom. There was, it seemed, at least two worlds: the world as it seemed, and the world as we would prefer to imagine it….even if that world itself contains unpalatable truths, such as the idea that the leaders of every nation state would conspire to conceal the truth of the Moon Landings, or that billions of tax dollars were therefore being diverted to feed…what? The military-industrial complex?

‘This looks well proof, innit?’

It doesn’t just end at the Moon Landings; it couldn’t. Such an attitude has, in my experience, been the foundation of my experiences of working with this Net Generation, these citizens of cyberspace. I teach RS and philosophy, so I understandably attract the conversations of the mystical and counter-rational. That’s fine, and for the best; the oceans of the mind are not charted solely by reason and measurement. Other compasses are necessary to understand the black box of the human psyche. So I know first hand the way that children and teenagers often wrestle with issues of meaning, value and experience, empirical or otherwise. And I also know how many of them turn to ideas and belief systems that are not merely counter-experiential, but counter-rational.

Like the scores- and I mean scores- of serious young men who sidle up to me after lessons and ask me if I have heard of the Protocols of the Elders of Zion, and would I be able to discuss it with them? (The Protocols, for the sainted uninitiated is a fraudulent crypto-history/ manifesto, allegedly written by the Jewish Puppet Masters, and ‘accidentally’ reveals their global ambitions. It is as vile a forgery as you could hope to read, and it has existed for over one hundred years. Despite its utter annihilation as a fake many decades ago, it still enjoys a furtive and political life wherever anti-Semitism needs a prop. Ghastly). There are students who have enquired if I know anything about the New World Order; about the assassination of Diana by the Royals, of JFK by the CIA, and more recently, the enormous number of students who apparently believe that 9/11 was the work of a diabolic Israeli black-op. All of these, theories as empty and aimless as a barrel going over Niagara Falls, yet somehow, inexplicably resuscitated in the minds of the credulous child.

The anti-empiricism doesn’t stop there. I teach a unit on ‘War and Conflict’ to year 9 pupils; I thought it apposite to teach about the Iraq war which, to me at last, still feels relatively recent. Every class I have taught for the last few years has, for the most part, believed that the war was started because of a) Al Qaeda and  b) 9/11. Most of my students believed that Hussein and Bin Laden were in some kind of Axis of Evil, and that Desert Storm was the direct response to the fall of the Two Towers.

Now I’d like to point out that these are the same children who are practically hard wired into fibre optic cables, so ubiquitous is their online experience. They could barely be more connected were they to have a wireless connection node implanted in their cerebella. But still, they seem as susceptible to the storm fronts of illusion and nonsense as any of their 8-bit forebears; just as swayed by fantasy and information tricks as anyone else. Why is that?

‘…and if you look closely, you can see a pixie.’

The answer lies in this basic truism: being drenched in facts doesn’t make you any smarter; it doesn’t provide the possessor of an iota more discernment or critical faculty. This is the difference between facts and knowledge. Many educational half-wits have hailed the Google generation as having been liberated from the tyranny of knowledge; that in an age of universal IT suffrage, any fact can be conjured from the ether like a magician’s dove, and we should therefore focus on teaching children skills, not facts. If everyone who thought that was reduced to a single person, I’d spanner them. Our data-rich descendants are fallible after all; the cyber natives are just as liable to err as anyone else.

And why on earth wouldn’t they be? They are human, or so I believe, although from a cursory reading of the over-excited froth that is regularly written about them by progressive educationalists, you could be forgiven for thinking they had developed an X-gene that made them the perfected inheritors of a Golden, Utopian tomorrow. I have some news: they’re just like us. They ARE us. If a student wants to find out if the world really is controlled by a race of Reptilian overlords, or a more mundane segment of the Primate clan, it isn’t as simple as just clicking on a link. Because more often that not, the link will take you a million miles away from anything resembling ‘life experienced’, and into ‘life imagined’, realms of fairy tale so fabulous that they resemble a fever-dream. Try it. Search Google for Who killed Diana? and see what kind of balance you find. May God have mercy on our souls.

Searching for the truth involves more than just asking a question: it involves asking the right questions. Not all questions will lead you to the real answers, or as close to the real answer as our epistemological faculties will permit. Asking ‘When will Satan end the world?’ won’t get you an inch closer to the veracity of that claim’s assumptions, not on Bing anyway.

The ability to tease apart speculation from cautious theory is a skill that has enjoyed varying respectability as the centuries have progressed. Some reliance on the evidence of our senses has always been with us, otherwise we wouldn’t have made it this far. David Hume famously exploded the irrationality, the ‘sophistry and illusion’ as he described it, of believing in anything that could not be experienced or rationally inferred. The growth of the scientific method has transformed our world, physically at least. But how much of it has penetrated the methodology of our everyday analysis?

I, too have attended training courses on Brain Gym (sponsored and paid for, I hasten to add, by the Fast Track, a by-now deceased arm of the DfE’s previous attempt to invite the brightest and best into education. Alas, they got me) and believe me, the first time around, it sounds perfectly plausible. Who’s got the time to go away and dig up the data that reveals it to be a load of shit? Besides, the DfE were paying for it, so surely they must have checked it out, right?

We’re not swimming in information any more. We’re drowning. Children and adults alike are drenched in a deluge of data that takes time, effort and perspicacity to distinguish the dreck from the good stuff. There will always be people patient and stubborn enough to want to get to the bottom of things, and with the advent of the Internet, the resources to do so are now available to more people- children included- to do so. The problem is, not many of them necessarily will bother. We’re lazy; we’re busy; we’re just too damned indifferent to the truth.

And it’s one of the reasons that an enormous intellectual cavern exists in educational theory, and why the charlatans, the hucksters and the spivs have moved in to fill it with empty theory and card tricks. the battle, I might point out, goes on…

I’m off now to pay tribute to my Scaly overlords, and see if the Internet can tell me how to build a tinfoil Fascinator to stop the CIA controlling my brain. If I start blogging about Personalised Learning Styles, you’ll know they got me, in which case put a diamond bullet through my heart and head for the Vatican. End of Transmission.

British Children ‘bad losers’, claims survey by people who want to sell you something.

‘Leave me! Save yourself!’

An article on today’s BBC Education portal highlights the no doubt vertiginous decline in the average school child’s ability to lose gracefully or, indeed, to win gracefully. I wonder how they came up with a sufficiently large data set to arrive at this conclusion, given that school sports days appear to have gone the way of the Daily Sport and vanished from most school’s year planner, on the basis that the very rumour of competition will have less physically able children curled up in a foetal ball of misery and self-loathing akin to a diabetic fit.

But given that they actually found enough children who were allowed to actually compete with one another, the findings seem, on the surface, to be a cause for concern.

‘Two-thirds of parents of eight to 16-year-olds said their children reacted badly when they lost, the poll found.
A further two-thirds of respondents said parents behaved badly when watching children’s matches.
Some 1,008 parents and 1,007 children aged eight to 16 were questioned for the survey by Opinion Matters.’

How awful. And, I’m sure, quite true, quite true. Still, it always gets my whiskers twitching when I see survey findings being posted slavishly as headlines, despite the cowardy-custard get-out of putting it in between two lazy apostrophes to indicate a possible lack of veracity. Try ”David Cameron wears tights,’ an unnamed source claimed last night.’ The writer gets the neon headline, the reader gets mauled by a grisly, unforgettable image, and all is well, except for the reader’s orientation to any actual truth claims. So, how firm is this piece of educational news?

‘After you.’ ‘No, after you.’

Well, it’s been conducted by Opinion Matters. Who they? Well, if you’re Marks and Spencers, or Vitabiotics (‘where nature meets science!’) you’ll know all about them. They’re an ‘independent’ market research company (what does independent even mean in this kind of context? It seems to mean, ‘will work for money’) who, as their website proudly says, ‘make it our priority to aid our clients to generate headlines and create coverage that highlights and reinforces their branding and key messages in media.’ In other words, they conduct research that will get their clients into the press. You’ll have seen this kind of thing many, many times before- have a look at the Vitabiotic link above, where they advise the client that a survey or two- a bit of science- will get lazy journalists writing about the product, even though the survey in question relies on ‘perceptions’ and subjective opinions. So they’re used to dealing with sensitive subjects with impartiality and rigour. So far, so good.  Have a look at the website. How do they sleep? (As Don Draper would say: ‘On a bed of money.’)

And who commissioned the research from these paragons of objective investigation and veracity? Surely not the Marylebone Cricket Club and the Cricket Foundation? It certainly was. One might reasonably ask why such caryatid columns of sportsmanship and integrity are interested in finding out how badly Britain’s youth react to winning and losing. Perhaps they’re just curious.
 Or perhaps not. As we discover buried away in the legendary bowels of the article (which you get a chocolate biscuit if you’re still reading by that point) is the golden nugget that these august bodies have recently started ‘offering sportsmanship lessons to state schools’. So, no financial or commercial interest in this research at all. Still, at least the researchers were independent. Somehow.

The data says ______.

So there we have it: a story based on research commissioned by bodies who have a vested interest in the findings, conducted by people who are delighted to make sure that the findings are found, and published uncritically as fact by a public body funded by the license fee. The sad thing is that I probably have a lot of love for the Cricket Foundation. (I mean, it’s the Cricket Foundation, for God’s sake.) And the idea that children should be more sporting. And the idea that engaging kids with good role models in sport might be a good idea. It’s just as charmless as Liberace to see it all stirred into the same cauldron, sold as fact, and served up as news. I think we all deserve a bit better than that.

Atten-HUT! Troops to Teachers sees battlefield promotions lauded- but is the science solid?

 I enjoyed Panorama tonight; I always do. There’s something so intuitively respectable about the BBC’s venerable investigative magazine that I would default to unqualified admiration even if it were to tell me that spaghetti grew on trees. This week: Troops To Teachers (TTT)- Michael Gove’s drive to inject a bit of military discipline back into classrooms by aggressively recruiting and retraining ex-military servicemen. It apes the Troops to Teachers program in the US, launched 18 years ago after the first Gulf War, and since then it’s seen over 15,000 men and women swap green berets for cardigans with leather patches (or whatever the symbolic equivalent is in America).

If you watched the program you would be forgiven for assuming the the program is an unqualified success; we were treated to the example of Lordswood Boys’ School in England, which entertains no less than 1 in 12 staff from  military backgrounds, which shouldn’t really be a surprise seeing as how the smallish Birmingham comprehensive has an assistant head who used to be in the Infantry, an ex-Sergeant from the Territorials, and a former sergeant major acting as a shooting instructor. Quite. Still, variety is the spice of life, and one thing that schools have to be praised for is diversity of strategies, trying different things, and adapting tactics to meet the needs of the local community. Looking at the prospectus and the Ofsted report, it seems a bit of a success story. Students like Hakeem Nawas spoke proudly of how it had transformed his self-esteem and motivation to be involved in Cadet activities, and Neil Macintosh, the aforementioned Assistant Head proposed that ex-military were ‘more resilient…less down-hearted…and more robust.’ As Mandy Rice Davies, said, ‘Well, he would, wouldn’t he.’

Actually I have no issue with this: in fact I admire many of the principles that inspire it. I particularly liked how the servicemen spoke about how they maintained order- they didn’t have to raise their voices, they said. The students agreed. ‘They just look at you,’ one said. I know what he means. Screaming your head off is usually a sign that you’ve blown your stack, and for most kids it’s better than telly. Speak silently, they say, and carry a big stick. I couldn’t agree more. Who do you respect more- the small dog with the big bark, or the silent dog with the claw hammer behind his back? Exactly.

Troops to Hogwarts

Then we were off to Huntingdon Middle School, in Virginia’s Newport News City (honestly- I wouldn’t make up a name like that because you wouldn’t believe it), where a clutch of ex-military had taken over their classes like Desert Storm. The story here was the same, it seemed- lines of biddable, disciplined and enthusiastic students queued up to enter classrooms, and we were presented with crocodiles of marching students who were noticeably not selling crack pipes to grandmothers or auditioning for The Wire. Glee, maybe.

Geoff Lloyd, the poster boy for this school’s TTT project spoke proudly about bringing ‘discipline into an undisciplined world,’ and frankly, I couldn’t agree more. His robust, direct attitude to being a responsible adult in a classroom full of students who need clear boundaries and someone they can rely on was more inspirational than a dozen Dead Poets’ Societies or Dangerous Minds. I would put him on my fictional Heroes of Education list, but unfortunately he’s a real person, so he’ll have to content himself with a notional award instead.

So far, so good. As I say, I actually applaud many of the aims of this program. I think that what many of the ex-servicemen said made perfect sense- courage, responsibility, discipline and carrying your own water. Amen to that, brother.

And then- with the inevitability of the Sun rising- came the research. Because that’s what we do whenever we want to justify something: we wheel out the academics who biddably endorse whatever is being flogged to us. And that’s when it got interesting for me. William Owings of the Old Dominion University sat in an agreeable, respectable setting and enthusiastically waved the flag for the TTT program, his eyes twinkling as he did so. He twinkled a lot. ‘Ex-military stay in the profession twice as long as non-servicemen,’ we were told. ‘Troops in the T2T program outscore all other teachers,’ it was said. Owings also provided my favourite quote of the show- T2T had provided a ‘stellar performance,’ he said. Twinkle, twinkle.

Now that didn’t strike me as the careful, cool, neutral perspective of the scientist, I thought. And as soon as someone starts to mention educational research, my spider sense starts to tingle, and frankly I start to sweat a bit. Because, as regulars to this blog will be painfully aware, I’m allergic to the way that some educational research is used to hustle strategies and big ideas that are composed, it is eventually seen, of equal parts moonshine and optimism. As a teacher of some years, I’ve been making a list of the Initiatives and Great Ideas that the hucksters of education try to flog us, and my hackles start to mambo whenever someone calls along and says, ‘Hey, you guys! I have a great new idea for turning schools around! I just need your credit card number and your uncritical commitment…’ I’m just funny like that.

So I did a bit of, rooting around on t’interweb. Just what IS the Old Dominion University, anyway? It sounds awfully grand. And it is, I am sure, a paragon of academic vigour, rigour and propriety, even if its mission statement does say that ‘Our philosophy is simple: Knowledge should be productive. Research-driven solutions that make sound business sense.’ Which isn’t really a philosophy, is it? More of an admission that if something is worth something, it has to be worth money. Ah, it brings a tear to your eye, doesn’t it? .

As I say, I’m sure it has the noblest intentions. It also has an interesting link to the Troops to Teachers program, as its website says: ‘The state office for Virginia TTT is located on the ODU campus.’ That’s the state office. Of course, that doesn’t suggest that the Old Dominion University might be a less than partial witness to the efficiency of the TTT program. I’m just saying, that’s all. Isn’t that a marvellous coincidence, though?

So I did what few civilians have done before: I had a peek at a couple of the papers quoted on their website as showing terrific, supportive data that confirmed the TTT program as a winner, and the ones that William Owings was quoting so freely on Panorama. You can find two of them here and here. They are, as most social science papers are, a thrill a minute, and I heartily recommend you print them off and read them on the way to work tomorrow. Unless you drive. Or listen to Coldplay while you read them.

The 2005 study was, broadly speaking, a survey of Teachers who had gone through the program, and of their supervisors. It asked if they felt that they had been appropriately trained to approved standards. It also asked supervisors of these teachers if they felt they were as good as, or better than teachers of similar experience who hadn’t come through the program. The answer was strongly in favour in both cases. How many were surveyed? A fair few. Over 2000 teachers and their supervisors were sent surveys. That’s not a bad study by any standards. Except that the response rate was 65%. We don’t know why the other third didn’t reply. We don’t know what attempts were made to convert those no-shows. We don’t know on what basis the surveys were sent. We can probably assume that surveys weren’t sent, or at least answered, by teachers who had dropped out of the program.

And it’s details like that, that make this kind of research so hard to value meaningfully. Big numbers are good, but without transparency about who answered, what their motives were, and show inaccuracies were avoided, the purity and reliability of this kind of data is always going to be hard to measure, let alone accept. I’m certainly not impugning William Owings, or any of his co-writers, but these are substantial, significant impediments to the development of social scientific research credibility.

Another problem is that this paper relies on perceptions- ‘how well do you feel …’ questions. These questions fall short, IMO of the clinical precision and neutrality of the genuinely inquisitive, and stray into the territory of market research. When did you stop beating your wife? Who’s to say that the TTT candidates were actually trained properly? What’s to prevent the supervisors betraying their own inclinations, preferences and prejudices through their own opinions. Nothing. Nothing at all. This isn’t the same as measuring the temperature at which mercury boils- it’s like interviewing a series of marathon runners at the finishing line and asking if they feel out of breath.

The paper does acknowledge some of this. Actually, it seems to acknowledge all of this:

‘the study does not provide evidence of T3’s self-reported or actual teaching behaviours. Neither does it provide empirical observations of school administrators watching T3s’ actual teaching behaviours. Nor does it provide evidence of students’ learning gains as a result of working for a period of defined time with T3s as compared with other teachers of similar experience. Further study of the actual teaching practices from T3 self-report or assessment of their students’ measured achievement, although very complex and difficult studies to undertake, would provide important information about T3s’ quality as well as feedback about how to strengthen T3 preparation.’

In other words, we know it’s all just opinion and self analysis. But we don’t think it’s a problem. Of course, opinion and subjective experience have a place in analysis; but it’s not the same place as objective, viewer-independent data. It doesn’t prove anything more than the people who responded felt the way they felt. It’s not corroboration that these teachers are better: it is what it is.

The other paper I looked at, from 2010 (and also by our hero from Panorama), focused on TTT candidates who went on to become Principals. This time it was 107 subjects; ah, boo, much smaller. Their supervisors (I didn’t even know Heads had supervisors) overwhelmingly (90% plus) said that they thought such principals were better on a variety of scales than similar, non TTT Principals. Yes, you may also find it unsurprising that supervisors, who I assume are involved in the selection and support of these principals, overwhelmingly thought that they were doing a jolly good job, and hadn’t they made excellent decisions hiring them? Again, we don’t know the conversion rate, the response rate etc.. I’m sure it was fabulous, given that 107 is a very small number. Still, the data comes out rather well, doesn’t it?

So is there nothing concrete at all to support the view that TTT candidates have a, if you will, tactical advantage over their civilian counterparts? Not a bit of it. Here it is:

‘In a 2008 Florida study comparing measured academic achievement of elementary,
middle, and high school students taught by TTTs, results indicate that compared to all
teachers, students served by Troops teachers performed about equally well in Reading and
achieved a small but statistically significant advantage in Mathematics. In comparisons
where each Troop teacher was individually matched to another teacher, teaching the same
subject in the same school, with approximately the same amount of teaching experience,
students served by Troops teachers achieved substantially and statistically significantly
higher in both Reading and Mathematics (Nunnery,, 2008; Nunnery, et. al., 2009).’

Call me a gutless limey cynic, but ‘equally well in Reading’ and ‘a small but statistically significant advantage’ in Maths doesn’t exactly strike me as cause to start popping the champagne for the cause yet. Incidentally, the Nunnery paper mentioned above by Owings is co-written by…..yes, William Owings. And it wasn’t published in an academic journal, but, as the report says, ‘submitted to ‘Educational Administration Quarterly
October 2008′. I can submit a poem written on bog paper to the Sunday Times. Does that mean I can say it was printed?   Have a look at the front page. It’s got a lovely ‘Troops to Teachers’ logo all over the front. I’m don’t have a Ph.D. in this exact subject, but I suspect that means they might have something to do with the report….

(I stopped reading it at that point, because I don’t believe in an afterlife, and I value every precious minute I possess.)

In fact, so do the previous two papers I mentioned, both of which are prefaced by the sentence, ‘A Report Prepared for Mike Melo, Director, Virginia Office of Troops to Teachers,’ and ‘Report to Dr. William McAleer, Executive Director, Troops to Teachers, Pensacola, Florida.’ So all of the reports mentioned were written for (can I presume commissioned?) the TTT itself. Hey, waitaminute…..

It’s not that I’m against the idea of ex-military training for schools: good luck to ’em, I say. And I think that there might be something in the idea that men and women who have experience with leadership, developing self-discipline and oiling rifles might have something useful to teach children (sniping, for instance). But it doesn’t do anyone any good to use research like this that seeks to support proposals with empirical claims that can at the very least be contested as meaningful or verifiable in any real sense. Michael Gove needs to look elsewhere for better arguments, and maybe we might start to take research based policy more seriously.

What’s red and green and pointless? The endless, anxious debate about the colour of marking.

What’s got two thumbs and couldn’t give a damn what colour his marking pen was?

*points two thumbs at chest*

The meme de jour of horror flicks is to have the  final frame hinting at the imminent return of the hellish antagonist- see: Carrie; Saw; Freddy; Jason, ad nauseum. Well here’s another teaching myth that I thought had been staked a long time ago, but apparently keeps rising from the grave with  the certainty of sunrise. Does it matter in what colour you mark students’ books?
No it doesn’t  
I only mention this because someone emailed me this question recently, and I had to rub my eyes and pinch myself (not easy) to check if I was dreaming. Are people still asking this? Apparently, yes. Dracula has returned. So it’s time to dip my crossbow bolts in holy water and bless my silver candelabra, and get ready to knock the brains out of this one, although believe me, it won’t take much.

When I started teaching, this was received wisdom; it was dogma; it was part of the catechisms of the Church of Progressive Teaching: do not mark in red ink, I was told. When I asked why, I was solemnly told that it was ‘bad’. Which is great, because for a minute I thought they were going to be vague about it. I doubted it then, because it just seemed so counter-intuitive. How could it matter in any real sense what colour I used? But, like many axioms absorbed in the infancy of one’s education, I complied, and dutifully stocked up on soothing, somehow more supportive shades of emerald. I wondered then which shade in particular was supposed to have the best effect? I think more work needs to be done.
But let’s settle this. There is no research whatsoever to support the view that marking should be done in green ink, purple, vermilion, or a thousand other shades of the spectrum, as a preference to red. Let me repeat that: there is none. So why do so many people think it’s true? I’ve investigated, so you don’t have to. Jus’ doing my job, ma’am.
Well, what there has been is an enormous amount of psychological investigation into colour preferences in an enormous number of groups. There’s also been a lot of studies that have looked into the symbolic associations that people have towards certain colours. This is nothing new: people have always attached meaning to just about everything: black has connotations of mystery, magic, fear, fascism; blue connotes to calm, the ocean, etc. You can add your own. There are recurring themes that reflect the cultural expectations of the group investigated. Within each group there will be understandable variations ascribed to any particular colour- I find Sunflower Yellow irritating (mainly because I associate it with one of the companies I used to work for), whereas I’m told by SENCOs that autistic children apparently fall into soporific calm at the sight of it. I have no idea.
This kind of thing can be studied quite easily- get a sample, ask them what they associate with each colour, collate the data. Or perhaps be even more clever and do it a bit more ‘blind’- don’t tell them what you’re investigating, and show them pictures of people wearing different shades and hues and ask them to express their thoughts towards them, that sort of thing. The point is that this kind of data is easy to gather, and assuredly marketing people have been doing this since dinosaurs walked the Earth and they wanted to know what colour of loincloth sold best to AB1 Hunter/ Gatherers.
Of course, the idea that the colour of your marking pen matters goes beyond even this: the idea seems to be that people associate the colour red with aggression, threat, danger and negativity. People- seemingly quite sensible ones- have expressed the opinion that they remember  their own books being ‘covered with red ink,’ in a process that they describe as traumatic. Ergo, we should stop using red ink and get busy with shades more arborial.
Show me the research
Well, quite. So what research is there to support this? I’m happy to report that the answer is ‘bugger all’. Curious, I decided to do a bit of digging, and I was amazed to see that nothing on paper could explicitly support this theory at all. In fact, even the most recent research seems to struggle to prop it up, or barely offers any insight into the question except in the most tangential way. To give one example,  a 2010 study in the European Journal of Social Psychology by Rutchick, Slepian (of the gorgeously named Tufts University) and Ferris offers the cutting edge of research into this area, and frankly, it’s a hoot. It shows, or attempts to, that using red ink can prime markers (not students) to tackle papers more aggressively, more critically. Its central premise is that just seeing red (ah, such a loaded term) is enough to make markers…well, see red. Those using red pens in the research seemed to grade papers lower, and notice a higher level of mistakes than the other, non-red-ink group. They also seemed to indicate that, when confronted with word stems to complete, they usually went with more negative, aggressive ones over the altogether more harmonious, Scandinavian alternatives.Let me give you some examples; see how you do. Finish these words:
MIN_ _.
WRO_ _.
Now, I don’t know about you, but I went straight for ‘fail’, ‘minus’, ‘flunk’ and ‘wrong’. And so did more of the red ink group. It’s results like these that convinced the researchers that they were on to something with their red-ink bad hypothesis. The blueys, incidentally, had higher proportions of ‘fair’, ‘minty’, ‘clunk’ and ‘wrote’ apparently. You can’t argue with science.
Except that you can certainly argue with this science. For a start, what’s to say that the ‘negative’ connoting words weren’t simply more common idiom? “Minty’ isn’t exactly something that trips off my tongue regularly, but then, I don’t sell toothpaste. And what’s to say that there weren’t fewer toothpaste salespeople (for example) in this group than in the other? It’s these kind of uncontrolled variables that knock the guts out of a piece of research, because without reassurances that one group isn’t unfairly weighted with certain types of respondents there’s no way of knowing if your analysis is statistically significant.
What about the other part of the research- that the red inkers were more stringent/ harsher/ more negative in the number of mistakes they corrected? Again, we need to know if there was any meaningful way of making sure that one group wasn’t unfairly (or should that be unfailly?) loaded with pedants, grammar Nazis and careful essayists. I checked the paper to see how the group was selected. It says, and I quote:
‘The current findings are qualified by additional limitations, primarily concerning the participants in the studies. Due to time constraints associated with conducting the experiments in a realistic setting, little was known about the participants beyond their presence in the university environment; their age, ethnic background, level of education, and other factors were not assessed. Several of these individual differences, such as verbal ability, educational background, and field of study, could influence participants’ ability to detect errors, their propensity to mark them, and the harshness with which they make evaluations.’
You don’t say. So in essence, we have a bunch of people marking with red pens, and a bunch of people marking with blue pens, but we don’t know if there are any factors in each group that could produce the results that we found. But we think it’s got something to do with the pens. In fact, it’s not just the pens, it’s the fact that the ‘we propose that the red pen effect is driven by increased accessibility of the concepts of errors, poor performance, and evaluative harshness.’
Or, if I can reformulate that again, ‘we don’t know why, out of all the possible differences between the two groups, there should be a statistical difference in such things as number of errors recorded etc, but we’ve decided it’s the colour of the pen ink, because that’s what we’re investigating.’ Ladies and gentlemen, it’s research like this that makes your line manager tell you with an air of authority to ditch all your crimson ballpoints and go green in the classroom
So what do the authors think of this inability to account for the variations in the participants? ‘However, these uncontrolled differences should manifest as random variability, and thus make it more difficult to detect the effects we report.’ In other words, ‘We don’t know if they have any factors that could affect the experiment, but because we don’t know we reckon it should all work out jess’ fine, than ‘ee.’ Give me strength.
I often blow pretty hard about social science, for reasons just like this: although I don’t necessarily believe that all social science should be circumscribed by the methodology of the natural sciences, I do think that whenever it makes a specific empirical claim about the way people think and routinely act, that there should be some kind of empirical method to show that this is the case, and that the experiment can be meaningfully reproduced in other situations, tested and confirmed. If all you want your paper to do is to start a conversation, or to add to a debate, then by all means keep it anecdotal, or unrepresentative, or subjective. But when you start trying to prove something predictive, or worse, start telling others to change their behaviour on the basis of your research, then you need to turn up to the fancy dress party with more than a fig leaf, saying you’ve come as Adam.
Surely there’s more evidence?
Hang on, I hear you say, is that all the evidence there is? Well, yes, at least on this specific topic. Oh to be sure there are many studies that show that certain colours have certain connotations amongst certain people. And there are many other studies that show that certain colours can ‘prime’ us to respond in a variety of unconscious ways. But nothing- and I believe I’m throwing my perfumed gauntlet out here- NOTHING to suggest any clear link between the use of the red pen and…well, what is it the critics say is happening anyway? For one thing, the claims they make are maddeningly vague: red ink is ‘negative’; it’s ‘discouraging’. Oh really? And how would you even go about measuring that kind of thing? How on Earth could you control for it? Do we ask 1000 children aged 14 to describe their feelings towards corrections in their books? And another 1000 to say how they would have felt if it had been in mauve rather than vermilion? *slaps forehead at the boneheaded nonsense of it all*
But hang on: our heroes have something to say about the claimed ‘effect’ of red ink on students:
‘To our knowledge, this demoralization has not been empirically demonstrated, and would be an important complement to the current findings.’
That might just be the understatement of the red-inked century. Let me just repeat that:’ no empirical evidence’. No studies. None. No proof whatsoever other than a gut feeling in educators that when kids look at a book covered in red ink corrections, they get a funny feeling in their tummies, and perhaps it’s the red ink that’s to blame? As many people have commented, it would seem perhaps more intuitively likely that people brought negative associations to the fact that they were corrections rather than the fact that it was red. After all, red can have millions of associations: danger, certainly- the red of a wound – but also thrills, passion, romance-  the blood racing in your veins because you’re alive, for example.
Finally I would like to make one point: maybe I WANT kids to feel a little bit alert when they see red on their books? Maybe the connotations aren’t exclusively negative, but rather represent a state of heightened alertness corresponding to increased attention paid to corrections. Maybe, just maybe. The point is that, as Ben Goldacre famously repeats, ‘things are a bit more complex than that.’
And I sincerely hope that I haven’t offended anyone by typing in black ink. Click here for a more soothing draft in magenta.
Finally, I think what particularly offends me about this subject  is that, were it relating to an aspirin, or a new technique for removing surgical stitches, it would be subjected to an enormous level of scrutiny before it could be released, as it were, into the wild. Not so in the field of educational science, where mutants, hybrids and sickly, runtish ideas are set free to breed and settle where they will. If this was an aspirin, it would have been laughed at; in education, it’s adopted as a mantra. It would be laughable were it not so ubiquitous, or so representative of the slavish manner in which teachers are expected to assume a new position, no matter how servile, or follow any fashion, no matter how impractical, if their masters demand it.
I haven’t been able to track down exactly when this idea escaped the laboratory from which it spawned, or how, or by whom. I can find reports of schools in the UK (primaries at first, then secondaries, Australia (Queensland) and America (loads of places) adopting the abolition of scarlet scribing dating back to 2003, in this report from the BBC. To quote:
‘Penny Penn-Howard, head of school improvement for Sandwell Council, said: “The colour of the pen used for marking is not greatly significant except that the red pen has negative connotations and can be seen as a negative approach to improving pupils’ work. Therefore, it is quite legitimate for a school to have a consistent policy that it uses a different colour.”‘
Which is another example of why I’ll be glad to see the back of some people who claim to be employed to ‘improve schools’.  If it isn’t significant, why have a policy? How does red connote ‘negatively’ etc? Does that mean that tomato sauce and Christmas have negative associations? Blimey, someone better call the head of marketing at Coca Cola and tell them they’ve been getting it all wrong. Penny Penn-Howard has the answers.

My Stella Challenge to the red-ink flat-earthers

I’m not James Randi, and I don’t have a million dollars, but I bet anyone a pint that they can’t produce meaningful research that shows that the colour of the pen has any significant effect at all. I suspect my pint jar is safe from the toffee hammer for some time. In the meantime, I’ll be choosing to mark in red as much as humanly possible, more in spite of the apparently still current dogma than for any other reason. Mind you, it stands out nicely against all the black ink of my student’s work.  
Plus of course I tend to write in human blood. I like the connotations. It shows I care.