The Hawthorne effect - confounder or tool?

There was a big splash in the medical press this week about the WHO new surgical checklist, which has apparently ‘been shown to significantly reduce post-surgical morbidity and mortality’, leading to the National Patient Safety Agency (NPSA) planning to roll out the checklist across the UK, and a patient advocate complaining that doing this in all 500-odd of the NHS’s hospitals over 12 months is too slow!

My critical eye was caught by the phrase ‘pilot studies have shown…” and I immediately wondered about a well-known confounder of social research called the Hawthorne effect.

This can be a particular problem in evaluating educational interventions, but may be less familiar to people used to medical interventions, which tend to favour the double-blind randomised controlled trial, ideally with physically defined outcomes.

The Hawthorne effect occurs when the objects of a study are themselves capable of understanding that they are part of a study, and that therefore something is expected of them.

In the classic experiments in the Hawthorne Works (a huge electrical factory outside Chicago) in the late 1920’s, Landsberger was looking at the effects of work conditions on manufacturing output. He tried first improving the lighting on the shop floor, and found that productivity improved. However, when he lowered the light levels back to their original settings, he found that productivity improved again! The intervention which he thought was effective (the lighting levels) was in fact NOT responsible for the change in performance. What had the major effect seemed to be the knowledge that the workers were being observed, and that something was expected of them!

Now the authors of the surgical checklist pilot study, do acknowledge the possibility of a Hawthorne effect in the results they obtained. (Full text) If this is an explanation for much of the observed improvement in patient safety, then we would expect it to wear off as the novelty of the checklist wears off (perhaps as the checklist is finally rolled out across the NHS).

Just like the placebo effect can confound medical research, in the world of educational research the Hawthorne effect can be a bug bear. 

However, it is also an effect which you can use to positive effect as a study tool. If you make a change in your study patterns and set up an observer (which could be yourself) to measure the outcome, it’s likely that the very fact of making a change and studying its effects will produce an improvement in performance (albeit temporary).