Oral exams and sloppy thinking on exams

A recent story in the press about a proposal to substitute oral exams for continual assessment in foreign language exams raised a lot of hackles! The trouble with this kind of thing is that people see headlines and opinions, and comment from there, rather than go back to primary sources, and see what is actually being proposed. So I thought it would be useful to see that the original Dearing Report (2007), actually said. And there it was in section 3.22:

We also proposed a new approach to the assessment of speaking and listening, which rightly account for half the marks in the GCSE, on the grounds that the present method is too stressful and too short to be a reliable way of assessing what the candidates can do. It is interesting that when people spoke about the oral test, that however long ago it may have been, it is often remembered as a stressful experience. We therefore proposed that these parts of the examination should be over a period through moderated teacher assessment.

The Dearing report states that oral exams are stressful - well, yes exams are stressful, but the implication here is that they are unnecessarily stressful, and put people off taking language at GCSE and A-level. Secondly it is asserted that they are too short to accurately assess language skills. I haven't read the full report, but from my quick scan, it seems that these assertions are not justified. Research shows that the predictive value of exams rises quite quickly in the first few questions, and that a plateau is quickly reached after which further assessment yields very little new information about the candidate. Are oral exams too short, or do their results correlate well with other measures of linguistic ability? And how do we know that a stressful exam causes people to change their subject choices - it seems plausible, but is it what actually happens?

To my mind the real issue here is sloppy thinking in educational policy, which justifiably wants to encourage more people to study languages, but makes unjustified assertions. This seems to me to post-hoc justification of pre-determined decisions, rather than evidence-based policy making. So in this case the commentary does have something substantive to complain about – but let’s check things out before getting hot under the collars, eh?