‘All Eggs In One Basket’ – The Three Hour Exam

20130211-235331.jpg

Timothy Salthouse is a University of Virginia psychologist. He has conducted extensive research into testing, from intelligence to aptitude tests (from the age of eighteen to over eighty) in the snappily titled: ‘Implications of Within-Person Variability in Cognitive and Neuropsychological Functioning for the Interpretation of Change‘ (Neuropsychology 21, no.6 (2007). Now, why is this relevant to our current British education system and Michael Gove’s proposed changes to our curriculum and assessment at KS4 and beyond? Please let me explain. The research prompts serious reservations about something most teaching professionals know instinctively – putting all our eggs into one exam basket is both reductive and destructive.

Michael Gove has proposed that we should do away with coursework or other internal assessment procedures, except for specific subjects, such as Geography fieldwork or Drama practicals. That leaves subjects like mine, English, looking likely to end up with a summative judgement of a three hour examination. We all have our reservations about the exam system. Like the Heisenberg principle in quantum physics, we know that to precisely test one thing, we must inevitably be less precise with testing others. Therefore our testing system becomes narrower and narrower, to make a judgement on a narrow definition of the ‘progress’ of our students. In our results driven system, the curriculum gets ever condensed to meet the progress measures. All the while, the complexity and wealth of information our students have to deal with in our digital age is not narrowing at all, but growing exponentially! Surely our reservations about an ‘all eggs in one basket’ assessment aren’t just unfounded fears from educators seeking to survive in a judgement laden, punitive system?

Salthouse’s research presents us with really unsettling answers about the accuracy and efficacy of such a crucial and singular ‘all eggs in one basket’ assessment. His research has uncovered that there is a wide degree of variability ‘within the same individual’! That, on different days, people could sit the same test and perform in a vastly different fashion. This clearly raises the issue that any one single measurement provides an insufficient evaluation of a young person. His data showed that ‘the within-person deviation’ in test scores averaged about 50 percent of the between-person deviation for a variety of cognitive tasks. With such a bell curve of performance for individuals, sitting the same test, without specialist revision or preparation, simply on different days, how can we justify an ‘all eggs in one basket’ exam to culminate years of study? How fair is it for students that examinations on a Friday afternoon, for example, may suffer a degree of variability which may make students worse off than other students sitting a different exam board on a different day, with some bad weather? The variables are huge and the stakes are sky-rocket high. Of course, we see punitive attacks on entire schools for deficient performance.

This issue does not take further issues into account, such as the quality of examiners, or lack thereof. There is no professionalisation of examiners and the consistency of exam grading is annually brought into question, particularly for subjects such as English, which have a significant degree of extended interpretation. I could show you some exam papers of my past students which have been marked shockingly badly. Coupled with within-individual variation, such summative judgements become even more questionable. To ignore the breadth of quality internal assessment for such a high-stakes test smacks of ignorance.

In other curriculum and assessment models lauded by Michael Gove, such as the International Baccalaureate Diploma, there is a significant proportion of internal assessments; from portfolio work, to oral recordings and to extended coursework essays, externally moderated. The iGCSE assesses oral recordings for the speaking and listening component. If our politicians are scared of cheating in the system then provide a better model that deals with the gaming (or better still, remove the corrosive competition inherent in league tables with course comparison indicators!), such as using oral recordings; live moderation; draft evidence in essay work, or a portfolio approach. These assessment models may be more expensive, but they will mitigate the risk of the high stakes end of course exam model. Perhaps Gove has these in mind, he is just keeping his curriculum cards close to his chest – I hope so!

As Salthouse puts it: “…the existence of within-person variability complicates the assessment of cognitive and neuropsychological functioning and raises the possibility that single measurements may not be sufficient for precise evaluations of individuals, or for sensitive detection of change.” A bit of a mouthful, but the idea is simple: we simply cannot have a system where one bad day can scupper the life chances of any given young person. That is no model for a system looking to enhance deeper learning and militate against teaching to the test.

The culture of resits is ultimately corrosive to deeper learning. I do not advocate a resit culture, the perverse multiplication of exams, it gains nothing, except perhaps the ample profits for the exam boards! Yet, surely we have advanced beyond the antique paradigm of the ‘all eggs in one basket’ exam. Portfolios, speaking and listening assessments, well structured coursework all have their place in a more holistic approach to assessment. Let it be rigorous – I have no argument with that – but let’s not play roulette with the future of our students.

As Michael Gove concedes on the issue of the EBC qualifications replacing the GCSEs he is still intent on measures such as eliminating internal assessments for academic subjects, and other such narrowing effects upon educational outcomes. He clearly lauds the certain judgements of examinations, when evidence put forward by the likes of Timothy Salthouse calls their consistency and accuracy into question. We must therefore challenge the narrow and reductive proposals and put forward better curriculum and assessment models. We have a moral imperative to ensure that our students have a fit for purpose assessment model that is rounded and fit for the twenty first century.

Advertisements

Tags: , ,

About huntingenglish

I am Director of Learning and Research at Huntington Secondary School, York. I have taught English (including a bit of Media Studies) for over a decade. If it is tragic and gothic, laced with humour and bitter truths then I'll teach it! I have had the great privilege to have written a book, 'Teach Now! Becoming a Great English Teacher', edited by the brilliant Geoff Barton, and I am writing another for Routledge, entitled, 'The Confident Teacher'. I am Project Lead of the RISE (Research-leads Improving Students' Education) Project. An EEF funded randomized controlled trial to evaluate if and how a Research-lead can improve outcomes for students. I am a proud member of the Institute for Effective Education (IEE) Executive Board at York University. I am also a proud member of the ResearchEd Advisory Panel. I write regularly for the TES and Teach Secondary magazine. My first book, 'Teach Now! Becoming A Great English Teacher' was released by Routledge in 2014 and my next book, entitled 'The Confident Teacher' will be out at the end of 2015.

13 responses to “‘All Eggs In One Basket’ – The Three Hour Exam”

  1. headguruteacher says :

    This is an interesting article Alex. The all eggs in one basket scenario is a mistake for sure; the question is: what would be better? Fundamentally we’ve got a system where teacher assessment is not valued enough – and we need to work hard to create assessment processes that will allow that to be reversed. Tougher moderation? My experience of GCSE science coursework has been that is has led to overly formulaic work driven more by the criteria than by learning. Our current A level practical coursework is the reverse – lots of original investigations. There must be a way to transfer that to GCSE. For our iGCSE Physics, I must say it is a relief not to have modules. We can step off the exam treadmill for a while with less time spent on revision and more opportunities to link ideas together across the whole course. Final exam has two staggered papers which seems sensible. There is no ISA which is also a relief because it didn’t really test practical work anyway – practical marks were too easy to get. Each subject will have these nuanced issues particular to them. What we need is a sophisticated analysis of ‘best practice’ assessment of each one – not a crude generic model that the ‘3 hour exam’ suggests.

    • Alex Quigley says :

      Yes – the article was about dismissing the narrowing notion of an exam, without going into the nuanced depth of a replacement model. One key reason being that there does need to be subtly and nuance in each subject area. A think that the key common factor is that we want to elicit deeper thought and learning, with the typical attendant factor of extended writing and/or projects.

      I think the divide between essay based subjects and the practical theory is the essential dividing point. I am inexpert in the Sciences and other more practical subject areas, therefore I would defer to experts like yourself. When I consider the Humanities, the Social Sciences and English, I think a commonality of internal assessment approach can be achieved. When I consider the best models they do often cascade down from KS5.

      I think in English and the Humanities debate and oral responses should be validated by speaking and listening assessments. We could simply follow the IB model for English and record such assessments (the recording equipment is surely now accessible for all); having random samples called by the exam board for rigorous moderation. This is worth 20%, which I deem an appropriate model. I think there has to be extended writing conducted internally to display skills honed over extended periods. The crucial factor is that the internal marking is where the system falls down. We must have a system where, like the IB Diploma, work is completed (with evidence of developing thought – drafts etc.; with independent thought – original, individual titles and bibliographies etc.) and sent off to be marked. The issue being cost. Exam boards are looking to their profit margin. Also, we do not have a professionalised system of markers that are highly respected and who hold to a common standard. I loath useless quangos – so we don’t need a new body – but OFQUAL must do their job!

      Your point about ‘best practice’ is key. Let’s compile the evidence and build a model that embraces consensus to a degree where we recognise fairness. In an ideal world we would eliminate the punitive aspects of the system and have a supportive model, but that seems utopian in the current climate. I do think the profession recognise that the controlled assessment model is flawed, as is the expectation that every teacher will mark in an even balanced way, given the very same teachers will be judged on those self-same outcomes! Internal assessment must align with quality external marking, which is much more robust that the current powderpuff moderation systems (which are of course the cheapest model available!).

  2. David Didau (@LearningSpy) says :

    I take your point Alex – it’s just that the idea of getting rid of the aberration that is controlled assessment fills me with uncontainable glee! Anything has got to be better than the terrible way CA warps teaching! Here’s my post on it: http://learningspy.co.uk/2011/10/28/controlled-assessment-and-why-i-hate-it/
    But, teaching A level for the first time in years I’ve really enjoyed the process of writing coursework for both Literature & Language – amazed at the quality of work produced when students are given TIME.

    Tom, I’ve found that A level coursework is anything but formulaic but yes, that was always a problem at GCSE – but one which could perhaps be solved by a portfolio approach?

    Thanks, David

    • huntingenglish says :

      Whole-heartedly agree about controlled assessments. Yes, at A level the qulaity and structure is much better. For example, the English Language Investigations at A2 demand an independent approach and a thoroughness regarding time taken; choice over topic and a natural differentiation factor. The cold truth is that when we have league tables and floor targets, people will feel pressure to abuse the system. Like Chris states below, we mustn’t throw the baby out with the bath water – but make the process of internal assessments better and more rigorous (I can’t help thinking of Gove when I use that word!).

  3. Chris Chivers (@ChrisChivers2) says :

    Very much enjoyed the article.

    David’s idea of a portfolio approach would go some way to addressing measuring achievement, within a broader remit.

    My worry is the implication that it is one strike and you are out, and will have an impact on students whose learning approach is slower and steadier, but who go to pieces in examss, with last minute crammers perhaps achieving higher than coursework suggest. Which is working more closely to the world of work?

    Teachers and schools will put tests into the scheme, whether formal modules required or not. End of year 12 will have a test. Better internal, with additional marking demand on teachers, or external, with benefits to uni applications?

    My worry is that we are throwing baby out with bath water.
    Best wishes,
    Chris

    • huntingenglish says :

      Completely agree with your baby and bathwater approach. The portfolio approach aligns with what I see as a best fit approach to assessment. The point about what we are preparing them for is also valid. If we do not challenge their knowledge and understanding in speaking and listening assessments how well are we preparing them for their future life?

  4. Ian Lynch says :

    I have shown in a GCSE equivalent qualification accredited by OFSTED and accepted by the DfE for headline points, that the issue is easily resolved through an innovative combination of competence based coursework assessment and a short academic exam. Details are at https://theingots.org/community/node/33025

  5. kristianstillKristian Still says :

    Three hour examinations and the cycle of controlled assessments – there must beca better way. As I tend to teach lower ability groups, I witness first hand the impact of ‘what’s the point Sir, I never going to pass anyway’ exams. As a result, my revision processes have focused as much on building up confidence that they can sit 2+ hours exams as exam prep / up-skilling.

    I would be interested in seeing how exams can be structured to allow all students to demonstrate their abilities rather than just the brightest. For the record I am in support of open book exams Danish style.

  6. magentalemon says :

    Thanks for the insight on this post. Sadly I’ve studied in several countries and the ones where the “all the eggs in one basket” policy was applied, you ended up learning less than in those that didn’t.

    http://redbarcelona.wordpress.com/2013/02/15/education/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: