Tag Archive | Testing

Effective Revision Strategies

20130407-220657.jpg

There is a lot of cognitive science research that proves what revision strategies work best for embedding information into the long term memory – which is our goal in relation to exam success. Some of it is common sense, but other aspects may surprise you or challenge your thinking.

There are many time-consuming revision strategies that actually fool us into thinking we have embedded the knowledge into our long term memory. For example, simply re-reading texts or notes has been seen to have a low impact with regard to memory retention, especially considering how much time this can take, but students are happy because this is a relatively undemanding task that takes little mental effort and it feels like effective revision. Re-reading ‘Of Mice and Men’ for an English Literature exam doesn’t have the impact we need, especially given how time consuming it is as a revision activity, therefore other, better, strategies should be undertaken. Other edu-myths also cloud effective planning for exam revision. There is an old adage abound in education that: “We learn: 10 percent of what we read; 20 percent of what we hear; 30 percent of what we both see and hear; 50 percent of what we discussed with others; 80 percent of what we experience personally; 95 percent of what we teach to someone else.” This is a myth based on no evidence. It has become perpetuated because it is an easily reductive formula, but it is unfounded. David Didau lances this particularly boil to good effect here. We must go beyond these simplifications and seek answers from more reputable research to judge against our experience.

The following strategies are underpinned by more reputable scientific research and evidence:

Information retrieval over re-reading: It may prove more challenging in the short term, but getting students to try to remember the content of a given topic is more effective than making revision notes based on their original content, textbooks etc. ‘Concept mapping’ is an ideal teaching tool for this (think of its popular branding, image and colour laden brother ‘mind-mapping’!). At the end of each week for example, have students attempt to retrieve the information, without their notes or books. They create a hierarchy of connections that they can attempt to organise conceptually.

Research: http://learninglab.psych.purdue.edu/downloads/2012_Karpicke_CDPS.pdf. Thank you to @websofsubstance whose excellent blog post of retrieval helped me source this research: http://websofsubstance.wordpress.com/2013/04/06/golden-retrievers/

Collaborative retrieval: Typically we associate revision activities and memory as requiring individual focus. Indeed, there is some evidence that group work can inhibit some learning, but there is evidence that students working in groups can have a positive effect, where students work together ‘cross cueing’ the information they are recalling. Put simply, they help one another remember and retrieve aspects of key information they would not have remembered individually. Also, the social nature of working together can create memory cues that help individuals recall well over time. Of course, any errors in retrieval, either individually or collaboratively, need teacher correction.

Research: http://www.cumc.columbia.edu/dept/sergievsky/pdfs/shorttermandlongterm.pdf

‘Spacing’ versus ‘massed’ practice: This finding is common sense really. ‘Spacing‘ is when revising the same information two or three times across a few days improves the likelihood of retaining information in the long term memory (Nuttall, 1999). This may include revising a poem and making connections with another poem, then revisiting the key aspects of that poem in the subsequent lesson, before finally doing a ‘concept map’ at the end of the week to revise the learning from the lessons that week. ‘Massed‘ practice, or ‘cramming‘, can have a good short term effect on memory recall, but it fails in the long term in comparison to ‘spacing’ out revision. There is no exact time or number of days concerning how much ‘spaced’ time should be allocated; however, the research indicted the number of days ‘spacing’ is shorter the nearer the exam. In practical terms, over a half-term, we could revisit a concept after a couple of weeks, but nearer they exam we would cluster a couple more ‘revisions’ of the concept/information.

David Didau has written an excellent blog explaining spacing etc. and the implications for curriculum planning, and what ‘progress’ in learning may look like here.

Research: http://psi.sagepub.com/content/14/1/4.full.pdf?ijkey=Z10jaVH/60XQM&keytype=ref&siteid=sppsi and for an in-depth focus on ‘spacing’: http://uweb.cas.usf.edu/~drohrer/pdfs/Carpenter_et_al_2012EPR.pdf

Using ‘worked examples’: This is the common method of using past exemplars or creating your own through ‘shared writing‘ strategies. It gives students a working template for their revision and reduces obstacles that stops them learning more knowledge. Ideally, teachers should lead model worked examples of exam questions, thereby giving students a clear idea of an excellent answer, before fading back and letting students tackle exam questions independently. Of course, once more, quality feedback is key in this process.

A great blog by Joe Kirby goes into great depth about the ‘why’ of using ‘worked examples’ here.

Research: http://steinhardtapps.es.its.nyu.edu/create/courses/2174/reading/Renkl_et_al_EP.pdf

Regular in-class testing: Drilling answers to tests, under test conditions, can improve both short term and long term memory to boost revision (Roediger et al 2011). Like the retrieval practice of ‘concept mapping’, the very act of retrieval without resources to support proves more memorable than any ‘re-study’ activity. Taking a test can lead to students becoming less confident, therefore quick and accurate feedback is key to making testing highly effective and building confidence. There is research to say that teachers often drastically overestimate what they believe their students to know (Kelly, 1999) so repeated testing is a practical necessity. In terms of learning, there is much research that testing revision material has a positive impact on long term memory in comparison with simply revisiting material.

Another important consideration is that students naturally revise in a ‘massed’ learning style i.e. last minute cramming! It is labelled the ‘procrastination scallop‘ by Jack Michael here. This led to a recommended ‘exam a day’ approach, which forces students to distribute their revision more evenly, rather than just cramming. It may seem excessive, but getting students to do challenging retrieval that informs the teacher what they know and don’t know (and invariably if they have revised or not) regularly, like quizzes etc. could do the job.

Research: http://people.duke.edu/~ab259/pubs/Roediger&Butler(2010).pdf and the ‘exam a day’ research: http://www.teachpsych.com/ebooks/tips2011/I-07-01Leeming2002.pdf

A lot less scientific, but a fun revision strategy that works for many:

– Building a ‘palace of memory’ is a much less scientific way of improving memory recall, but it is apparently thousands of years old, originating with the Greek poet, Simonides of Ceos, in the fifth century BC. See this Guardian article for an excellent example of the method in action: http://www.guardian.co.uk/lifeandstyle/2012/jan/15/memory-palaces-lists

How does this equate to a revision programme?

I am now avoiding revision activities or homework revision tasks that recommend simply revisiting information. I will plan to interleave different topics each week, to create the necessary ‘spacing’ between topics (in my English GCSE class this will mean studying poetry for English Literature at the start of the week, the novel and short stories in the middle of the week, ending the week with English Language revision). I will give regular mini-tests, drilling individual answers, with ‘worked examples’ in the first instance to model a good answer. The feedback on their answers will be timely and regular. I want to undertake weekly retrieval activities that reflect upon what they have learnt that week (combining ‘spacing’ and ‘retrieval’)

It is clear that the process of revision happens inside and outside the classroom. Students who possess the grit and resilience to persist with the humdrum nature of revision tasks will have a greater chance at success, but teachers must also identify and plan revision strategies that work. Of course, our experience and intuition about what will work best for our students is important, but we should challenge our assumptions with the wider research that is easily accessible on the web.

Effective Exam Revision – ‘Drill Baby Drill!’

20130314-223636.jpg

It is that time of year already. The weeks have suddenly crashed forward like waves and exam dates loom bleakly in the distance. Of course, we start looking for creative ways to sustain the interest of students. We look to engage them in their learning so that they may pay some attention to our relentless pleading for effort and revision. We explain the grandiose importance of said exams and those simple letters on certificates that provide their ticket to their future lives. I have taught for a decade now and I may despair of the examinations themselves, and the crass system of judgement that is league tables, but if our students are to succeed in these exams then as we approach exam season the answer is simple. In the words made famous by that famous sage – Sarah Palin(!): “drill baby drill!”

Let me explain my rationale for what may seem a simplistic and backward looking approach. Yes, there are many constructivist techniques that will aid the mastery of knowledge and therefore be appropriate for revision. Spacing out revision to encompass different topics, with varied pedagogy is likely to help make some of their learning more memorable. The evidence at hand; however, proves that testing as practice, repeated, with skilled feedback has a very positive impact on learning – see research here. This has been labelled the ‘testing effect’, whereat the act of retrieving information for a test is proven to improve recall more than simply restudying information. This is not advocating lots of high stakes testing, but simply a recognition that testing for learning works and improves long term retention of knowledge: the holy grail for exam success. We mustn’t shy away from exam drills given the nature of the assessment.

Let me say that I am a teacher of English that values creativity and imagination hugely. I deplore exams which are narrow and reductive. I will prepare students for an AS level English Literature exam where the great literature we read and I teach, such as the poetry of Robert Browning, is reduced rather simplistically into a mere list of narrative devices. Yet, exams in themselves are a necessary evil – an imperfect judgement of knowledge and understanding – but one of the best tools we have to do the job. What I have come to learn myself is that repetition, and even rote learning, can actually create the fertile conditions for subsequent creativity and the potent force of imagination. In the words of the great American basketball coach, John Wooden: “Drilling creates a foundation on which individual initiative and imagination can flourish.”

I recently posted a blog on ‘deliberate practice’ here which reiterates the point that real improvement and mastery comes from drilled repetition, with quick, useful feedback. Of course, to make the many marginal gains required for true expertise there is much repetition and deliberate practice required. Students will likely not often experience the heralded ‘flow’ state during revision. Going over and over any body of knowledge takes grit, discipline and perseverance because it can often be simply boring. We must be honest with students about the power of conquering boredom. I think the skill of mastering boredom is a sure-fire path to ultimate success. What we mustn’t do, as teachers, is collude with many aspects of our instant gratification culture, and actually avoid the challenge – we must embrace the boredom and the difficulty of repeated exam practice. We must communicate with our students that not all learning can, or even should, be fun; it can often be hard, challenging and mentally gruelling.

Through repeated deliberate practice of exams students become skilled in the automatic state that comes with habit forming. It is like driving in a car for the thousandth time, we then switch off and drift into our own automatic mode, often having creative reveries as we drive. A good real life example of his notion is from the world of sport. Barcelona FC are world renowned for being the greatest football team in the world, perhaps of all time. Their creativity and skill is celebrated by every football fan. Of course, it isn’t sheer chance that has seen this come to pass. The Barcelona way, initiated by a great player, and believer in deliberate practice, Johan Cruyff, relies on the simplest of training drills – the ‘rondo’. See here:

The ‘rondo’ drill cannot replicate the pressure of the actual match in many ways, but the repeated drill hones that quick passing habit which is so key to their creative passing style – known as ‘tika taka’. It is a simplification of the real game, much like focusing in on exam technique, crafting and drafting the perfect exam answer singularly, rather than sitting full papers endlessly. what you can see in the video is the mastery, and the related pleasure, that comes from drilling and deliberate practice. What we must do is stop seeing repetition as being the enemy of creativity or higher order thinking. With exam revision repetition is our friend. We need to communicate that in the language we use to students. Like with writing, we cannot exhibit real creativity unless we have mastered the laws of grammar. Like Picasso painting a Cubist masterpiece, we cannot creatively break the rule unless we practise and understand those rules in the first place. We cannot make the cognitive leap of imaginative originality unless we have a solid grounding in the core knowledge of the basics.

It is common sense really, but too often teachers feel the pressure to teach ‘all singing, all dancing’ lessons, or make the learning fun and creative, when sometimes the most effective method is some good old-fashioned drilling of practice and well chosen drills of testing for learning.

‘All Eggs In One Basket’ – The Three Hour Exam

20130211-235331.jpg

Timothy Salthouse is a University of Virginia psychologist. He has conducted extensive research into testing, from intelligence to aptitude tests (from the age of eighteen to over eighty) in the snappily titled: ‘Implications of Within-Person Variability in Cognitive and Neuropsychological Functioning for the Interpretation of Change‘ (Neuropsychology 21, no.6 (2007). Now, why is this relevant to our current British education system and Michael Gove’s proposed changes to our curriculum and assessment at KS4 and beyond? Please let me explain. The research prompts serious reservations about something most teaching professionals know instinctively – putting all our eggs into one exam basket is both reductive and destructive.

Michael Gove has proposed that we should do away with coursework or other internal assessment procedures, except for specific subjects, such as Geography fieldwork or Drama practicals. That leaves subjects like mine, English, looking likely to end up with a summative judgement of a three hour examination. We all have our reservations about the exam system. Like the Heisenberg principle in quantum physics, we know that to precisely test one thing, we must inevitably be less precise with testing others. Therefore our testing system becomes narrower and narrower, to make a judgement on a narrow definition of the ‘progress’ of our students. In our results driven system, the curriculum gets ever condensed to meet the progress measures. All the while, the complexity and wealth of information our students have to deal with in our digital age is not narrowing at all, but growing exponentially! Surely our reservations about an ‘all eggs in one basket’ assessment aren’t just unfounded fears from educators seeking to survive in a judgement laden, punitive system?

Salthouse’s research presents us with really unsettling answers about the accuracy and efficacy of such a crucial and singular ‘all eggs in one basket’ assessment. His research has uncovered that there is a wide degree of variability ‘within the same individual’! That, on different days, people could sit the same test and perform in a vastly different fashion. This clearly raises the issue that any one single measurement provides an insufficient evaluation of a young person. His data showed that ‘the within-person deviation’ in test scores averaged about 50 percent of the between-person deviation for a variety of cognitive tasks. With such a bell curve of performance for individuals, sitting the same test, without specialist revision or preparation, simply on different days, how can we justify an ‘all eggs in one basket’ exam to culminate years of study? How fair is it for students that examinations on a Friday afternoon, for example, may suffer a degree of variability which may make students worse off than other students sitting a different exam board on a different day, with some bad weather? The variables are huge and the stakes are sky-rocket high. Of course, we see punitive attacks on entire schools for deficient performance.

This issue does not take further issues into account, such as the quality of examiners, or lack thereof. There is no professionalisation of examiners and the consistency of exam grading is annually brought into question, particularly for subjects such as English, which have a significant degree of extended interpretation. I could show you some exam papers of my past students which have been marked shockingly badly. Coupled with within-individual variation, such summative judgements become even more questionable. To ignore the breadth of quality internal assessment for such a high-stakes test smacks of ignorance.

In other curriculum and assessment models lauded by Michael Gove, such as the International Baccalaureate Diploma, there is a significant proportion of internal assessments; from portfolio work, to oral recordings and to extended coursework essays, externally moderated. The iGCSE assesses oral recordings for the speaking and listening component. If our politicians are scared of cheating in the system then provide a better model that deals with the gaming (or better still, remove the corrosive competition inherent in league tables with course comparison indicators!), such as using oral recordings; live moderation; draft evidence in essay work, or a portfolio approach. These assessment models may be more expensive, but they will mitigate the risk of the high stakes end of course exam model. Perhaps Gove has these in mind, he is just keeping his curriculum cards close to his chest – I hope so!

As Salthouse puts it: “…the existence of within-person variability complicates the assessment of cognitive and neuropsychological functioning and raises the possibility that single measurements may not be sufficient for precise evaluations of individuals, or for sensitive detection of change.” A bit of a mouthful, but the idea is simple: we simply cannot have a system where one bad day can scupper the life chances of any given young person. That is no model for a system looking to enhance deeper learning and militate against teaching to the test.

The culture of resits is ultimately corrosive to deeper learning. I do not advocate a resit culture, the perverse multiplication of exams, it gains nothing, except perhaps the ample profits for the exam boards! Yet, surely we have advanced beyond the antique paradigm of the ‘all eggs in one basket’ exam. Portfolios, speaking and listening assessments, well structured coursework all have their place in a more holistic approach to assessment. Let it be rigorous – I have no argument with that – but let’s not play roulette with the future of our students.

As Michael Gove concedes on the issue of the EBC qualifications replacing the GCSEs he is still intent on measures such as eliminating internal assessments for academic subjects, and other such narrowing effects upon educational outcomes. He clearly lauds the certain judgements of examinations, when evidence put forward by the likes of Timothy Salthouse calls their consistency and accuracy into question. We must therefore challenge the narrow and reductive proposals and put forward better curriculum and assessment models. We have a moral imperative to ensure that our students have a fit for purpose assessment model that is rounded and fit for the twenty first century.