Evidence Driven Education

20130106-101856.jpg

I started the school year talking with my faculty about our success in the summer and throughout the previous year and of course the areas we needed to improve. Upon reflection we could identify some clear reasoning why the successes occurred, hard earned as they were. The reasons primarily centred on having a team of very good teachers who taught very good lessons consistently (to bastardise a Bill Clinton phrase: “It is the teaching stupid!“) – which is backed up by hard-nosed lesson judgements. This was bolstered by our effective managing of data and our concentration upon the things that mattered that we could control, like controlled assessments (I think we are all agreed that is a dirty word that we will be happy to be rid of soon enough). What struck me, particularly having read John Hattie’s ‘Visible Learning for Teachers‘, was that we had some experienced ‘intuitions‘ about why we did well, some hard data and some soft data, all leading to the same conclusions, but that to continue to replicate that success we needed to be much more systematic with our evaluations and our evidence.

I wanted us to focus even more closely on what happened in the good lessons that made them consistently good – after all, that is point of what we are trying to do isn’t it – get better at teaching. I wanted to explicitly know this so that regardless of what assessment models or curricular systems are imposed upon us in the coming year, or even future years, we could still teach great lessons consistently: essentially “keeping he main thing the main thing”. We also reflected upon the truth that there is no ‘one size fits all formula’ for good teaching, but that did not stop us analysing the evidence of high impact strategies. Knowing that, and what interventions worked best, would be a crystallisation of all the answers we need. It takes effort, but done properly the rewards are huge (I don’t underestimate such effort when we are all pushed to the limits to do the job well – perhaps schools should have their own ‘Delivery Units‘ to to do the job across the school?)

I therefore wanted to make a concerted attempt to think with both the heart and the head (or the ‘Elephant’ and the ‘Rider’ to unite my previous post) and to source the best evidence possible for great teaching. That included the epic meta-analyses of John Hattie and his team. I am in no doubt of the efficacy of this type of research; it is essential for medical research and it does add value to educational debate. I was, and am, however circumspect at the same time. Instinctively I asked how could individual teaching strategies be fairly judged within school contexts when there are a whole host of other factors at work simultaneously – a complex web even the sharpest of minds would struggle to delineate. For example, how could the success of ‘questioning‘ be fairly judged when at the same time ‘teacher subject knowledge‘, ‘class size‘ and a whole host of other effect sizes are at work? Therefore, I surmised that such data is imperfect. Yet, the more I read, the more I couldn’t avert myself from the fact that this was still the best way to source the answers about what made good teaching and good interventions effective. What is key is that the sheer scale and selectivity of the trials improved the quality and accuracy of the data, and continues to do so.

The ethical basis of ‘testing’ on children is a valid objection to such an evidence based approach. The idea of a ‘control group‘ not having an intervention instinctively sits uneasily with me. In practice, with a technology trial for example, it is hard to deny a class the right to use technology which may be most appropriate with a task. However, when I thought about it, I considered that many interventions can actually have a negative effect, or be a distraction, so it was a case, once more of thinking differently, thinking more scientifically and less emotionally. I am simply not in my job to be unethical, quite the reverse. What would be unethical would be to not undertake RCTs (randomly controlled trials) and to instead base our teaching, our interventions, indeed our entire system upon a hunch, or on a personal basis, solely based on ideology. When we move educational policy at break-neck speed, we are likely to take in unnecessary risks, which I deem wholly unethical. This is my primary objection to Gove’s Ebacc proposals. There is no evidence, no research and no trials to support his radical change. I find his approach arrogant and potentially dangerous. He is so caught up in the political expediency of a ‘shock doctrine‘ style swift change that he ignores the experts and the evidence. the obvious question is why then should educational policy not be driven by such an evidence base?

What we cannot do is simply rely upon the ‘noble myth‘ (described by Plato as well meaning, but flawed reasoning to perpetuate comfort for the greater good) of our intuitions alone, however experienced we are. Our well meaning, but flawed emotional response to ‘what has always worked‘ for us is always going to be too narrow in scope, too bound with our own emotional bias to be sufficient. We need to focus in on the practice and the pedagogy, which often means stepping back from the personal. Yes, we are all emotional beings (thankfully so – the best of us often being those most in touch with their emotional intuition), who teach with our head and heart, but we must reflect and make adjustments and plan improvements with as scientific an approach as possible if we are to properly define what is good teaching. As an English teacher, this strikes against some of my natural instincts, stemming from the Romantic ideal of individual genius and the power of emotional intuition to find ‘the answer‘.

Of course, any one source of evidence is too narrow if we have the opportunity to source more evidence from a variety of methods. The best answers, as I have stated, are to be found when we have the greatest breadth of the evidence: including hard data (ideally through rigorous control based trials), but also including soft data – like student voice and teacher feedback – and our personal and professional intuition as experienced experts – those aforementioned ‘noble myths‘. What is crucial is that we have policy makers, school leaders, subject leaders or teachers who do not unthinkingly implement changes based upon statistical evidence, provided by the likes of Hattie, without taking a full account of the unique context of their country, their school and their students. This would be foolhardy and it is a valid concern levelled at evidence led policy that we must address. School leaders, for example, will be sold snake-oil by gurus looking to sell their foolproof educational wares based on what they present as the most rigorous of evidence (of course that data will be flawed and manipulated, as such data can be). Some leaders are simply looking for a quick fix to their problems, when quick fixes don’t exist in schools! Therefore we must question the methodology behind the evidence and weigh up the factors impacting upon the evidence – again, taking a more scientific approach.

So what is to be done? In our faculty it is about trialling strategies and becoming more systematic about that trialling. It is about sharing our good practice and our good pedagogy, but crucially then evaluating its impact in a more rigorous fashion. In all honesty, our current evidence does not stand up to the scrutiny I outline above. Therefore it is important that we so the hard work to make this so. Focusing with utter consistency upon the pedagogy and the practice…and the evidence of impact.

This model of sourcing better evidence to justify change needs to replicated at school level and even on a national level. This is happening, but typically away from view. The Education Endowment Fund is currently running trials across a thousand schools in Britain to source evidence to direct policy – read this fascinating research on school interventions for instance: Teaching and Learning Toolkit. The debate is happening and policy people in Whitehall are listening: listen to Ben Goldacre’s brilliant analysis here on how evidence led policy is being undertaken in Whitehall (the education focused section begins around the twenty seven minute mark – including debate about phonics teaching). We must challenge the many ‘noble myths‘ that attend our educational discourse and source as must high quality evidence of impact as we can.

Advertisements

Tags: , , , , ,

About huntingenglish

I am Director of Learning and Research at Huntington Secondary School, York. I have taught English (including a bit of Media Studies) for over a decade. If it is tragic and gothic, laced with humour and bitter truths then I'll teach it! I have had the great privilege to have written a book, 'Teach Now! Becoming a Great English Teacher', edited by the brilliant Geoff Barton, and I am writing another for Routledge, entitled, 'The Confident Teacher'. I am Project Lead of the RISE (Research-leads Improving Students' Education) Project. An EEF funded randomized controlled trial to evaluate if and how a Research-lead can improve outcomes for students. I am a proud member of the Institute for Effective Education (IEE) Executive Board at York University. I am also a proud member of the ResearchEd Advisory Panel. I write regularly for the TES and Teach Secondary magazine. My first book, 'Teach Now! Becoming A Great English Teacher' was released by Routledge in 2014 and my next book, entitled 'The Confident Teacher' will be out at the end of 2015.

10 responses to “Evidence Driven Education”

  1. Christopher Waugh says :

    It’s exciting to see a secondary English department at such an advanced stage that it’s beginning to reflect on its own practice and move towards developing its own evidence basis!

    I’ll be following your progress with a keen eye for what I can glean. I’m particularly interested in any methodology you develop for efficiently generating data that might help us evaluate our own work in schools.

    This is something that is in the foreground of my mind too, as I am exploring and innovating in my classroom work, but find the exiting measures (raw exam grades, student evaluations, ‘residuals’) to be blunt, unreliable or downright misleading.

    Thanks for sharing your work!

    Chris

    • huntingenglish says :

      “This is something that is in the foreground of my mind too, as I am exploring and innovating in my classroom work, but find the exiting measures (raw exam grades, student evaluations, ‘residuals’) to be blunt, unreliable or downright misleading.”

      I feel exactly the same. We are running an iPad pilot. I could ‘select’ my evidence to prove its success, but it wouldn’t be wholly accurate. I think that is what we do with evidence too much in education: find the evidence to prove our opinions and interventions! I would like to explore RCTs and the EEF model further. I’m no scientist but I want to really learn this stuff and apply it.

      • Christopher Waugh says :

        I’m interested too!

        Thanks again for sharing your thinking and work. I’ll do the same if I come up with anything. Another approach I’m working on is trying to get some of the Higher Education facilities to take an interest in my classroom with the hope that they might direct a student to research something.

      • huntingenglish says :

        Yes – a good idea. An untapped resource for us too.

  2. Mrs B (@Mrz__Black) says :

    We are beginning a lesson study project in science, modelled on the Japanese programme. the project is being run by the science learning centre in Sheffield. The emphasis being on the learning by the students. IRIS and collaborative lesson planning and intense scrutiny of a lesson repeated several times and feedback and tweaking as a group is the basic process we are going to be following. If you are interested i will keep you updated via twitter #lessonstudy

  3. Stephen Pickles says :

    I am interested in in your thoughts about the sources of evidence that you have access to. As a former university librarian in the field, I was always very conscious of the of how little of the wonderful collections we had were accessible to anyone other than higher education staff and students – in particular the research journal literature which remains for the most part locked up behind publisher paywalls. HE tries to maximise access wherever it can – eg http://eprints.ioe.ac.uk/
    But it is taking a long time to liberate anything more than a small proportion of the total output.

    • huntingenglish says :

      I have very little to go on I would have to invest a great deal of time to find it. I think Universities should be more proactive about sharing their research – there should be a particular expectation for local Universities to share with local schools.

  4. 3arn0wl says :

    I’m so pleased that you put the teacher right up top there – they are a school’s top asset, and yet, that’s so often forgotten or, at the very least, their efforts go unrecognised, not complimented or unrewarded.

    I’m pleased that you demonstrate that teachers are keen to improve their practice too. To me, it’s the hallmark of a good teacher that they reflect on their work continually, and try to find ways to improve it.

    But as you say, any number of influences (good and bad) constantly affect the quality of teaching and learning, such that it’s probably impossible to make a fair test.

    You intimated that you look forward to a time when Controlled Assessments are done away with. Surely you’d agree that students need to acquire skills as well as knowledge in all subjects? And that as those skills are integral to the subject, then the skill level the student reaches needs to be assessed too? I was wondering what your solution to that is?

Trackbacks / Pingbacks

  1. The Noble Myth « The Cowboy Campfire - January 14, 2013

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: