Over the last few weeks I’ve found myself moving from being broadly sympathetic to educational ‘reform’ to being quite critical of it. One comment on my blog was “You appear to be doing that thing where you write loads, but it is hard to identify any clear points.” Point taken. I’ll see what I can do in this post.
my search for the evidence
I’ve been perplexed by the ideas underpinning the current English education system since my children started encountering problems with it about a decade ago. After a lot of searching, I came to the conclusion that the entire system was lost in a constructivist wilderness. I joined the TES forum to find out more, and discovered that on the whole, teachers weren’t – lost, that is. I came across references to evidence-based educational research and felt hopeful.
Some names were cited; Engelmann, Hirsch, Hattie, Willingham. I pictured a growing body of rigorous research and searched for the authors’ work. Apart from Hattie’s, I couldn’t find much. Willingham was obviously a cognitive psychologist but I couldn’t find his research either. I was puzzled. Most of the evidence seemed to come from magazine articles and a few large-scale studies – notorious for methodological problems. I then heard about Daisy Christodoulou’s book Seven Myths about Education and thought that might give me some pointers. I searched her blog.
In one post, Daisy cites work from the field of information theory by Kirschner, Sweller & Clark, Herb Simon and John Anderson. I was familiar with the last two researchers, but couldn’t open the Simon papers and Anderson’s seemed a bit technical for a general readership. I hadn’t come across the Kirschner, Sweller and Clark reference so I read it. I could see what they were getting at, but thought their reasoning was flawed.
Then it dawned on me. This was the evidence bit of the evidence-based research. It consisted of some early cognitive science/information theory, some large-scale studies and a meta-analysis, together with a large amount of opinion. To me that didn’t constitute a coherent body of evidence. But I was told that there was more to it, which is why I attended the ResearchED conference last weekend. There was more to it, but the substantial body of research didn’t materialise. So where does that leave me?
I still agree with some points that the educational reformers make;
• English-speaking education systems are dominated by constructivist pedagogical approaches
• the implementation of ‘minimal guidance’ approaches has failed to provide children with a good education
• we have a fairly reliable, valid body of knowledge about the world and children should learn about it
• skills tend to be domain-specific
• cognitive science can tell us a lot about how children learn
• the capacity of working memory is limited
• direct instruction is an effective way of teaching.
But I have several reservations that make me uneasy about the education reform ‘movement’.
1. the evidence.
Some is cited frequently. Here’s a summary.
If I’ve understood it correctly, Engelmann and Becker’s DISTAR programme (Direct Instruction System for Teaching Arithmetic and Reading) had far better outcomes for basic maths and reading, higher order cognitive skills (in reading and maths) and responsibility and self-esteem than any other programme in the Project Follow-Through evaluation carried out in 1977.
At around the same time, ED Hirsch had realised that his students’ comprehension of texts was impaired by their poor general knowledge, and in 1983 he published an outline of his concept of what he called ‘cultural literacy’.
A couple of decades later, Daniel Willingham, a cognitive psychologist, started to apply theory from cognitive science to education.
In 2008, John Hattie published Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement – the result of 15 years’ work. The effect sizes Hattie found for various educational factors are ranked here.
Kirschner, Sweller and Clark’s 2006 paper Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching is also often cited. John Sweller developed the concept of ‘cognitive load’ in the 1980s, based on the limited capacity of working memory.
2. the conclusions that can be drawn from the evidence
The DISTAR programme, often referred to as Direct Instruction (capitalised), is clearly very effective for teaching basic maths and literacy. This is an outcome not to be sniffed at, so it would be worth exploring why DISTAR hasn’t been more widely adopted. Proponents of direct instruction often claim it’s because of entrenched ideological opposition; it might also be to do with the fact that it’s a proprietary programme, that teacher input is highly constrained, and that schools have to teach more than basic maths and literacy.
ED Hirsch’s observation that students need prior knowledge before they can comprehend texts involving that knowledge is a helpful one, but has more to say about curriculum design than pedagogy. There are some major issues around all schools using an identical curriculum, who controls the content and how children’s knowledge of the curriculum is assessed.
Daniel Willingham has written extensively on how findings from cognitive science can be applied to education. Cognitive science is clearly a rich source of useful information. The reason I couldn’t find his research (mainly about procedural memory) appears to be because at some point he changed his middle initial from B to T. I’d assumed it was by someone else.
Although I have doubts about Kirschner Sweller and Clark’s paper, again the contribution from cognitive science is potentially valuable.
John Hattie’s meta-analyses provide some very useful insights into the effectiveness of educational influences.
The most substantial bodies of evidence cited are clearly cognitive science and Hattie’s meta-analyses, which provide a valuable starting point for further exploration of the influences he ranks. Those are my conclusions.
But other conclusions are being drawn – often that the evidence cited above supports the view that direct instruction is the most effective way of teaching and that traditional educational methods (however they are defined) are superior to progressive ones (however they are defined). Those conclusions seem to me to be using the evidence to support beliefs about educational methods, rather than deriving beliefs about educational methods from the evidence.
3. who’s evaluating the evidence?
A key point made by proponents of direct instruction is that students need to have knowledge before they can do anything effective with it. Obviously they do. But this principle appears to be being overlooked by the very people who are emphasizing it.
If you want to understand and apply findings from a meta-analysis you need to be aware of common problems with meta-analyses, how reliable they are, what you need to bear in mind about complex constructs etc. You don’t need to have read everything there is to read about meta-analyses, just to be aware of potential pitfalls. If you want to apply findings from cognitive science, it would help to have at least a broad overview of cognitive science first. That’s because, if you don’t have much prior knowledge, you have no way of knowing how reliable or valid information is. If it’s from a peer-reviewed paper, there’s a good chance it’s reliable because the reviewers would have looked at the theory, the data, the analysis and conclusions. How valid it is (ie how well it maps on to the real world) is another matter. I want to look at some of what ED Hirsch has written to illustrate the point.
Hirsch on psychology and science
Hirsch’s work is often referred to by education reformers. I think he’s right to emphasise the importance of students’ knowledge and I’m impressed by his Core Knowledge framework. There’s now a UK version (slightly less impressive) and his work has influenced the new English National Curriculum. But when I started to check out some of what Hirsch has written I was disconcerted to find that he doesn’t seem to practice what he preaches. In an article in Policy Review he sets out seven ‘reliable general principles’ derived from cognitive science to guide teachers. The principles are sound, even if he has misconstrued ‘chunking’ and views rehearsal as a ‘disagreeable need’.
But Hirsch’s misunderstanding of the history of psychology suggests that not everything he says about psychology might be entirely reliable. He says;
“Fifty years ago [the article is dated 2002] psychology was dominated by the guru principle. One declared an allegiance to B.F. Skinner and behaviorism, or to Piaget and stage theory, or to Vygotsky and social theory. Today, by contrast, a new generation of “cognitive scientists,” while duly respectful of these important figures, have leavened their insights with further evidence (not least, thanks to new technology), and have been able to take a less speculative and guru-dominated approach. This is not to suggest that psychology has now reached the maturity and consensus level of solid-state physics. But it is now more reliable than it was, say, in the Thorndike era with its endless debates over “transfer of training.””
This paragraph is riddled with misconceptions. Skinner was indeed an influential psychologist, but behaviourism was controversial – Noam Chomsky was a high profile critic. Piaget was influential in educational circles – but children’s cognitive development formed one small strand of the wide range of areas being investigated by psychologists. Vygotsky’s work has also been influential in education, but it didn’t become widely known in the West until after the publication in 1978 of Mind in Society – a collection of his writings translated into English – so he couldn’t have had ‘guru’ status in psychology in the 1950s. And to suggest that cognitive scientists are ‘duly respectful’ of Skinner, Piaget and Vygotsky as ‘important figures’ in their field, suggests a complete misunderstanding of the roots of cognitive science and of what matters to cognitive scientists. But you wouldn’t be able to question what Hirsch is saying if you had no prior information. And in this article, Hirsch doesn’t support his assertions with references, so you couldn’t check them out.
In a conference address that also forms a chapter in book entitled The Great Curriculum Debate, Hirsch attributes progressive educational methods to the Romantic movement and in turn to religious beliefs, completely overlooking the origins in psychological research of ‘progressive’ educational methodologies and, significantly, the influence of Freud’s work.
In the grand scheme of things, of course, Hirsch’s view of psychology in the 1950s, or his view of the origins of progressive education don’t matter that much. What does matter is that Hirsch himself is seen as something of a guru largely because of his emphasis on students needing to have sound prior knowledge, but here he clearly hasn’t checked out his own.
What’s more important is Hirsch’s view of science. In the last section of the article on classroom research and cargo cults, entitled ‘on convergence and consensus’, in which he compares classroom research with that from cognitive psychology, he says “independent convergence has always been the hallmark of dependable science“. That’s true in the sense that if several researchers approaching a problem from different directions all come to the same conclusion, they would be reasonably confident that their conclusion was a valid one.
Hirsch illustrates the role of convergence using the example of germ theory. He says “in the nineteenth century, for example, evidence from many directions converged on the germ theory of disease. Once policymakers accepted that consensus, hospital operating rooms, under penalty of being shut down, had to meet high standards of cleanliness.” What’s interesting is that Hirsch slips, almost imperceptibly, from ‘convergence’ into ‘consensus’. In scientific research, convergence is important, but consensus can be extremely misleading because it can be, and often has been, wrong. Ironically, not long before high standards of cleanliness were imposed on hospitals, the consensus had been that cross-contamination theory was wrong, as Semmelweis discovered to his cost. Reliable findings aren’t the same as valid ones.
Hirsch then goes on to say “What policymakers should demand from the [education] research community is consensus.” No they shouldn’t. Consensus can be wrong. What policymakers need to demand from education research is methodological rigour. We already have the relevant expertise, it just needs to be applied to education. Again, if you have no frame of reference against which you can evaluate what Hirsch is saying, you’d be quite likely to assume that he’s right about convergence and consensus – and you’d be none the wiser about the importance of good research design.
what the teachers say
I’m genuinely enthusiastic about teachers wanting to base their practice on evidence. I recognize that this is a work in progress and it’s only just begun. I can quite understand why someone whose teaching has been transformed by a finding from cognitive science might want to share that information as widely as possible. But ironically, some of the teachers involved appear to be doing exactly the opposite of what they recommend teachers do with students.
If you’re not familiar with a knowledge domain, but want to use findings from it, it’s worth getting an overview of it first. This doesn’t involve learning loads of concrete facts, it involves getting someone with good domain knowledge to give you an outline of how it works, so you can see how the concrete facts fit in. It also involves making sure you know what domain-specific skills are required to handle the concrete facts, and whether or not you have them. It also means not making overstated claims. Applying seven principles from cognitive science means you are applying seven principles from cognitive science. That’s all. It’s important to avoid making claims that aren’t supported by the evidence.
What struck me about the supporters of educational reform is that science teachers are noticeable by their absence. Most of the complaints about progressive education seem to relate to English, Mathematics and History. These are all fields that deal with highly abstracted information that is especially vulnerable to constructivist worldviews, so they might have been disproportionately influenced by ‘minimal guidance’ methods. It’s a bit more difficult to take an extreme constructivist approach to physics, chemistry, biology or physical geography because reality tends to intervene quite early on. The irony is that science teachers might be in a better position than teachers of English, Maths or History to evaluate evidence from educational research. And psychology teachers and educational psychologists would have the relevant domain knowledge, which would help avoid reinventing the wheel. I’d recommend getting some of them on board.