going round in circles

Central to the Tiger Teachers’ model of cognitive science is the concept of cognitive load. Cognitive load refers to the amount of material that working memory is handling at any one time. It’s a concept introduced by John Sweller, a researcher frequently cited by the Tiger Teachers. Cognitive load is an important concept for education because human working memory capacity is very limited – we can think about only a handful of items at the same time. If students’ cognitive load is too high, they won’t be able to solve problems or will fail to learn some material.

I’ve had concerns about the Tiger Teachers’ interpretations of concepts from cognitive science, and about how they apply those concepts to their own learning, but until recently I hadn’t paid much attention to the way their students were being taught. I had little information about it for a start, and if it ‘worked’ for a particular group of teachers and students, I saw no reason to question it.

increasing cognitive load

The Michaela Community School recently blogged about solving problems involving circle theorems. Vince Ulam, a mathematician and maths teacher*, took issue with the diagrammatic representations of the problems.

The diagrams of the circles and triangles are clearly not accurate; they don’t claim to be. In an ensuing Twitter discussion, opinion was divided over whether or not the accuracy of diagrams mattered. Some people thought it didn’t matter if the diagrams were intended only as a representation of an algebraic or arithmetic problem. One teacher thought inaccurate diagrams would ensure the students didn’t measure angles or guess them.

The problem with the diagrams is not that they are imprecise – few people would quibble over a sketch diagram representing an angle of 28° that was actually 32°. It’s that they are so inaccurate as to be misleading. For example, there’s an obtuse angle that clearly isn’t obtuse, an angle of 71° is more acute than one of 28°, and a couple of isosceles triangles are scalene. As Vince points out, this makes it impossible for students to determine anything by inspection – an important feature of trigonometry. Diagrams with this level of inaccuracy also have implications for cognitive load, something that the Tiger Teachers are, rightly, keen to minimise.

My introduction to trigonometry at school was what the Tiger Teachers would probably describe as ‘traditional’. A sketch diagram illustrating a trigonometry problem was acceptable, but was expected to present a reasonably accurate representation of the problem. A diagram of an isosceles triangle might not be to scale, but it should be an isosceles triangle. An obtuse angle should be an obtuse angle, and an angle of 28° should not be larger than one of 71°.

Personally, I found some of the inaccurate diagrams so inaccurate as to be quite disconcerting. After all those years of trigonometry, the shapes of isosceles triangles, obtuse angles, and the relative sizes of angles of ~30° or ~70°, are burned into my brain, as the Tiger Teachers would no doubt expect them to be. So seeing a scalene triangle masquerading as an isosceles, an acute angle claiming to be 99°, and angles of 28° and 71° trading places, set up a somewhat unnerving Necker shift. In each case my brain started flipping between two contradictory representations; what the diagram was telling me and what the numbers were telling me.

It was the Stroop effect but with lines and numbers rather than letters and colours; and the Stroop effect increases cognitive load.  Even students accustomed to isosceles triangles not always looking like isosceles triangles would experience an increased cognitive load whilst looking at these diagrams, because they’ll have to process two competing representations; what their brain is telling them about the diagram and what it’s telling them about the numbers.  I had similar misgivings about the ‘CUDDLES’ approach used to teach French at Michaela.

CUDDLES and cognitive load

The ‘traditional’ approach to teaching foreign languages is to start with a bunch of simple nouns, adjectives and verbs, do a lot of rehearsal, and work up from there; that approach keeps cognitive load low from the get-go.   The Michaela approach seems to be to start with some complex language and break it down in a quasi-mathematical fashion involving underlining some letters, dotting others and telling stories about words.

Not only do students need to learn the words, what they represent and how French speakers use them, they have to learn a good deal of material extraneous to the language itself. I can see how the extraneous material acts as a belt-and-braces approach to ‘securing’ knowledge, but it must increase cognitive load because the students have to think about that as well as the language.

The Tiger Teacher’s approach to teaching is intriguing, but I still can’t figure out the underlying rationale; it certainly isn’t about reducing cognitive load.  Why does the Tiger Teachers’ approach to teaching matter?  Because now Nick Gibb is signed up to it, it will probably become educational policy, regardless of the validity of the evidence.

Note:  I resisted the temptation to call this post ‘non angeli sed anguli’.

*Amended from ‘maths teacher’ –  Old Andrew correctly pointed out that this was an assumption on my part. Vince Ulam assures me my assumption was correct.  I guess he should know.

the debating society

One of my concerns about the model of knowledge promoted by the Tiger Teachers is that it hasn’t been subjected to sufficient scrutiny.   A couple of days ago on Twitter I said as much.  Jonathan Porter, a teacher at the Michaela Community School, thought my criticism unfair because the school has invited critique by publishing a book and hosting two debating days. Another teacher recommended watching the debate between Guy Claxton and Daisy Christodoulou Sir Ken is right: traditional education kills creativity. She said it may not address my concerns about theory. She was right, it didn’t. But it did suggest a constructive way to extend the Tiger Teachers’ model of knowledge.

the debate

Guy, speaking for the motion and defending Sir Ken Robinson’s views, highlights the importance of schools developing students’ creativity, and answers the question ‘what is creativity?’ by referring to the findings of an OECD study; that creativity emerges from six factors – curiosity, determination, imagination, discipline, craftsmanship and collaboration. Daisy, opposing the motion, says that although she and Guy agree on the importance of creativity and its definition, they differ over the methods used in schools to develop it.

Daisy says Guy’s model involves students learning to be creative by practising being creative, which doesn’t make sense. It’s a valid point. Guy says knowledge is a necessary but not sufficient condition for developing creativity; other factors are involved. Another valid point. Both Daisy and Guy debate the motion but they approach it from very different perspectives, so they don’t actually rigorously test each other’s arguments.

Daisy’s model of creativity is a bottom-up one. Her starting point is how people form their knowledge and how that develops into creativity. Guy’s model, in contrast, is a top-down one; he points out that creativity isn’t a single thing, but emerges from several factors. In this post, I propose that Daisy and Guy are using the same model of creativity, but because Daisy’s focus is on one part and Guy’s on another, their arguments shoot straight past each other, and that in isolation, both perspectives are problematic.

Creativity is a complex construct, as Guy points out. A problem with his perspective is that the factors he found to be associated with creativity are themselves complex constructs. How does ‘curiosity’ manifest itself? Is it the same in everyone or does it vary from person to person? Are there multiple component factors associated with curiosity too? Can we ask the same questions about ‘imagination’? Daisy, in contrast, claims a central role for knowledge and deliberate practice. A problem with Daisy’s perspective is, as I’ve pointed out elsewhere, that her model of knowledge peters out when it comes to the complex cognition Guy refers to. With bit more information, Daisy and Guy could have done some joined-up thinking.  To me, the two models look like the representation below, the grey words and arrows indicating concepts and connections referred to but not explained in detail.

slide1

cognition and expertise

If I’ve understood it correctly, Daisy’s model of creativity is essentially this: If knowledge is firmly embedded in long-term memory (LTM) via lots of deliberate practice and organised into schemas, it results in expertise. Experts can retrieve their knowledge from LTM instantly and can apply it flexibly. In short, creativity is a feature of expertise.

Daisy makes frequent references to research; what scientists think, half a century of research, what all the research has shown. She names names; Herb Simon, Anders Ericsson, Robert Bjork. She reports research showing that expert chess players, football players or musicians don’t practise whole games or entire musical works – they practise short sequences repeatedly until they’ve overlearned them. That’s what enables experts to be creative.

Daisy’s model of expertise is firmly rooted in an understanding of cognition that emerged from artificial intelligence (AI) research in the 1950s and 1960s. At the time, researchers were aware that human cognition was highly complex and often seemed illogical.  Computer science offered an opportunity to find out more; by manipulating the data and rules fed into a computer, researchers could test different models of cognition that might explain how experts thought.

It was no good researchers starting with the most complex illogical thinking – because it was complex and illogical. It made more sense to begin with some simpler examples, which is why the AI researchers chose chess, sport and music as domains to explore. Expertise in these domains looks pretty complex, but the complexity has obvious limits because chess, sport and music have clear, explicit rules. There are thousands of ways you can configure chess pieces or football players and a ball during a game, but you can’t configure them any-old-how because chess and football have rules. Similarly, a musician can play a piece of music in many different ways, but they can’t play it any-old-how because then it wouldn’t be the same piece of music.

In chess, sport and music, experts have almost complete knowledge, clear explicit rules, and comparatively low levels of uncertainty.   Expert geneticists, doctors, sociologists, politicians and historians, in contrast, often work with incomplete knowledge, many of the domain ‘rules’ are unknown, and uncertainty can be very high. In those circumstances, expertise  involves more than simply overlearning a great many facts and applying them flexibly.

Daisy is right that expertise and creativity emerge from deliberate practice of short sequences – for those who play chess, sport or music. Chess, soccer and Beethoven’s piano concerto No. 5 haven’t changed much since the current rules were agreed and are unlikely to change much in future. But domains like medicine, economics and history still periodically undergo seismic shifts in the way whole areas of the domains are structured, as new knowledge comes to light.

This is the point at which Daisy’s and Guy’s models of creativity could be joined up.  I’m not suggesting some woolly compromise between the two. What I am suggesting is that research that followed the early AI work offers the missing link.

I think the missing link is the schema.   Daisy mentions schemata (or schemas if you prefer) but only in terms of arranging historical events chronologically. Joe Kirby in Battle Hymn of the Tiger Teachers also recognises that there can be an underlying schema in the way students are taught.  But the Tiger Teachers don’t explore the idea of the schema in any detail.

schemas, schemata

A schema is the way people mentally organise their knowledge. Some schemata are standardised and widely used – such as the periodic table or multiplication tables. Others are shared by many people, but are a bit variable – such as the Linnaean taxonomy of living organisms or the right/left political divide. But because schemata are constructed from the knowledge and experience of the individual, some are quite idiosyncratic. Many teachers will be familiar with students all taught the same material in the same way, but developing rather different understandings of it.

There’s been a fair amount of research into schemata. The schema was first proposed as a psychological concept by Jean Piaget*. Frederic Bartlett carried out a series of experiments in the 1930s demonstrating that people use schemata, and in the heyday of AI the concept was explored further by, for example, David Rumelhart, Marvin Minsky and Robert Axelrod. It later extended into script theory (Roger Schank and Robert Abelson), and how people form prototypes and categories (e.g. Eleanor Rosch, George Lakoff). The schema might be the missing link between Daisy’s and Guy’s models of creativity, but both models stop before they get there. Here’s how the cognitive science research allows them to be joined up.

Last week I finally got round to reading Jerry Fodor’s book The Modularity of Mind, published in 1983. By that time, cognitive scientists had built up a substantial body of evidence related to cognitive architecture. Although the evidence itself was generally robust, what it was saying about the architecture was ambiguous. It appeared to indicate that cognitive processes were modular, with specific modules processing specific types of information e.g. visual or linguistic. It also indicated that some cognitive processes operated across the board, e.g. problem-solving or intelligence. The debate had tended to be rather polarised.  What Fodor proposed was that cognition isn’t a case of either-or, but of both-and; that perceptual and linguistic processing is modular, but higher-level, more complex cognition that draws on modular information, is global.   His prediction turned out to be pretty accurate, which is why Daisy’s and Guy’s models can be joined up.

Fodor was familiar enough with the evidence to know that he was very likely to be on the right track, but his model of cognition is a complex one, and he knew he could have been wrong about some bits of it. So he deliberately exposes his model to the criticism of cognitive scientists, philosophers and anyone else who cared to comment, because that’s how the scientific method works. A hypothesis is tested. People try to falsify it. If they can’t, then the hypothesis signposts a route worth exploring further. If they can, then researchers don’t need to waste any more time exploring a dead end.

joined-up thinking

Daisy’s model of creativity has emerged from a small sub-field of cognitive science – what AI researchers discovered about expertise in domains with clear, explicit rules. She doesn’t appear to see the need to explore schemata in detail because the schemata used in chess, sport and music are by definition highly codified and widely shared.  That’s why the AI researchers chose them.  The situation is different in the sciences, humanities and arts where schemata are of utmost importance, and differences between them can be the cause of significant conflict.  Guy’s model originates in a very different sub-field of cognitive science – the application of high-level cognitive processes to education. Schemata are a crucial component; although Guy doesn’t explore them in this debate, his previous work indicates he’s very familiar with the concept.

Since the 1950s, cognitive science has exploded into a vast research field, encompassing everything from the dyes used to stain brain tissue, through the statistical analysis of brain scans, to the errors and biases that affect judgement and decision-making by experts. Obviously it isn’t necessary to know everything about cognitive science before you can apply it to teaching, but if you’re proposing a particular model of cognition, having an overview of the field and inviting critique of the model would help avoid unnecessary errors and disagreements.  In this debate, I suggest schemata are noticeable by their absence.

*First use of schema as a psychological concept is widely attributed to Piaget, but I haven’t yet been able to find a reference.

The Tiger Teachers and cognitive science

Cognitive science is a key plank in the Tiger Teachers’ model of knowledge. If I’ve understood it properly the model looks something like this:

Cognitive science has discovered that working memory has limited capacity and duration, so pupils can’t process large amounts of novel information. If this information is secured in long-term memory via spaced, interleaved practice, students can recall it instantly whenever they need it, freeing up working memory for thinking.

What’s wrong with that? Nothing, as it stands. It’s what’s missing that’s the problem.

Subject knowledge

One of the Tiger Teachers’ beefs about the current education system is its emphasis on transferable skills. They point out that skills are not universally transferable, many are subject-specific, and in order to develop expertise in higher-level skills novices need a substantial amount of subject knowledge. Tiger Teachers’ pupils are expected to pay attention to experts (their teachers) and memorise a lot of facts before they can comprehend, apply, analyse, synthesise or evaluate. The model is broadly supported by cognitive science and the Tiger Teachers apply it rigorously to children. But not to themselves, it seems.

For most Tiger Teachers cognitive science will be an unfamiliar subject area. That makes them (like most of us) cognitive science novices. Obviously they don’t need to become experts in cognitive science to apply it to their educational practice, but they do need the key facts and concepts and a basic overview of the field. The overview is important because they need to know how the facts fit together and the limitations of how they can be applied.   But with a few honourable exceptions (Daisy Christodoulou, David Didau and Greg Ashman spring to mind – apologies if I’ve missed anyone out), many Tiger Teachers don’t appear to have even thought about acquiring expertise, key facts and concepts or an overview. As a consequence facts are misunderstood or overlooked, principles from other knowledge domains are applied inappropriately, and erroneous assumptions made about how science works. Here are some examples:

It’s a fact…

“Teachers’ brains work exactly the same way as pupils’” (p.177). No they don’t. Cognitive science (ironically) thinks that children’s brains begin by forming trillions of connections (synapses). Then through to early adulthood, synapses that aren’t used get pruned, which makes information processing more efficient. (There’s a good summary here.)  Pupils’ brains are as different to teachers’ brains as children’s bodies are different to adults’ bodies. Similarities don’t mean they’re identical.

Then there’s working memory. “As the cognitive scientist Daniel Willingham explains, we learn by transferring knowledge from the short-term memory to the long term memory” (p177). Well, kind of – if you assume that what Willingham explicitly describes as “just about the simplest model of the mind possible”  is an exhaustive model of memory. If you think that, you might conclude, wrongly, “the more knowledge we have in long-term memory, the more space we have in our working memory to process new information” (p.177). Or that “information cannot accumulate into long-term memory while working memory is being used” (p.36).

Long-term memory takes centre stage in the Tiger Teachers’ model of cognition. The only downside attributed to it is our tendency to forget things if we don’t revisit them (p.22). Other well-established characteristics of long-term memory – its unreliability, errors and biases – are simply overlooked, despite Daisy Christodoulou’s frequent citation of Daniel Kahneman whose work focused on those flaws.

With regard to transferable skills we’re told “cognitive scientist Herb Simon and his colleagues have cast doubt on the idea that there are any general or transferable cognitive skills” (p.17), when what they actually cast doubt on is the ideas that all skills are transferable or that none are.

The Michaela cognitive model is distinctly reductionist; “all there is to intelligence is the simple accrual and tuning of many small units of knowledge that in total produce complex cognition” (p.19). Then there’s “skills are simply just a composite of sequential knowledge – all skills can be broken down to irreducible pieces of knowledge” (p.161).

The statement about intelligence is a direct quote from John Anderson’s paper ‘A Simple Theory of Complex Cognition’ but Anderson isn’t credited, so you might not know he was talking about simple encodings of objects and transformations, and that by ‘intelligence’ he means how ants behave rather than IQ. I’ve looked at Daisy Christodoulou’s interpretation of Anderson’s model here.

The idea that intelligence and skills consist ‘simply just’ of units of knowledge ignores Anderson’s procedural rules and marginalises the role of the schema – the way people configure their knowledge. Joe Kirby mentions “procedural and substantive schemata” (p. 17), but seems to see them only in terms of how units of knowledge are configured for teaching purposes; “subject content knowledge is best organised into the most memorable schemata … chronological, cumulative schemata help pupils remember subject knowledge in the long term” (p.21). The concept of schemata as the way individuals, groups or entire academic disciplines configure their knowledge, that the same knowledge can be configured in different ways resulting in different meanings, or that configurations sometimes turn out to be profoundly wrong, doesn’t appear to feature in the Tiger Teachers’ model.

Skills: to transfer or not to transfer?

Tiger Teachers see higher-level skills as subject-specific. That hasn’t stopped them applying higher-level skills from one domain inappropriately to another. In her critique of Bloom’s taxonomy, Daisy Christodoulou describes it as a ‘metaphor’ for the relationship between knowledge and skills. She refers to two other metaphors; ED Hirsch’s scrambled egg and Joe Kirby’s double helix (Seven Myths p.21).  Daisy, Joe and ED teach English, and metaphors are an important feature in English literature. Scientists do use metaphors, but they use analogies more often, because in the natural world patterns often repeat themselves at different levels of abstraction. Daisy, Joe and ED are right to complain about Bloom’s taxonomy being used to justify divorcing skills from knowledge. And the taxonomy itself might be wrong or misleading.   But it is a taxonomy and it is based on an important scientific concept – levels of abstraction – so should be critiqued as such, not as if it were a device used by a novelist.

Not all evidence is equal

A major challenge for novices is what criteria they can use to decide whether or not factual information is valid. They can’t use their overview of a subject area if they don’t have one. They can’t weigh up one set of facts against another if they don’t know enough facts. So Tiger Teachers who are cognitive science novices have to fall back on the criteria ED Hirsch uses to evaluate psychology – the reputation of researchers and consensus. Those might be key criteria in evaluating English literature, but they’re secondary issues for scientific research, and for good reason.

Novices then have to figure out how to evaluate the reputation of researchers and consensus. The Tiger Teachers struggle with reputation. Daniel Willingham and Paul Kirschner are cited more frequently than Herb Simon, but with all due respect to Willingham and Kirschner, they’re not quite in the same league. Other key figures don’t get a mention.  When asked what was missing from the Tiger Teachers’ presentations at ResearchEd, I suggested, for starters, Baddeley and Hitch’s model of working memory. It’s been a dominant model for 40 years and has the rare distinction of being supported by later biological research. But it’s mentioned only in an endnote in Willingham’s Why Don’t Students Like School and in Daisy’s Seven Myths about Education. I recommended inviting Alan Baddeley to speak at ResearchEd – he’s a leading authority on memory after all.   One of the teachers said he’d never even heard of him. So why was that teacher doing a presentation on memory at a national education conference?

The Tiger Teachers also struggle with consensus. Joe Kirby emphasises the length of time an idea has been around and the number of studies that support it (pp.22-3), overlooking the fact that some ideas can dominate a field for decades, be supported by hundreds of studies and then turn out to be profoundly wrong; theories about how brains work are a case in point.   Scientific theory doesn’t rely on the quantity of supporting evidence; it relies on an evaluation of all relevant evidence – supporting and contradictory – and takes into account the quality of that evidence as well.  That’s why you need a substantial body of knowledge before you can evaluate it.

The big picture

For me, Battle Hymn painted a clearer picture of the Michaela Community School than I’d been able to put together from blog posts and visitors’ descriptions. It persuaded me that Michaela’s approach to behaviour management is about being explicit and consistent, rather than simply being ‘strict’. I think having a week’s induction for new students and staff (‘bootcamp’) is a great idea. A systematic, rigorous approach to knowledge is vital and learning by rote can be jolly useful. But for me, those positives were all undermined by the Tiger Teachers’ approach to their own knowledge.  Omitting key issues in discussions of Rousseau’s ideas, professional qualifications or the special circumstances of schools in coastal and rural areas, is one thing. Pontificating about cognitive science and then ignoring what it says is quite another.

I can understand why Tiger Teachers want to share concepts like the limited capacity of working memory and skills not being divorced from knowledge.  Those concepts make sense of problems and have transformed their teaching.  But for many Tiger Teachers, their knowledge of cognitive science appears to be based on a handful of poorly understood factoids acquired second or third hand from other teachers who don’t have a good grasp of the field either. Most teachers aren’t going to know much about cognitive science; but that’s why most teachers don’t do presentations about it at national conferences or go into print to share their flimsy knowledge about it.  Failing to acquire a substantial body of knowledge about cognitive science makes its comprehension, application, analysis, synthesis and evaluation impossible.  The Tiger Teachers’ disregard for principles they claim are crucial is inconsistent, disingenuous, likely to lead to significant problems, and sets a really bad example for pupils. The Tiger Teachers need to re-write some of the lyrics of their Battle Hymn.

The Tiger Teachers’ model of knowledge: what’s missing?

“If all else fails for Michaela at least we’re going to do a great line in radical evangelical street preachers.” Jonathan Porter, Head of Humanities at the Michaela Community School was referring to an impassioned speech from Katharine Birbalsingh, the school’s head teacher at the recent launch of their book, Battle Hymn of the Tiger Teachers: The Michaela Way.

Michaela Community School’s sometimes blistering critique of the English education system, coupled with its use of pedagogical methods abandoned by most schools decades ago, has drawn acclaim, criticism and condemnation. There’s a strong, shared narrative about the Michaela Way amongst the contributors to Battle Hymn. If I’ve understood it correctly, it goes like this:

There’s a crisis in the English education system due to progressive ideas that have dominated teacher training since the 1960s. Child-centred methods have undermined discipline. Poor behaviour and lack of respect makes it impossible for teachers to teach. Subject knowledge has been abandoned in favour of higher-level skills wrongly claimed to be transferable. The way to combat the decline is via strict discipline, teacher authority, a knowledge-based curriculum and didactic teaching.

Knowledge is power

“Knowledge is power” is the Michaela motto. Tiger Teachers are required to have extensive knowledge of their own subject area in order to teach their pupils. Pupils are considered to be novices and as such are expected to acquire a substantial amount of factual knowledge before they can develop higher-level subject-specific skills.

Given the emphasis on knowledge, you’d expect the Tiger Teachers to apply this model not only to their pupils, but to any subjects they are unfamiliar with.   But they don’t. It appears to apply only to what pupils are taught in school.

A couple of years ago at a ResearchEd conference, I queried some claims made about memory. I found myself being interrogated by three Tiger Teachers about what I thought was wrong with the model of memory presented. I said I didn’t think anything was wrong with it; the problem was what it missed out. There are other examples in Battle Hymn of missing key points. To illustrate, I’ve selected four. Here’s the first:

Rousseau

Rousseau is widely recognised as the originator of the progressive educational ideas so derided in the Michaela narrative.   If you were to rely on other Tiger Teachers for your information about Rousseau, you might picture him as a feckless Romantic philosopher who wandered the Alps fathering children whilst entertaining woolly, sentimental, unrealistic thoughts about their education.   You wouldn’t know that he argued in Émile, ou de L’Éducation not so much for the ‘inevitable goodness’ of children as Jonathan Porter claims (p.77), but that children (and adults) aren’t inherently bad – a view that flew in the face of the doctrine of original sin espoused by the Geneva Calvinism that Rousseau had rejected and the Catholicism he (temporarily) converted to soon after.

At the time, children were often expected to learn by rote factual information that was completely outside their experience, that was meaningless to them. Any resistance would have been seen as a sign of their fallen nature, rather than an understandable objection to a pointless exercise. Rousseau advocated that education work with nature, rather than against it. He claimed the natural world more accurately reflected the intentions of its Creator than the authoritarian, man-made religious institutions that exerted an extensive and often malign influence over people’s day-to-day lives.   Not surprisingly, Émile was promptly banned in Geneva and Paris.

Although Jonathan Porter alludes to the ‘Enlightenment project’ (p.77), he doesn’t mention Rousseau’s considerable influence in other spheres. The section of Émile that caused most consternation was entitled ‘The Creed of a Savoyard Priest’. It was the only part Voltaire thought worth publishing. In it, Rousseau tackles head-on Descartes’ proposition ‘I think, therefore I am’. He sets out the questions about perception, cognition, reasoning, consciousness, truth, free will and the existence of religions, that perplexed the thinkers of his day and that cognitive science has only recently begun to find answers to. I’m not defending Rousseau’s educational ideas, I think Voltaire’s description “a hodgepodge of a silly wet nurse in four volumes” isn’t far off the mark, but to draw valid conclusions from Rousseau’s ideas about education, you need to know why he was proposing them.

Battle Hymn isn’t a textbook or an academic treatise, so it would be unreasonable to expect it to tackle at length all the points it alludes to. But it is possible to signpost readers to relevant issues in a few words. There’s nothing technically wrong with the comments about Rousseau in Battle Hymn, or Robert Peal’s Progressively Worse (a core text for Tiger Teachers) or Daisy Christodoulou’s Seven Myths about Education (another core text); but what’s missed out could result in conclusions being drawn that aren’t supported by the evidence.

Teacher qualifications

Another example is teacher qualifications. Michaela teachers don’t think much of their initial teacher training (ITT); they claim it didn’t prepare them for the reality of teaching (p.167),  it indoctrinates teachers into a ‘single dogmatic orthodoxy’ (p.171), outcomes are poor (p.158), and CPD in schools is ‘more powerful’ (p.179). The conclusion is not that ITT needs a root-and-branch overhaul, but that it should be replaced with something else; in-school training or … no qualification at all. Sarah Clear says she’s “an unqualified teacher and proud” (p.166) and argues that although the PGCE might be a necessary precaution to prevent disaster, it doesn’t actually do that (p.179), so why bother with it?

Her view doesn’t quite square with Dani Quinn’s perspective on professional qualifications. Dani advocates competition in schools because there’s competition in the professional world. She says; “Like it or not, when it comes to performance, it is important to know who is the best” and cites surgeons and airline pilots as examples (p.133). But her comparison doesn’t quite hold water. Educational assessment tends to be norm-referenced (for reasons Daisy Christodoulou explores here) but assessments of professional performance are almost invariably criterion-referenced in order to safeguard minimum standards of technical knowledge and skill. But neither Dani nor Sarah mention norm-referenced and criterion-referenced assessment – which is odd, given Daisy Christodoulou’s involvement with Michaela. Again, there’s nothing technically wrong with what’s actually said about teacher qualifications; but the omission of relevant concepts increases the risk of reaching invalid conclusions.

Replicating Michaela

A third example is from the speech given by Katharine Birbalsingh at the book launch. It was triggered by this question: “How would you apply Michaela in primary? Could you replicate it in coastal areas or rural areas and how would that work?”

Katharine responds: “These are all systems and values that are universal. That could happen anywhere. Of course it could happen in a primary. I mean you just insist on higher standards with regard to the behaviour and you teach them didactically because everyone learns best when being taught didactically … You would do that with young children, you would do that with coastal children and you would do that with Yorkshire children. I don’t see why there would be a difference.” She then launches into her impassioned speech about teaching and its social consequences.

You could indeed apply Michaela’s systems, values, behavioural expectations and pedagogical approach anywhere. It doesn’t follow that you could apply them everywhere. Implicit in the question is whether the Michaela approach is scalable. It’s not clear whether Katharine misunderstood the question or answered the one she wanted to answer, but her response overlooks two important factors.

First, there’s parent/pupil choice. Brent might be one of the most deprived boroughs in the country, but it’s a deprived borough in a densely populated, prosperous city that has many schools and a good public transport system. If parents or pupils don’t like Michaela, they can go elsewhere. But in rural areas, for many pupils there’s only one accessible secondary school – there isn’t an elsewhere to go to.

Then there’s teacher recruitment. If you’re a bright young graduate, as most of the Michaela staff seem to be, the capital offers a vibrant social life and  a wide range of interesting alternative career alternatives should you decide to quit teaching. In a rural area there wouldn’t be the same opportunities.  Where I live, in a small market town in a sparsely populated county, recruitment in public sector services has been an ongoing challenge for many years.

Coastal towns have unique problems because they are bounded on at least one side by the sea. This makes them liminal spaces, geographically, economically and socially. Many are characterised by low-skilled, low-paid, seasonal employment and social issues different to those of an inner city. For teachers, the ‘life’ bit of the work-life balance in London would be very different from what they could expect in out-of-season Hartlepool.

Of course there’s no reason in principle why a replica Michaela shouldn’t transform the educational and economic prospects of coastal or rural areas.   But in practice, parent/pupil choice and teacher recruitment would be challenges that by definition Michaela hasn’t had to face because it’s a classic ‘special case’.  And it’s not safe to generalise from special cases. Again, there’s nothing technically wrong with what Katharine said about replicating Michaela; it’s what she didn’t say that’s key.  The same is true for the Tiger Teachers’ model of cognitive science, the subject of the next post.

References

Birbalsingh, K (2016).  Battle Hymn of the Tiger Teachers: The Michaela Way.  John Catt Educational.

Christodoulou, D (2014).  Seven Myths about Education.  Routledge.

Peal, R (2014).  Progressively Worse: The Burden of Bad Ideas in British Schools.  Civitas.

Rousseau, J-J (1974/1762).  Émile.  JM Dent.

getting the PISA scores under control

The results of the OECD’s 2015 Programme for International Student Assessment (PISA) were published a couple of weeks ago. The PISA assessment has measured the performance of 15 year-olds in Reading, Maths and Science every three years since 2000. I got the impression that teachers and academics (at least those using social media) were interested mainly in various aspects of the analysis. The news media, in contrast, focussed on the rankings. So did the OECD and politicians according to the BBC website. Andreas Schleicher of the OECD mentions Singapore ‘getting further ahead’ and John King US Education Secretary referred to the US ‘losing ground’.

What they are talking about are some single-digit changes in scores of almost 500 points. Although the PISA analysis might be informative, the rankings tell us very little. No one will get promoted or relegated as a consequence of their position in the PISA league table. Education is not football. What educational performance measures do have in common with all other performance measures – from football to manufacturing – is that performance is an outcome of causal factors. Change the causal factors and the performance will change.

common causes vs special causes

Many factors impact on performance. Some fluctuations are inevitable because of the variation inherent in raw materials, climatic conditions, equipment, human beings etc. Other changes in performance occur because a key causal factor has changed significantly. The challenge is in figuring out whether fluctuations are due to variation inherent in the process, or whether they are due to a change in the process itself – referred to as common causes and special causes, respectively.

The difference between common causes and special causes is important because there’s no point spending time and effort investigating common causes. Your steel output might have suffered because of a batch of inferior iron ore, your team might have been relegated because two key players sustained injuries, or your PISA score might have fallen a couple of points  due to a flu epidemic just before the PISA tests. It’s impossible to prevent such eventualities and even if you could, some other variation would crop up instead. However, if performance has improved or deteriorated following a change in supplier, strategy or structure you’d want to know whether or not that special cause has had a real impact.

spotting the difference

This was the challenge facing Walter A Shewhart, a physicist, engineer and statistician working for the Western Electric Company in the 1920s. Shewhart figured out a way of representing variations in performance so that quality controllers could see at a glance whether the variation was due to common causes or special causes. The representation is generally known as a control chart. I thought it might be interesting to plot some PISA results as a control chart, to see if changes in scores represented a real change or whether they were the fluctuations you’d expect to see due to variation inherent in the process.

If I’ve understood Shewhart’s reasoning correctly, it goes like this: Even if you don’t change your process, fluctuations in performance will occur due to the many different factors that impact on the effectiveness of your process. In the case of the UK’s PISA scores, each year similar students have learned and been assessed on very similar material, so the process remains unchanged; what the PISA scores measure is student performance.   But student performance can be affected by a huge number of factors; health, family circumstances, teacher recruitment, changes to the curriculum a decade earlier etc.

For statistical purposes, the variation caused by those multiple factors can be treated as random. (It isn’t truly random, but for most intents and purposes can be treated as if it is.) This means that over time, UK scores will form a normal distribution – most will be close to the mean, a few will be higher and a few will be lower. And we know quite a bit about the features of normal distributions.

Shewhart came up with a formula for calculating the upper and lower limits of the variation you’d expect to see as a result of common causes. If a score falls outside those limits, it’s worth investigating because it probably indicates a special cause. If it doesn’t, it isn’t worth investigating, because it’s likely to be due to common causes rather than a change to the process. Shewhart’s method is also useful for finding out whether or not an intervention has made a real difference to performance.  Donald Wheeler, in Understanding Variation: The key to managing chaos, cites the story of a manager spotting a change in performance outside the control limits and discovering it was due to trucks being loaded differently without the supervisor’s knowledge.

getting the PISA scores under control

I found it surprisingly difficult, given the high profile of the PISA results, to track down historical data and I couldn’t access it via the PISA website – if anyone knows of an accessible source I’d be grateful. Same goes for any errors in my calculations.  I decided to use the UK’s overall scores for Mathematics as an example. In 2000 and 2003 the UK assessments didn’t meet the PISA criteria, so the 2000 score is open to question and the 2003 score was omitted from the tables.

I’ve followed the method set out in Donald Wheeler’s book, which is short, accessible and full of examples. At first glance the formulae might look a bit complicated, but the maths involved is very straightforward. Year 6s might enjoy applying it to previous years’ SATs results.

Step 1: Plot the scores and find the mean.

year 2000* 2003* 2006 2009 2012 2015 mean (Xbar§)
UK maths score 529 495 492 494 492 500.4

Table 1: UK maths scores 2000-2015

* In 2000 and 2003 the UK assessments didn’t meet the PISA criteria, so the 2000 score is open to question and the 2003 score was omitted from the results.

§  I was chuffed when I figured out how to type a bar over a letter (the symbol for mean) but it got lost in translation to the blog post.

pisa-fig-1Fig 1: UK Maths scores and mean score

Step 2: Find the moving range (mR) values and calculate the mean. The moving range is the differences between consecutive scores, referred to as mR values.

year 2000 2003 2006 2009 2012 2015 mean

(R bar)

UK maths score 529 495 492 494 492
mR values 34 3 2 2 10.25

Table 2: moving range (mR values) 2000-2015

pisa-fig-2Fig 2: Differences between consecutive scores (mR values)

Step 3: Calculate the Upper Control Limit for the mR values (UCLR). To do this we multiply the mean of the mR values (Rbar) by 3.27.

UCLR = 3.27 x Rbar = 3.27 x 10.25 = 33.52

pisa-fig-3Fig 3: Differences between scores (mR values) showing upper control limit (UCLR)

Step 4: Calculate the Upper Natural Process Limit (UNPL) for the individual scores using the formula UNPL = + (2.66 x Xbar)

UNPL = + (2.66 x Xbar ) = 500.4 + (2.66 x 10.25) = 500.4 + 27.27 = 527.67

Step 5: Calculate the Lower Natural Process Limit (LNPL) for the individual scores using the formula LNPL = – (2.66 x Xbar )

LNPL = – (2.66 x Xbar) = 500.4 – (2.66 x 10.25) = 500.4 – 27.27 = 473.13

We can now plot the UK’s Maths scores showing the upper and lower natural process limits – the limits of the variation you’d expect to see as a result of common causes.

pisa-fig-4Fig 4: UK Maths scores showing upper and lower natural process limits

What Fig 4 shows is that the UK’s 2000 Maths score falls just outside the upper natural process limit, so even if the OECD hadn’t told us it was an anomalous result, we’d know that something different happened to the process in that year. You might think this is pretty obvious because there’s such a big difference between the 2000 score and all the others. But what if the score had been just a bit lower?  I put in some other numbers:

score  Xbar  Rbar UCLR UNPL LNPL
529 (actual) 500.4 10.25 33.52 527.67 473.13
520 498.6 8 26.16 519.88 477.32
510 496.6 5.5 17.99 511.23 481.97
500 494.6 3 9.81 502.58 486.62

Table 3: outcomes of alternative scores for year 2000

Table 3 shows if the score had been 520, it would still have been outside the natural process limits, but a score of 510 would have been within them.

pisa-fig-5 Fig 5: UK Maths scores showing upper and lower natural process limits for a year 2000 score of 510

ups, downs and targets

The ups and downs of test results are often viewed as more important than they really are; up two points good, down two points bad – even though a two-point fluctuation might be due to random variation.

The process control model has significant implications for target-setting too. Want to improve your score?  Then you need to work harder or smarter. Never mind the fact that students and teachers can work their socks off only to find that their performance is undermined by a crisis in recruiting maths teachers or a whole swathe of schools converting to academies. Working harder or smarter but ignoring natural variation supports what’s been called Ackoff’s proposition – that “almost every problem confronting our society is a result of the fact that our public policy makers are doing the wrong things and are trying to do them righter”.

To get tough on PISA scores we need to get tough on the causes of PISA scores.

 Reference

Wheeler, DJ (1993).  Understanding variation: The key to managing chaos.  SPC Press Inc, Knoxville, Tennessee.

Reforming the SEND system – for good

In the previous post, I claimed that teacher training and targets were two factors that explained why the current SEND system couldn’t work  –  and why it has never worked effectively.  In this post, I’ll explain my claims about teacher training and targets and suggest how the SEND system could become both effective and sustainable.

 Teacher training

For any system – education, health or social care – to meet the needs of a varied population, two ingredients are vital; expertise and flexibility. Practitioners need the knowledge and experience to deal with any needs they might encounter and the system has to be able to adapt to whatever needs arise.

Bizarrely, teachers have always been expected to teach the 98% or so of children who attend mainstream schools, but have only ever been trained to teach the 80% who don’t have SEN, not the 20% who do. And since funding was withdrawn for special education Master’s degrees in the mid-1980s, SEN expertise has gradually leached out of the education system as a whole as special education teachers have retired. It’s only since 2009 that new SENCOs (special educational needs co-ordinators) have been required to be qualified teachers, and only recent appointees are required to have SEN training. There is still a massive gap in SEND expertise within the education system. How can teachers teach children if they don’t know how to meet their educational needs?

Targets

Setting targets sounds like an obvious way to improve performance. You set the target, expect someone to meet it whatever that takes, and provide some sticks and carrots for their encouragement. Targets, accompanied by sticks and carrots, were part and parcel of the early education system but were abandoned because they didn’t work.  And as quality control researchers have been telling us since at least the 1920s, performance depends on the factors that contribute to it. In the current education system, the measure of school performance is actually pupil performance in SATs or GCSEs. But how children perform in tests is influenced by many factors; their health, family circumstances, life events, quality of teaching, their own learning etc. Schools have little or no control over most of those factors, so to measure school performance by pupil performance in tests is pointless.

Despite the evidence, the current education system still sets targets.  And the sticks and carrots expected to encourage schools to raise their (pupil) performance mean that there are no incentives for a school to invest resources in the education of students who are unlikely to improve the school’s test results. If students aren’t going to meet the ‘expected standard’ however hard they or the school try, why invest resources in them? Why not focus on the children likely to meet the ‘expected standard’ with a bit of extra effort.

So, teacher training and targets have been major factors in marginalising the education of children with SEND. But even if the government had a forehead-slapping moment, cried ‘How foolish we’ve been!’, required all teachers to be trained to teach all the children in their classes, and abandoned its ‘expected standards’ criteria, it would take years to transform the system into a SEND-friendly one. Children with SEND don’t have years to spare and their parents have to deal with the here and now. So what needs to be done?

Parents can’t police the system

This post was prompted by a recent conversation I had with a parent carer forum. The parent carer forum was of the opinion that parents with good knowledge of the national framework and their local offer can use that knowledge to get commissioners and providers to make suitable educational provision for children.

It’s certainly true that knowledge of the national framework and the local offer (however incomplete) can help. How effective it is at getting commissioners and providers to meet their statutory obligations is another matter. Since the new system was introduced, I’ve been told repeatedly that it’s improved outcomes for parents and children. Maybe – but I have yet to see any. What I have seen is parents who know the national framework backwards having to resort to mediation, tribunal, formal complaint, the Local Government Ombudsman and in some cases being advised that their only option is Judicial Review – exactly the kind of problems that prompted the revision of the SEN system in 2014.

Until I had the conversation with the parent carer forum, I’d assumed these hurdles were the unwanted and unintended consequences of flaws in legislation that had been rushed through (the pilot study didn’t finish until after the legislation came into force). Then the penny dropped. The only explanation that made sense was that individual parents challenging commissioners and providers is the government’s chosen method of enforcing the new legislation.

That’s a terrible way of enforcing legislation.  For many parents of children with SEND, it’s as much as they can do to hold the family together. To expect parents in already challenging circumstances to police a flawed system that was rushed through at a time when LAs are struggling with huge budget cuts is, to put vulnerable families in harm’s way. Not only is that strategy likely to fail to bring about compliance on the part of commissioners and providers, it’s morally reprehensible.  For 150 years, if a school failed a child parents have been able to appeal to school boards, independent governors or their LEA for support. Not any more. Parents (and children with SEND) are on their own.

What needs changing and who can change it?

The system still needs to change and if parents don’t change it no one else will, so what to do? Since my family entered the SEN ‘world’ 14 years ago, I’ve seen parents fighting lone battles with their LA; the same battles replicated hundreds, if not thousands, of times. I’ve seen parents new to the system set up support or campaign groups only to discover they are just one in a long line of support or campaign groups that have either burned out or at best brought about change that hasn’t actually made much difference on the ground.

What the individual parents and campaign groups have lacked is focus and organisation. I don’t mean they’ve been unfocussed or disorganised; some of them could focus and organise for England. And there’s no doubt that parent groups were instrumental in getting the SEND system changed. It’s rather that there’s been a lot of duplication of effort and the focus has been on single issues or fighting on all fronts at once rather than on the key points in the system that are causing the problems.

I think the key points are these;

  • Mainstream teachers should know how to teach all the children in mainstream schools.
  • Each child needs an education suitable for them as an individual rather than for the average child in their age group, as the law already requires.
  • Assessment and funding should be the responsibility of separate bodies – the new legislation didn’t do away with the LAs’ conflict of interest.
  • There should be an independent body (with teeth) responsible for implementation and compliance that should support parents in their dealings with commissioners and providers. Parents should not have to resort to legal action except in extreme cases.
  • Parents struggling with the system need more support than they are currently offered. A buddying system matching up parents in similar positions dealing with the same local authority might help. As would training in negotiation.

Much of the negotiation undertaken by individual parents and parent groups is with schools, LA officers or the DfE. And problems with the SEND system are generally seen not as being with the structure of the education system or the SEND legislation, but with implementation. But the problem runs deeper than implementation, and deeper than the SEND legislation. It lies with the structure of the education system as a whole, and with the market model espoused by successive governments. Instead of lobbying LA officers and DfE officials who are trying to implement the law as it stands, groups of parents should be lobbying their local councillors and MPs to ensure that teachers are suitably trained, arbitrary targets are abandoned, and responsibility for implementing the system is distributed more widely. These changes won’t require significant new legislation, but they might require a big shift in thinking.

 

 

 

A short history of special education

In 2006 a House of Commons Select Committee described the special educational needs system as ‘no longer fit for purpose’. By September 2014 a new system was in place. Two years on, it’s safe to say it hasn’t been an unmitigated success. To understand why the new system hasn’t worked – and indeed can’t work – it might help to take a look at the history of special educational needs and disability (SEND).

A short history of SEND

Education became compulsory in England in 1880. Some local school boards set up special schools or special classes within mainstream schools for physically ‘handicapped’* children, but provision was patchy. What took people by surprise was the number of mentally handicapped children who turned up to school.

At the time, teachers were expected to teach according to the official Code – essentially a core curriculum – and many schools in the fledgling national educational system were seriously under-resourced. Teachers were often untrained, paper was very expensive (hence the use of rote learning and slates) and many schools operated in conditions like the one below – with several classes in one room. They just weren’t equipped to cope with children with learning difficulties or disabilities.

Shepherd Street School in Preston in 1902

Shepherd Street School in Preston in 1902§

Two Royal Commissions were set up to investigate the education of handicapped children, and reported in 1889 and 1896 respectively. Both recommended the integration of the children in mainstream schools where possible and that special provision (classes or schools) should be made by school boards. The emphasis was on children acquiring vocational skills so they could earn a living. Those with the most severe mental handicap were deemed ‘ineducable’.

The Royal Commissions’ recommendations, and many others made over the next few decades, were clearly well-intentioned. Everybody wanted the best outcomes for the children. The challenge was how to get there. After WW2, concerns about the special education system increased. Parents felt they had little control, the number of pupils in special schools was rising, and children were still being institutionalised or marginalised from society. In 1973 Margaret Thatcher, then Education Secretary, commissioned a review of the education of handicapped children, led by Mary Warnock, whose Committee of Enquiry reported in 1978. A year later Margaret Thatcher became Prime Minister and some of the Warnock recommendations came into force in the Education Act 1981.

The Warnock report introduced a very different way of understanding ‘handicapped’ children. They were no longer seen as being different, but as having special educational needs – as did up to 20% of the school population. Special educational needs were defined in terms of the support children needed, rather than in terms of their physical or mental impairments. What was envisaged was that many children in special schools would gradually migrate to mainstream, supported by Statements of Special Educational Need. And mainstream schools would gradually become more inclusive, adapting their buildings, equipment and teaching methods to meet an ever wider range of educational needs. The new system might have worked well if the rest of the education system hadn’t changed around it.

Context is crucial; one size doesn’t fit all

The Warnock recommendations were made in the context of a very flexible education system. In 1981 Local Education Authorities (LEAs), schools and teachers had a great deal of autonomy in what was taught, how it was taught and when. That all changed with the 1988 Education Reform Act that heralded a compulsory National Curriculum, SATS and Ofsted. Central government essentially wrested control of education from local bodies, something that had been actively opposed for the previous 100 years – few people wanted education to become a political football.

The new education system was at heart a one-size-fits-all affair. Governments find one-size-fits-all systems very appealing. They look as if they are going to be cheaper to run because professional training, equipment and resources can be standardised and performance can be easily measured. Unfortunately for governments, human populations are not one-size, but are very varied. If a universal service is to meet the needs of a whole population, it won’t do that if it’s designed to meet only the needs of the average person. A stark choice faces those designing universal systems; either they can design a system that meets everybody’s needs and resource it properly, or they can design a system that doesn’t meet everybody’s needs and then spend years trying to sort out the ensuing muddle.

The 1880 education system was one-size-fits-all and the next century was spent sorting out the problems that resulted for handicapped children. There was a brief period after 1981 when the education system took a big step towards meeting the needs of all children, but seven years later it flipped back to one-size-fits-all. The last 30 years have been spent trying unsuccessfully to limit the damage for children with SEND.

So what’s the alternative? The answer isn’t further reform of the SEND system, because the causes of the problems don’t lie within the SEND system, but with the broader education system. Two key causes are teacher training and targets – the subjects of the next post.

*I’ve used the term ‘handicapped’ because it was in widespread use in the education system until the Warnock Committee changed the terminology.
§ © Harris Museum and Art Gallery http://www.mylearning.org/victorian-school-and-work-in-preston/images/1-3215/