The Tiger Teachers and cognitive science

Cognitive science is a key plank in the Tiger Teachers’ model of knowledge. If I’ve understood it properly the model looks something like this:

Cognitive science has discovered that working memory has limited capacity and duration, so pupils can’t process large amounts of novel information. If this information is secured in long-term memory via spaced, interleaved practice, students can recall it instantly whenever they need it, freeing up working memory for thinking.

What’s wrong with that? Nothing, as it stands. It’s what’s missing that’s the problem.

Subject knowledge

One of the Tiger Teachers’ beefs about the current education system is its emphasis on transferable skills. They point out that skills are not universally transferable, many are subject-specific, and in order to develop expertise in higher-level skills novices need a substantial amount of subject knowledge. Tiger Teachers’ pupils are expected to pay attention to experts (their teachers) and memorise a lot of facts before they can comprehend, apply, analyse, synthesise or evaluate. The model is broadly supported by cognitive science and the Tiger Teachers apply it rigorously to children. But not to themselves, it seems.

For most Tiger Teachers cognitive science will be an unfamiliar subject area. That makes them (like most of us) cognitive science novices. Obviously they don’t need to become experts in cognitive science to apply it to their educational practice, but they do need the key facts and concepts and a basic overview of the field. The overview is important because they need to know how the facts fit together and the limitations of how they can be applied.   But with a few honourable exceptions (Daisy Christodoulou, David Didau and Greg Ashman spring to mind – apologies if I’ve missed anyone out), many Tiger Teachers don’t appear to have even thought about acquiring expertise, key facts and concepts or an overview. As a consequence facts are misunderstood or overlooked, principles from other knowledge domains are applied inappropriately, and erroneous assumptions made about how science works. Here are some examples:

It’s a fact…

“Teachers’ brains work exactly the same way as pupils’” (p.177). No they don’t. Cognitive science (ironically) thinks that children’s brains begin by forming trillions of connections (synapses). Then through to early adulthood, synapses that aren’t used get pruned, which makes information processing more efficient. (There’s a good summary here.)  Pupils’ brains are as different to teachers’ brains as children’s bodies are different to adults’ bodies. Similarities don’t mean they’re identical.

Then there’s working memory. “As the cognitive scientist Daniel Willingham explains, we learn by transferring knowledge from the short-term memory to the long term memory” (p177). Well, kind of – if you assume that what Willingham explicitly describes as “just about the simplest model of the mind possible”  is an exhaustive model of memory. If you think that, you might conclude, wrongly, “the more knowledge we have in long-term memory, the more space we have in our working memory to process new information” (p.177). Or that “information cannot accumulate into long-term memory while working memory is being used” (p.36).

Long-term memory takes centre stage in the Tiger Teachers’ model of cognition. The only downside attributed to it is our tendency to forget things if we don’t revisit them (p.22). Other well-established characteristics of long-term memory – its unreliability, errors and biases – are simply overlooked, despite Daisy Christodoulou’s frequent citation of Daniel Kahneman whose work focused on those flaws.

With regard to transferable skills we’re told “cognitive scientist Herb Simon and his colleagues have cast doubt on the idea that there are any general or transferable cognitive skills” (p.17), when what they actually cast doubt on is the ideas that all skills are transferable or that none are.

The Michaela cognitive model is distinctly reductionist; “all there is to intelligence is the simple accrual and tuning of many small units of knowledge that in total produce complex cognition” (p.19). Then there’s “skills are simply just a composite of sequential knowledge – all skills can be broken down to irreducible pieces of knowledge” (p.161).

The statement about intelligence is a direct quote from John Anderson’s paper ‘A Simple Theory of Complex Cognition’ but Anderson isn’t credited, so you might not know he was talking about simple encodings of objects and transformations, and that by ‘intelligence’ he means how ants behave rather than IQ. I’ve looked at Daisy Christodoulou’s interpretation of Anderson’s model here.

The idea that intelligence and skills consist ‘simply just’ of units of knowledge ignores Anderson’s procedural rules and marginalises the role of the schema – the way people configure their knowledge. Joe Kirby mentions “procedural and substantive schemata” (p. 17), but seems to see them only in terms of how units of knowledge are configured for teaching purposes; “subject content knowledge is best organised into the most memorable schemata … chronological, cumulative schemata help pupils remember subject knowledge in the long term” (p.21). The concept of schemata as the way individuals, groups or entire academic disciplines configure their knowledge, that the same knowledge can be configured in different ways resulting in different meanings, or that configurations sometimes turn out to be profoundly wrong, doesn’t appear to feature in the Tiger Teachers’ model.

Skills: to transfer or not to transfer?

Tiger Teachers see higher-level skills as subject-specific. That hasn’t stopped them applying higher-level skills from one domain inappropriately to another. In her critique of Bloom’s taxonomy, Daisy Christodoulou describes it as a ‘metaphor’ for the relationship between knowledge and skills. She refers to two other metaphors; ED Hirsch’s scrambled egg and Joe Kirby’s double helix (Seven Myths p.21).  Daisy, Joe and ED teach English, and metaphors are an important feature in English literature. Scientists do use metaphors, but they use analogies more often, because in the natural world patterns often repeat themselves at different levels of abstraction. Daisy, Joe and ED are right to complain about Bloom’s taxonomy being used to justify divorcing skills from knowledge. And the taxonomy itself might be wrong or misleading.   But it is a taxonomy and it is based on an important scientific concept – levels of abstraction – so should be critiqued as such, not as if it were a device used by a novelist.

Not all evidence is equal

A major challenge for novices is what criteria they can use to decide whether or not factual information is valid. They can’t use their overview of a subject area if they don’t have one. They can’t weigh up one set of facts against another if they don’t know enough facts. So Tiger Teachers who are cognitive science novices have to fall back on the criteria ED Hirsch uses to evaluate psychology – the reputation of researchers and consensus. Those might be key criteria in evaluating English literature, but they’re secondary issues for scientific research, and for good reason.

Novices then have to figure out how to evaluate the reputation of researchers and consensus. The Tiger Teachers struggle with reputation. Daniel Willingham and Paul Kirschner are cited more frequently than Herb Simon, but with all due respect to Willingham and Kirschner, they’re not quite in the same league. Other key figures don’t get a mention.  When asked what was missing from the Tiger Teachers’ presentations at ResearchEd, I suggested, for starters, Baddeley and Hitch’s model of working memory. It’s been a dominant model for 40 years and has the rare distinction of being supported by later biological research. But it’s mentioned only in an endnote in Willingham’s Why Don’t Students Like School and in Daisy’s Seven Myths about Education. I recommended inviting Alan Baddeley to speak at ResearchEd – he’s a leading authority on memory after all.   One of the teachers said he’d never even heard of him. So why was that teacher doing a presentation on memory at a national education conference?

The Tiger Teachers also struggle with consensus. Joe Kirby emphasises the length of time an idea has been around and the number of studies that support it (pp.22-3), overlooking the fact that some ideas can dominate a field for decades, be supported by hundreds of studies and then turn out to be profoundly wrong; theories about how brains work are a case in point.   Scientific theory doesn’t rely on the quantity of supporting evidence; it relies on an evaluation of all relevant evidence – supporting and contradictory – and takes into account the quality of that evidence as well.  That’s why you need a substantial body of knowledge before you can evaluate it.

The big picture

For me, Battle Hymn painted a clearer picture of the Michaela Community School than I’d been able to put together from blog posts and visitors’ descriptions. It persuaded me that Michaela’s approach to behaviour management is about being explicit and consistent, rather than simply being ‘strict’. I think having a week’s induction for new students and staff (‘bootcamp’) is a great idea. A systematic, rigorous approach to knowledge is vital and learning by rote can be jolly useful. But for me, those positives were all undermined by the Tiger Teachers’ approach to their own knowledge.  Omitting key issues in discussions of Rousseau’s ideas, professional qualifications or the special circumstances of schools in coastal and rural areas, is one thing. Pontificating about cognitive science and then ignoring what it says is quite another.

I can understand why Tiger Teachers want to share concepts like the limited capacity of working memory and skills not being divorced from knowledge.  Those concepts make sense of problems and have transformed their teaching.  But for many Tiger Teachers, their knowledge of cognitive science appears to be based on a handful of poorly understood factoids acquired second or third hand from other teachers who don’t have a good grasp of the field either. Most teachers aren’t going to know much about cognitive science; but that’s why most teachers don’t do presentations about it at national conferences or go into print to share their flimsy knowledge about it.  Failing to acquire a substantial body of knowledge about cognitive science makes its comprehension, application, analysis, synthesis and evaluation impossible.  The Tiger Teachers’ disregard for principles they claim are crucial is inconsistent, disingenuous, likely to lead to significant problems, and sets a really bad example for pupils. The Tiger Teachers need to re-write some of the lyrics of their Battle Hymn.

The Tiger Teachers’ model of knowledge: what’s missing?

“If all else fails for Michaela at least we’re going to do a great line in radical evangelical street preachers.” Jonathan Porter, Head of Humanities at the Michaela Community School was referring to an impassioned speech from Katharine Birbalsingh, the school’s head teacher at the recent launch of their book, Battle Hymn of the Tiger Teachers: The Michaela Way.

Michaela Community School’s sometimes blistering critique of the English education system, coupled with its use of pedagogical methods abandoned by most schools decades ago, has drawn acclaim, criticism and condemnation. There’s a strong, shared narrative about the Michaela Way amongst the contributors to Battle Hymn. If I’ve understood it correctly, it goes like this:

There’s a crisis in the English education system due to progressive ideas that have dominated teacher training since the 1960s. Child-centred methods have undermined discipline. Poor behaviour and lack of respect makes it impossible for teachers to teach. Subject knowledge has been abandoned in favour of higher-level skills wrongly claimed to be transferable. The way to combat the decline is via strict discipline, teacher authority, a knowledge-based curriculum and didactic teaching.

Knowledge is power

“Knowledge is power” is the Michaela motto. Tiger Teachers are required to have extensive knowledge of their own subject area in order to teach their pupils. Pupils are considered to be novices and as such are expected to acquire a substantial amount of factual knowledge before they can develop higher-level subject-specific skills.

Given the emphasis on knowledge, you’d expect the Tiger Teachers to apply this model not only to their pupils, but to any subjects they are unfamiliar with.   But they don’t. It appears to apply only to what pupils are taught in school.

A couple of years ago at a ResearchEd conference, I queried some claims made about memory. I found myself being interrogated by three Tiger Teachers about what I thought was wrong with the model of memory presented. I said I didn’t think anything was wrong with it; the problem was what it missed out. There are other examples in Battle Hymn of missing key points. To illustrate, I’ve selected four. Here’s the first:

Rousseau

Rousseau is widely recognised as the originator of the progressive educational ideas so derided in the Michaela narrative.   If you were to rely on other Tiger Teachers for your information about Rousseau, you might picture him as a feckless Romantic philosopher who wandered the Alps fathering children whilst entertaining woolly, sentimental, unrealistic thoughts about their education.   You wouldn’t know that he argued in Émile, ou de L’Éducation not so much for the ‘inevitable goodness’ of children as Jonathan Porter claims (p.77), but that children (and adults) aren’t inherently bad – a view that flew in the face of the doctrine of original sin espoused by the Geneva Calvinism that Rousseau had rejected and the Catholicism he (temporarily) converted to soon after.

At the time, children were often expected to learn by rote factual information that was completely outside their experience, that was meaningless to them. Any resistance would have been seen as a sign of their fallen nature, rather than an understandable objection to a pointless exercise. Rousseau advocated that education work with nature, rather than against it. He claimed the natural world more accurately reflected the intentions of its Creator than the authoritarian, man-made religious institutions that exerted an extensive and often malign influence over people’s day-to-day lives.   Not surprisingly, Émile was promptly banned in Geneva and Paris.

Although Jonathan Porter alludes to the ‘Enlightenment project’ (p.77), he doesn’t mention Rousseau’s considerable influence in other spheres. The section of Émile that caused most consternation was entitled ‘The Creed of a Savoyard Priest’. It was the only part Voltaire thought worth publishing. In it, Rousseau tackles head-on Descartes’ proposition ‘I think, therefore I am’. He sets out the questions about perception, cognition, reasoning, consciousness, truth, free will and the existence of religions, that perplexed the thinkers of his day and that cognitive science has only recently begun to find answers to. I’m not defending Rousseau’s educational ideas, I think Voltaire’s description “a hodgepodge of a silly wet nurse in four volumes” isn’t far off the mark, but to draw valid conclusions from Rousseau’s ideas about education, you need to know why he was proposing them.

Battle Hymn isn’t a textbook or an academic treatise, so it would be unreasonable to expect it to tackle at length all the points it alludes to. But it is possible to signpost readers to relevant issues in a few words. There’s nothing technically wrong with the comments about Rousseau in Battle Hymn, or Robert Peal’s Progressively Worse (a core text for Tiger Teachers) or Daisy Christodoulou’s Seven Myths about Education (another core text); but what’s missed out could result in conclusions being drawn that aren’t supported by the evidence.

Teacher qualifications

Another example is teacher qualifications. Michaela teachers don’t think much of their initial teacher training (ITT); they claim it didn’t prepare them for the reality of teaching (p.167),  it indoctrinates teachers into a ‘single dogmatic orthodoxy’ (p.171), outcomes are poor (p.158), and CPD in schools is ‘more powerful’ (p.179). The conclusion is not that ITT needs a root-and-branch overhaul, but that it should be replaced with something else; in-school training or … no qualification at all. Sarah Clear says she’s “an unqualified teacher and proud” (p.166) and argues that although the PGCE might be a necessary precaution to prevent disaster, it doesn’t actually do that (p.179), so why bother with it?

Her view doesn’t quite square with Dani Quinn’s perspective on professional qualifications. Dani advocates competition in schools because there’s competition in the professional world. She says; “Like it or not, when it comes to performance, it is important to know who is the best” and cites surgeons and airline pilots as examples (p.133). But her comparison doesn’t quite hold water. Educational assessment tends to be norm-referenced (for reasons Daisy Christodoulou explores here) but assessments of professional performance are almost invariably criterion-referenced in order to safeguard minimum standards of technical knowledge and skill. But neither Dani nor Sarah mention norm-referenced and criterion-referenced assessment – which is odd, given Daisy Christodoulou’s involvement with Michaela. Again, there’s nothing technically wrong with what’s actually said about teacher qualifications; but the omission of relevant concepts increases the risk of reaching invalid conclusions.

Replicating Michaela

A third example is from the speech given by Katharine Birbalsingh at the book launch. It was triggered by this question: “How would you apply Michaela in primary? Could you replicate it in coastal areas or rural areas and how would that work?”

Katharine responds: “These are all systems and values that are universal. That could happen anywhere. Of course it could happen in a primary. I mean you just insist on higher standards with regard to the behaviour and you teach them didactically because everyone learns best when being taught didactically … You would do that with young children, you would do that with coastal children and you would do that with Yorkshire children. I don’t see why there would be a difference.” She then launches into her impassioned speech about teaching and its social consequences.

You could indeed apply Michaela’s systems, values, behavioural expectations and pedagogical approach anywhere. It doesn’t follow that you could apply them everywhere. Implicit in the question is whether the Michaela approach is scalable. It’s not clear whether Katharine misunderstood the question or answered the one she wanted to answer, but her response overlooks two important factors.

First, there’s parent/pupil choice. Brent might be one of the most deprived boroughs in the country, but it’s a deprived borough in a densely populated, prosperous city that has many schools and a good public transport system. If parents or pupils don’t like Michaela, they can go elsewhere. But in rural areas, for many pupils there’s only one accessible secondary school – there isn’t an elsewhere to go to.

Then there’s teacher recruitment. If you’re a bright young graduate, as most of the Michaela staff seem to be, the capital offers a vibrant social life and  a wide range of interesting alternative career alternatives should you decide to quit teaching. In a rural area there wouldn’t be the same opportunities.  Where I live, in a small market town in a sparsely populated county, recruitment in public sector services has been an ongoing challenge for many years.

Coastal towns have unique problems because they are bounded on at least one side by the sea. This makes them liminal spaces, geographically, economically and socially. Many are characterised by low-skilled, low-paid, seasonal employment and social issues different to those of an inner city. For teachers, the ‘life’ bit of the work-life balance in London would be very different from what they could expect in out-of-season Hartlepool.

Of course there’s no reason in principle why a replica Michaela shouldn’t transform the educational and economic prospects of coastal or rural areas.   But in practice, parent/pupil choice and teacher recruitment would be challenges that by definition Michaela hasn’t had to face because it’s a classic ‘special case’.  And it’s not safe to generalise from special cases. Again, there’s nothing technically wrong with what Katharine said about replicating Michaela; it’s what she didn’t say that’s key.  The same is true for the Tiger Teachers’ model of cognitive science, the subject of the next post.

References

Birbalsingh, K (2016).  Battle Hymn of the Tiger Teachers: The Michaela Way.  John Catt Educational.

Christodoulou, D (2014).  Seven Myths about Education.  Routledge.

Peal, R (2014).  Progressively Worse: The Burden of Bad Ideas in British Schools.  Civitas.

Rousseau, J-J (1974/1762).  Émile.  JM Dent.

getting the PISA scores under control

The results of the OECD’s 2015 Programme for International Student Assessment (PISA) were published a couple of weeks ago. The PISA assessment has measured the performance of 15 year-olds in Reading, Maths and Science every three years since 2000. I got the impression that teachers and academics (at least those using social media) were interested mainly in various aspects of the analysis. The news media, in contrast, focussed on the rankings. So did the OECD and politicians according to the BBC website. Andreas Schleicher of the OECD mentions Singapore ‘getting further ahead’ and John King US Education Secretary referred to the US ‘losing ground’.

What they are talking about are some single-digit changes in scores of almost 500 points. Although the PISA analysis might be informative, the rankings tell us very little. No one will get promoted or relegated as a consequence of their position in the PISA league table. Education is not football. What educational performance measures do have in common with all other performance measures – from football to manufacturing – is that performance is an outcome of causal factors. Change the causal factors and the performance will change.

common causes vs special causes

Many factors impact on performance. Some fluctuations are inevitable because of the variation inherent in raw materials, climatic conditions, equipment, human beings etc. Other changes in performance occur because a key causal factor has changed significantly. The challenge is in figuring out whether fluctuations are due to variation inherent in the process, or whether they are due to a change in the process itself – referred to as common causes and special causes, respectively.

The difference between common causes and special causes is important because there’s no point spending time and effort investigating common causes. Your steel output might have suffered because of a batch of inferior iron ore, your team might have been relegated because two key players sustained injuries, or your PISA score might have fallen a couple of points  due to a flu epidemic just before the PISA tests. It’s impossible to prevent such eventualities and even if you could, some other variation would crop up instead. However, if performance has improved or deteriorated following a change in supplier, strategy or structure you’d want to know whether or not that special cause has had a real impact.

spotting the difference

This was the challenge facing Walter A Shewhart, a physicist, engineer and statistician working for the Western Electric Company in the 1920s. Shewhart figured out a way of representing variations in performance so that quality controllers could see at a glance whether the variation was due to common causes or special causes. The representation is generally known as a control chart. I thought it might be interesting to plot some PISA results as a control chart, to see if changes in scores represented a real change or whether they were the fluctuations you’d expect to see due to variation inherent in the process.

If I’ve understood Shewhart’s reasoning correctly, it goes like this: Even if you don’t change your process, fluctuations in performance will occur due to the many different factors that impact on the effectiveness of your process. In the case of the UK’s PISA scores, each year similar students have learned and been assessed on very similar material, so the process remains unchanged; what the PISA scores measure is student performance.   But student performance can be affected by a huge number of factors; health, family circumstances, teacher recruitment, changes to the curriculum a decade earlier etc.

For statistical purposes, the variation caused by those multiple factors can be treated as random. (It isn’t truly random, but for most intents and purposes can be treated as if it is.) This means that over time, UK scores will form a normal distribution – most will be close to the mean, a few will be higher and a few will be lower. And we know quite a bit about the features of normal distributions.

Shewhart came up with a formula for calculating the upper and lower limits of the variation you’d expect to see as a result of common causes. If a score falls outside those limits, it’s worth investigating because it probably indicates a special cause. If it doesn’t, it isn’t worth investigating, because it’s likely to be due to common causes rather than a change to the process. Shewhart’s method is also useful for finding out whether or not an intervention has made a real difference to performance.  Donald Wheeler, in Understanding Variation: The key to managing chaos, cites the story of a manager spotting a change in performance outside the control limits and discovering it was due to trucks being loaded differently without the supervisor’s knowledge.

getting the PISA scores under control

I found it surprisingly difficult, given the high profile of the PISA results, to track down historical data and I couldn’t access it via the PISA website – if anyone knows of an accessible source I’d be grateful. Same goes for any errors in my calculations.  I decided to use the UK’s overall scores for Mathematics as an example. In 2000 and 2003 the UK assessments didn’t meet the PISA criteria, so the 2000 score is open to question and the 2003 score was omitted from the tables.

I’ve followed the method set out in Donald Wheeler’s book, which is short, accessible and full of examples. At first glance the formulae might look a bit complicated, but the maths involved is very straightforward. Year 6s might enjoy applying it to previous years’ SATs results.

Step 1: Plot the scores and find the mean.

year 2000* 2003* 2006 2009 2012 2015 mean (Xbar§)
UK maths score 529 495 492 494 492 500.4

Table 1: UK maths scores 2000-2015

* In 2000 and 2003 the UK assessments didn’t meet the PISA criteria, so the 2000 score is open to question and the 2003 score was omitted from the results.

§  I was chuffed when I figured out how to type a bar over a letter (the symbol for mean) but it got lost in translation to the blog post.

pisa-fig-1Fig 1: UK Maths scores and mean score

Step 2: Find the moving range (mR) values and calculate the mean. The moving range is the differences between consecutive scores, referred to as mR values.

year 2000 2003 2006 2009 2012 2015 mean

(R bar)

UK maths score 529 495 492 494 492
mR values 34 3 2 2 10.25

Table 2: moving range (mR values) 2000-2015

pisa-fig-2Fig 2: Differences between consecutive scores (mR values)

Step 3: Calculate the Upper Control Limit for the mR values (UCLR). To do this we multiply the mean of the mR values (Rbar) by 3.27.

UCLR = 3.27 x Rbar = 3.27 x 10.25 = 33.52

pisa-fig-3Fig 3: Differences between scores (mR values) showing upper control limit (UCLR)

Step 4: Calculate the Upper Natural Process Limit (UNPL) for the individual scores using the formula UNPL = + (2.66 x Xbar)

UNPL = + (2.66 x Xbar ) = 500.4 + (2.66 x 10.25) = 500.4 + 27.27 = 527.67

Step 5: Calculate the Lower Natural Process Limit (LNPL) for the individual scores using the formula LNPL = – (2.66 x Xbar )

LNPL = – (2.66 x Xbar) = 500.4 – (2.66 x 10.25) = 500.4 – 27.27 = 473.13

We can now plot the UK’s Maths scores showing the upper and lower natural process limits – the limits of the variation you’d expect to see as a result of common causes.

pisa-fig-4Fig 4: UK Maths scores showing upper and lower natural process limits

What Fig 4 shows is that the UK’s 2000 Maths score falls just outside the upper natural process limit, so even if the OECD hadn’t told us it was an anomalous result, we’d know that something different happened to the process in that year. You might think this is pretty obvious because there’s such a big difference between the 2000 score and all the others. But what if the score had been just a bit lower?  I put in some other numbers:

score  Xbar  Rbar UCLR UNPL LNPL
529 (actual) 500.4 10.25 33.52 527.67 473.13
520 498.6 8 26.16 519.88 477.32
510 496.6 5.5 17.99 511.23 481.97
500 494.6 3 9.81 502.58 486.62

Table 3: outcomes of alternative scores for year 2000

Table 3 shows if the score had been 520, it would still have been outside the natural process limits, but a score of 510 would have been within them.

pisa-fig-5 Fig 5: UK Maths scores showing upper and lower natural process limits for a year 2000 score of 510

ups, downs and targets

The ups and downs of test results are often viewed as more important than they really are; up two points good, down two points bad – even though a two-point fluctuation might be due to random variation.

The process control model has significant implications for target-setting too. Want to improve your score?  Then you need to work harder or smarter. Never mind the fact that students and teachers can work their socks off only to find that their performance is undermined by a crisis in recruiting maths teachers or a whole swathe of schools converting to academies. Working harder or smarter but ignoring natural variation supports what’s been called Ackoff’s proposition – that “almost every problem confronting our society is a result of the fact that our public policy makers are doing the wrong things and are trying to do them righter”.

To get tough on PISA scores we need to get tough on the causes of PISA scores.

 Reference

Wheeler, DJ (1993).  Understanding variation: The key to managing chaos.  SPC Press Inc, Knoxville, Tennessee.

Reforming the SEND system – for good

In the previous post, I claimed that teacher training and targets were two factors that explained why the current SEND system couldn’t work  –  and why it has never worked effectively.  In this post, I’ll explain my claims about teacher training and targets and suggest how the SEND system could become both effective and sustainable.

 Teacher training

For any system – education, health or social care – to meet the needs of a varied population, two ingredients are vital; expertise and flexibility. Practitioners need the knowledge and experience to deal with any needs they might encounter and the system has to be able to adapt to whatever needs arise.

Bizarrely, teachers have always been expected to teach the 98% or so of children who attend mainstream schools, but have only ever been trained to teach the 80% who don’t have SEN, not the 20% who do. And since funding was withdrawn for special education Master’s degrees in the mid-1980s, SEN expertise has gradually leached out of the education system as a whole as special education teachers have retired. It’s only since 2009 that new SENCOs (special educational needs co-ordinators) have been required to be qualified teachers, and only recent appointees are required to have SEN training. There is still a massive gap in SEND expertise within the education system. How can teachers teach children if they don’t know how to meet their educational needs?

Targets

Setting targets sounds like an obvious way to improve performance. You set the target, expect someone to meet it whatever that takes, and provide some sticks and carrots for their encouragement. Targets, accompanied by sticks and carrots, were part and parcel of the early education system but were abandoned because they didn’t work.  And as quality control researchers have been telling us since at least the 1920s, performance depends on the factors that contribute to it. In the current education system, the measure of school performance is actually pupil performance in SATs or GCSEs. But how children perform in tests is influenced by many factors; their health, family circumstances, life events, quality of teaching, their own learning etc. Schools have little or no control over most of those factors, so to measure school performance by pupil performance in tests is pointless.

Despite the evidence, the current education system still sets targets.  And the sticks and carrots expected to encourage schools to raise their (pupil) performance mean that there are no incentives for a school to invest resources in the education of students who are unlikely to improve the school’s test results. If students aren’t going to meet the ‘expected standard’ however hard they or the school try, why invest resources in them? Why not focus on the children likely to meet the ‘expected standard’ with a bit of extra effort.

So, teacher training and targets have been major factors in marginalising the education of children with SEND. But even if the government had a forehead-slapping moment, cried ‘How foolish we’ve been!’, required all teachers to be trained to teach all the children in their classes, and abandoned its ‘expected standards’ criteria, it would take years to transform the system into a SEND-friendly one. Children with SEND don’t have years to spare and their parents have to deal with the here and now. So what needs to be done?

Parents can’t police the system

This post was prompted by a recent conversation I had with a parent carer forum. The parent carer forum was of the opinion that parents with good knowledge of the national framework and their local offer can use that knowledge to get commissioners and providers to make suitable educational provision for children.

It’s certainly true that knowledge of the national framework and the local offer (however incomplete) can help. How effective it is at getting commissioners and providers to meet their statutory obligations is another matter. Since the new system was introduced, I’ve been told repeatedly that it’s improved outcomes for parents and children. Maybe – but I have yet to see any. What I have seen is parents who know the national framework backwards having to resort to mediation, tribunal, formal complaint, the Local Government Ombudsman and in some cases being advised that their only option is Judicial Review – exactly the kind of problems that prompted the revision of the SEN system in 2014.

Until I had the conversation with the parent carer forum, I’d assumed these hurdles were the unwanted and unintended consequences of flaws in legislation that had been rushed through (the pilot study didn’t finish until after the legislation came into force). Then the penny dropped. The only explanation that made sense was that individual parents challenging commissioners and providers is the government’s chosen method of enforcing the new legislation.

That’s a terrible way of enforcing legislation.  For many parents of children with SEND, it’s as much as they can do to hold the family together. To expect parents in already challenging circumstances to police a flawed system that was rushed through at a time when LAs are struggling with huge budget cuts is, to put vulnerable families in harm’s way. Not only is that strategy likely to fail to bring about compliance on the part of commissioners and providers, it’s morally reprehensible.  For 150 years, if a school failed a child parents have been able to appeal to school boards, independent governors or their LEA for support. Not any more. Parents (and children with SEND) are on their own.

What needs changing and who can change it?

The system still needs to change and if parents don’t change it no one else will, so what to do? Since my family entered the SEN ‘world’ 14 years ago, I’ve seen parents fighting lone battles with their LA; the same battles replicated hundreds, if not thousands, of times. I’ve seen parents new to the system set up support or campaign groups only to discover they are just one in a long line of support or campaign groups that have either burned out or at best brought about change that hasn’t actually made much difference on the ground.

What the individual parents and campaign groups have lacked is focus and organisation. I don’t mean they’ve been unfocussed or disorganised; some of them could focus and organise for England. And there’s no doubt that parent groups were instrumental in getting the SEND system changed. It’s rather that there’s been a lot of duplication of effort and the focus has been on single issues or fighting on all fronts at once rather than on the key points in the system that are causing the problems.

I think the key points are these;

  • Mainstream teachers should know how to teach all the children in mainstream schools.
  • Each child needs an education suitable for them as an individual rather than for the average child in their age group, as the law already requires.
  • Assessment and funding should be the responsibility of separate bodies – the new legislation didn’t do away with the LAs’ conflict of interest.
  • There should be an independent body (with teeth) responsible for implementation and compliance that should support parents in their dealings with commissioners and providers. Parents should not have to resort to legal action except in extreme cases.
  • Parents struggling with the system need more support than they are currently offered. A buddying system matching up parents in similar positions dealing with the same local authority might help. As would training in negotiation.

Much of the negotiation undertaken by individual parents and parent groups is with schools, LA officers or the DfE. And problems with the SEND system are generally seen not as being with the structure of the education system or the SEND legislation, but with implementation. But the problem runs deeper than implementation, and deeper than the SEND legislation. It lies with the structure of the education system as a whole, and with the market model espoused by successive governments. Instead of lobbying LA officers and DfE officials who are trying to implement the law as it stands, groups of parents should be lobbying their local councillors and MPs to ensure that teachers are suitably trained, arbitrary targets are abandoned, and responsibility for implementing the system is distributed more widely. These changes won’t require significant new legislation, but they might require a big shift in thinking.

 

 

 

A short history of special education

In 2006 a House of Commons Select Committee described the special educational needs system as ‘no longer fit for purpose’. By September 2014 a new system was in place. Two years on, it’s safe to say it hasn’t been an unmitigated success. To understand why the new system hasn’t worked – and indeed can’t work – it might help to take a look at the history of special educational needs and disability (SEND).

A short history of SEND

Education became compulsory in England in 1880. Some local school boards set up special schools or special classes within mainstream schools for physically ‘handicapped’* children, but provision was patchy. What took people by surprise was the number of mentally handicapped children who turned up to school.

At the time, teachers were expected to teach according to the official Code – essentially a core curriculum – and many schools in the fledgling national educational system were seriously under-resourced. Teachers were often untrained, paper was very expensive (hence the use of rote learning and slates) and many schools operated in conditions like the one below – with several classes in one room. They just weren’t equipped to cope with children with learning difficulties or disabilities.

Shepherd Street School in Preston in 1902

Shepherd Street School in Preston in 1902§

Two Royal Commissions were set up to investigate the education of handicapped children, and reported in 1889 and 1896 respectively. Both recommended the integration of the children in mainstream schools where possible and that special provision (classes or schools) should be made by school boards. The emphasis was on children acquiring vocational skills so they could earn a living. Those with the most severe mental handicap were deemed ‘ineducable’.

The Royal Commissions’ recommendations, and many others made over the next few decades, were clearly well-intentioned. Everybody wanted the best outcomes for the children. The challenge was how to get there. After WW2, concerns about the special education system increased. Parents felt they had little control, the number of pupils in special schools was rising, and children were still being institutionalised or marginalised from society. In 1973 Margaret Thatcher, then Education Secretary, commissioned a review of the education of handicapped children, led by Mary Warnock, whose Committee of Enquiry reported in 1978. A year later Margaret Thatcher became Prime Minister and some of the Warnock recommendations came into force in the Education Act 1981.

The Warnock report introduced a very different way of understanding ‘handicapped’ children. They were no longer seen as being different, but as having special educational needs – as did up to 20% of the school population. Special educational needs were defined in terms of the support children needed, rather than in terms of their physical or mental impairments. What was envisaged was that many children in special schools would gradually migrate to mainstream, supported by Statements of Special Educational Need. And mainstream schools would gradually become more inclusive, adapting their buildings, equipment and teaching methods to meet an ever wider range of educational needs. The new system might have worked well if the rest of the education system hadn’t changed around it.

Context is crucial; one size doesn’t fit all

The Warnock recommendations were made in the context of a very flexible education system. In 1981 Local Education Authorities (LEAs), schools and teachers had a great deal of autonomy in what was taught, how it was taught and when. That all changed with the 1988 Education Reform Act that heralded a compulsory National Curriculum, SATS and Ofsted. Central government essentially wrested control of education from local bodies, something that had been actively opposed for the previous 100 years – few people wanted education to become a political football.

The new education system was at heart a one-size-fits-all affair. Governments find one-size-fits-all systems very appealing. They look as if they are going to be cheaper to run because professional training, equipment and resources can be standardised and performance can be easily measured. Unfortunately for governments, human populations are not one-size, but are very varied. If a universal service is to meet the needs of a whole population, it won’t do that if it’s designed to meet only the needs of the average person. A stark choice faces those designing universal systems; either they can design a system that meets everybody’s needs and resource it properly, or they can design a system that doesn’t meet everybody’s needs and then spend years trying to sort out the ensuing muddle.

The 1880 education system was one-size-fits-all and the next century was spent sorting out the problems that resulted for handicapped children. There was a brief period after 1981 when the education system took a big step towards meeting the needs of all children, but seven years later it flipped back to one-size-fits-all. The last 30 years have been spent trying unsuccessfully to limit the damage for children with SEND.

So what’s the alternative? The answer isn’t further reform of the SEND system, because the causes of the problems don’t lie within the SEND system, but with the broader education system. Two key causes are teacher training and targets – the subjects of the next post.

*I’ve used the term ‘handicapped’ because it was in widespread use in the education system until the Warnock Committee changed the terminology.
§ © Harris Museum and Art Gallery http://www.mylearning.org/victorian-school-and-work-in-preston/images/1-3215/

We’re all different

We’re all different. Tiny variations in our DNA before and at conception. Our parents smoked/didn’t smoke, drank/didn’t drink, followed ideal/inadequate diets, enjoyed robust health or had food intolerances, allergies, viral infections. We were brought up in middle class suburbia/a tower block and attended inadequate/outstanding schools. All those factors contribute to who we are and what we are capable of achieving.

That variation, inherent in all biological organisms, is vital to our survival as a species. Without it, we couldn’t adapt to a changing environment or form communities that successfully protect us. The downside of that inherent variation is that some of us draw a short straw. Some variations mean we don’t make it through childhood, have lifelong health problems or die young. Or that we become what Katie Ashford, SENCO at Michaela Community School, Wembley calls the ‘weakest pupils‘.

Although the factors that contribute to our development aren’t, strictly speaking, random, they are so many and so varied, they might as well be random. That means that in a large population, the measurement of any characteristic affected by many factors – height, blood pressure, intelligence, reading ability – will form what’s known as a normal distribution; the familiar bell curve.

 The bell curve

If a particular characteristic forms a bell-shaped distribution, that allows us to make certain predictions about a large population. For that characteristic, 50% of the population will score above average and 50% below average; there will be relatively few people who are actually average. We’ll know that around 70% of the population will score fairly close to average, around 25% noticeably above or below it, and around 5% considerably higher or lower. That’s why medical reference ranges for various characteristics are based on the upper and lower measurements for 95% of the population; if your blood glucose levels or thyroid function is in the lowest or highest 2.5%, you’re likely to have a real problem, rather than a normal variation.

So in terms of general ability that means around 2.5% of the population will be in a position to decide whether they’d rather be an Olympic athlete, a brain surgeon or Prime Minister (or all three), whereas another 2.5% will find everyday life challenging.

What does a normal distribution mean for education? Educational attainment is affected by many causal factors, so by bizarre coincidence the attainment of 50% of school pupils is above average, and 50% below it. Around 20% of pupils have ‘special educational needs’ and around 2.5% will have educational needs that are significant enough to warrant a Statement of Special Educational Needs (recently replaced by Education Health and Care Plans).

Special educational needs

In 1978, the Warnock report pointed out that based on historical data, up to 20% of school pupils would probably have special educational needs at some point in their school career. ‘Special educational needs’ has a precise but relative meaning in law. It’s defined in terms of pupils requiring educational provision additional to or different from “educational facilities of a kind generally provided for children of the same age in schools within the area of the local education authority”.

Statements of SEN

The proportion of pupils with statements of SEN remained consistently at around 2.8% between 2005 and 2013 (after which the SEN system changed). http://www.publications.parliament.uk/pa/cm200506/cmselect/cmeduski/478/478i.pdf https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/225699/SFR30-2013_Text.pdf

It could be, of course, that these figures are an artifact of the system; anecdotal evidence suggests that some local authorities considered statutory assessments only for children who scored below the 2nd percentile on the WISC scale. Or it could be that measures of educational attainment do reflect the effectively random nature of the causes of educational attainment. In other words, a single measure of educational attainment can tell us whether a child’s attainment is unusually high or low; it can’t tell us why it’s unusually high or low. That often requires a bit of detective work.

If they can do it, anyone can

Some people feel uncomfortable with the idea of human populations having inherent variation; it smacks of determinism, excuses and complacency. So from time to time we read inspiring accounts of children in a school in a deprived inner city borough all reading fluently by the age of 6, or of the GCSE A*-C grades in a once failing school leaping from 30-60% in a year.  The implication is that if they can do it, anyone can. That’s a false assumption. Those things can happen in some schools. But they can’t happen in all schools simultaneously because of the variation inherent in human populations and because of the nature of life events (see previous post).

Children of differing abilities don’t distribute themselves neatly across schools. Some schools might have no children with statements and others might have many. Even if all circumstances were equal (which they’re not) clustering occurs within random distributions. This is a well-know phenomenon in epidemiology; towns with high numbers of cancer patients or hospitals with high numbers of unexpected deaths where no causal factors are identified tend to attract the attention of conspiracy theorists. This clustering illusion isn’t so well known in educational circles. It’s all too easy to assume that a school has few children with special educational needs because of the high quality of teaching, or that a school has many children with SEN because teaching is poor. Obviously, it’s more complicated than that.

What helps the weakest pupils?

According to Katie what ‘the weakest pupils’ need is “more focus, more rigour and more practice if they are to stand any chance of catching up with their peers”.   Katie goes on to unpack what she means. More focus means classrooms that aren’t chaotic. More rigour means expecting children to read challenging texts. More practice means practicing the things they can’t do, not the things they can.

Katie’s post is based on the assumption that the weakest pupils can and should ‘catch up with their peers.’ But it’s not clear what she means by that. Does she mean the school not needing a bottom set? All pupils attaining at least the national average for their age group? All pupils clustered at the high end of the attainment range?  She doesn’t say.

In a twitter discussion, Katie agreed that there is variation inherent in a population, but

katie ashford bell curve

I agree with Katie that there is often room for improvement, and that her focus, getting all children reading, can make a big difference, but improvement is likely to entail more than more focus, more rigour and more practice. In an earlier post Katie complains that “Too many people overcomplicate the role of SENCO”.   She sees her role as very simple: “I avoid pointless meetings, unnecessary paperwork and attending timewasting conferences as much as possible. Instead, I teach, organise interventions, spend lots of time with the pupils, and make sure teachers and support staff have everything they need to teach their kids really, really well.

Her approach sounds very sensible.  But she doesn’t say what the interventions are. Or what the teachers and support staff need to teach their kids really, really well. Or what meetings, paperwork and conferences she thinks are pointless, unnecessary and timewasting. Katie doesn’t say how many children at Michaela have statements of special needs or EHCPs – presumably some children have arrived there with these in place. Or what she does about the meetings and paperwork involved. Or how she tracks individual children’s progress. (I’m not suggesting that statements and EHCPs are the way to go – just that currently they’re part of the system and SENCOs have to deal with them).

What puzzled me most about Katie’s interventions was that they bore little resemblance to those I’ve seen other SENCOs implement in mainstream schools. It’s possible that they’ve overcomplicated their role.   It could be that the SENCOs I’ve watched at work are in primary schools and that at secondary level it’s different. Another explanation is that they’ve identified the root causes of children’s learning difficulties and have addressed them.

They’ve introduced visual timetables, taught all pupils Makaton, brought in speech and language therapists to train staff, installed the same flooring throughout the building to improve the mobility of children with cerebral palsy or epilepsy and integrated music, movement and drama into the curriculum. They’ve developed assistive technology for children with sensory impairments and built up an extensive, accessible school library that includes easy-to-read books with content suitable for older kids for poor readers and more challenging texts with content suitable for younger kids for good readers. They’ve planted gardens and attended forest schools regularly to develop motor and sensory skills.

As a child who read avidly  –  including Dickens – I can see how many hours of reading and working through chapters of Dickens could improve the reading ability of many children. But I’m still struggling to see how that would work for a kid whose epilepsy results in frequent ‘absences’ of attention or who has weak control of eye movements, an auditory processing impairment or very limited working memory capacity.

I’m aware that ‘special educational needs’ is a contentious label and that it’s often only applied because children aren’t being taught well, or taught appropriately. I’m utterly committed to the idea of every child being given the best possible education. I just don’t see any evidence to support the idea that catching up with one’s peers is a measure of educational excellence, or that practicing what you’re not good at is a better use of time than doing what you are good at.

Section 7 of the Education Act 1996 (based on the 1944 Education Act) frames a suitable education in terms of an individual child’s age, ability and aptitude, and any special educational needs they may have. The education system appears to have recently lost sight of the aptitude element. I fear that an emphasis on ‘catching up’ with one’s peers and on addressing weaknesses rather than developing strengths will inevitably result in many children seeing themselves as failing to jump an arbitrary hurdle, rather than as individuals with unique sets of talents and aptitudes who can play a useful and fulfilling role in society.

I’d be interested in Katie’s comments.

standardised testing: what’s it good for?

A campaign by parents to keep their children off school on Tuesday 3rd May as a protest against SATs prompted a Twitter discussion about the pros and cons of standardised tests. One teacher claimed that they’re important because they hold schools to account. I think that’s a misuse of standardised tests. First, because test results are a poor proxy measure of teaching quality. Second, good teaching (and hard work on the part of the student) are necessary but not sufficient conditions for good test performance. Third, using test results to hold schools to account overlooks the natural variation inherent in large populations.

test results as a measure of teaching quality

Tests such as the National Curriculum Tests (commonly known as SATs) GCSEs and A levels sample students’ recall and understanding of a particular body of knowledge – the KS2 curriculum, GCSE/A level course. The knowledge is sampled because testing the student’s knowledge of all the material in the course would be very time consuming and unwieldy. In other words, test results are a proxy for the student’s knowledge of the course material.

But the course material itself is a proxy for all that’s known about a particular topic. KS2 students learn basic principles about how atoms and molecules behave, GCSE and A level students learn about atomic theory in more detail, but Chemistry undergraduates complain that they have to then unlearn much of what they were taught earlier because it was the simplified version.  So test results are actually a second order proxy for the student’s knowledge of a particular topic.

Then factors other than the student’s knowledge impact on test results. The student might be unwell on the day of the test, or might have slept badly the night before. In the months before the test they might have been absent from school for weeks with glandular fever or their parents might have split up. In other words, test results are affected by factors other than teaching and learning; factors beyond the control of either the school or the student.  In other words, test results are a weak proxy for both the quality of teaching and the student’s knowledge.

good teaching and hard work are necessary but not sufficient for good test performance

There’s an asymmetry between the causes of high and low test results. It’s difficult to get a high test score without hard work on the part of the student and good teaching on the part of the school.   But there are many reasons why a student might get a low score despite hard work and good teaching.

That’s at the individual level. Similarly at the school level it’s safe to conclude that a school with consistently good results in national tests is doing its job properly, but it’s not safe to conclude that a school that doesn’t get consistently good results isn’t.

The education system has been plagued over the years by two false assumptions about student potential. Either that all students have the potential to get good test scores and that good teaching is the key determining factor, or that students from certain demographic groups won’t get good test scores however well they’re taught. In reality it’s more complicated than that, of course. Students from leafy suburbs are more likely to do well in tests for many reasons; even if they are taught badly, they have access to resources that can sometimes compensate for that. Students from the kind of housing estate that motivates Iain Duncan Smith are at a higher risk of adverse life events scuppering their chances of getting good test results no matter how good the teaching at their school. And the older they get, the more adverse life events they are likely to encounter.

So, test results are a pretty good first order proxy for a student’s knowledge of course material. They are a not-so-good second order proxy for a student’s knowledge of the topic the course material represents. And only a weak proxy for quality of teaching.

life is just one damn thing after another*

Those in favour of standardised testing often cite cases of particular schools in deprived areas that have achieved amazing outcomes against the odds. Every child can read by the age of six, or is fluent in French, or whatever.   The implication is that if one school can do it, all schools can. In principle, that’s true. In principle, all head teachers can be visionaries, all teachers can be excellent and all families can buy in to what the school wants to achieve.

But in practice life doesn’t work like that. Head teachers get sick, senior staff have to work part-time because of family commitments, local housing is unaffordable making recruitment a nightmare, or for many families school is just one more thing they can’t quite keep up with.

On top of that, human beings are biological organisms. Like all populations of biological organisms we show considerable variation due to our genes, our environment and interactions between the two. It might be possible to improve test performance across the education system, but there are limits to the improvement that’s possible. Clean water and good sanitation increase life expectancy, but life expectancy doesn’t go on increasing indefinitely once communities have access to clean water and sanitation. Expecting more than 50% of children in primary schools to perform above average simply shows a poor grasp of natural variation – and statistics.

standardised testing: what is it good for?

Standardised testing in primary schools makes sense. It samples children’s knowledge of key material. It allows schools to benchmark attainment. Standardised testing as a performance measure can alert schools to problems that are impacting on children’s learning.

However, the reasons for differences in students’ performance in standardised tests are many and varied. Performance will not improve unless the reasons for poor performance are addressed. Sometimes those reasons are complex and not within the schools’ remit. To address them local families might need better public services, better jobs or better housing – arguably not the core responsibility of a school. Poor teaching might not be involved at all.

However, successive governments haven’t used test results simply as broad indicators of whether a school is on track or whether there are problems that need to be addressed (not necessarily by the school), but as a proxy for teaching quality.  Test results have been used to set performance targets and determine funding, regardless of whether schools can control the factors involved.

This shows a poor understanding of performance management§, and it’s hardly surprising that the huge amounts of money and incessant policy changes thrown at the education system over recent decades have had little impact on the quality of education of the population as a whole.

Notes

*A quotation attributed to Elbert Hubbard, an American  writer who died when the Lusitania was sunk in 1915.

§ The best book I’ve read on performance management is a slim volume by Donald Wheeler called Understanding variation: The key to managing chaos.  A clearly written, step-by-step guide to figuring out if the variation you’ve spotted is within natural limits or not.  Lots of references to things like iron smelting and lumber yards, but still very relevant to schools.