We’re all different

We’re all different. Tiny variations in our DNA before and at conception. Our parents smoked/didn’t smoke, drank/didn’t drink, followed ideal/inadequate diets, enjoyed robust health or had food intolerances, allergies, viral infections. We were brought up in middle class suburbia/a tower block and attended inadequate/outstanding schools. All those factors contribute to who we are and what we are capable of achieving.

That variation, inherent in all biological organisms, is vital to our survival as a species. Without it, we couldn’t adapt to a changing environment or form communities that successfully protect us. The downside of that inherent variation is that some of us draw a short straw. Some variations mean we don’t make it through childhood, have lifelong health problems or die young. Or that we become what Katie Ashford, SENCO at Michaela Community School, Wembley calls the ‘weakest pupils‘.

Although the factors that contribute to our development aren’t, strictly speaking, random, they are so many and so varied, they might as well be random. That means that in a large population, the measurement of any characteristic affected by many factors – height, blood pressure, intelligence, reading ability – will form what’s known as a normal distribution; the familiar bell curve.

 The bell curve

If a particular characteristic forms a bell-shaped distribution, that allows us to make certain predictions about a large population. For that characteristic, 50% of the population will score above average and 50% below average; there will be relatively few people who are actually average. We’ll know that around 70% of the population will score fairly close to average, around 25% noticeably above or below it, and around 5% considerably higher or lower. That’s why medical reference ranges for various characteristics are based on the upper and lower measurements for 95% of the population; if your blood glucose levels or thyroid function is in the lowest or highest 2.5%, you’re likely to have a real problem, rather than a normal variation.

So in terms of general ability that means around 2.5% of the population will be in a position to decide whether they’d rather be an Olympic athlete, a brain surgeon or Prime Minister (or all three), whereas another 2.5% will find everyday life challenging.

What does a normal distribution mean for education? Educational attainment is affected by many causal factors, so by bizarre coincidence the attainment of 50% of school pupils is above average, and 50% below it. Around 20% of pupils have ‘special educational needs’ and around 2.5% will have educational needs that are significant enough to warrant a Statement of Special Educational Needs (recently replaced by Education Health and Care Plans).

Special educational needs

In 1978, the Warnock report pointed out that based on historical data, up to 20% of school pupils would probably have special educational needs at some point in their school career. ‘Special educational needs’ has a precise but relative meaning in law. It’s defined in terms of pupils requiring educational provision additional to or different from “educational facilities of a kind generally provided for children of the same age in schools within the area of the local education authority”.

Statements of SEN

The proportion of pupils with statements of SEN remained consistently at around 2.8% between 2005 and 2013 (after which the SEN system changed). http://www.publications.parliament.uk/pa/cm200506/cmselect/cmeduski/478/478i.pdf https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/225699/SFR30-2013_Text.pdf

It could be, of course, that these figures are an artifact of the system; anecdotal evidence suggests that some local authorities considered statutory assessments only for children who scored below the 2nd percentile on the WISC scale. Or it could be that measures of educational attainment do reflect the effectively random nature of the causes of educational attainment. In other words, a single measure of educational attainment can tell us whether a child’s attainment is unusually high or low; it can’t tell us why it’s unusually high or low. That often requires a bit of detective work.

If they can do it, anyone can

Some people feel uncomfortable with the idea of human populations having inherent variation; it smacks of determinism, excuses and complacency. So from time to time we read inspiring accounts of children in a school in a deprived inner city borough all reading fluently by the age of 6, or of the GCSE A*-C grades in a once failing school leaping from 30-60% in a year.  The implication is that if they can do it, anyone can. That’s a false assumption. Those things can happen in some schools. But they can’t happen in all schools simultaneously because of the variation inherent in human populations and because of the nature of life events (see previous post).

Children of differing abilities don’t distribute themselves neatly across schools. Some schools might have no children with statements and others might have many. Even if all circumstances were equal (which they’re not) clustering occurs within random distributions. This is a well-know phenomenon in epidemiology; towns with high numbers of cancer patients or hospitals with high numbers of unexpected deaths where no causal factors are identified tend to attract the attention of conspiracy theorists. This clustering illusion isn’t so well known in educational circles. It’s all too easy to assume that a school has few children with special educational needs because of the high quality of teaching, or that a school has many children with SEN because teaching is poor. Obviously, it’s more complicated than that.

What helps the weakest pupils?

According to Katie what ‘the weakest pupils’ need is “more focus, more rigour and more practice if they are to stand any chance of catching up with their peers”.   Katie goes on to unpack what she means. More focus means classrooms that aren’t chaotic. More rigour means expecting children to read challenging texts. More practice means practicing the things they can’t do, not the things they can.

Katie’s post is based on the assumption that the weakest pupils can and should ‘catch up with their peers.’ But it’s not clear what she means by that. Does she mean the school not needing a bottom set? All pupils attaining at least the national average for their age group? All pupils clustered at the high end of the attainment range?  She doesn’t say.

In a twitter discussion, Katie agreed that there is variation inherent in a population, but

katie ashford bell curve

I agree with Katie that there is often room for improvement, and that her focus, getting all children reading, can make a big difference, but improvement is likely to entail more than more focus, more rigour and more practice. In an earlier post Katie complains that “Too many people overcomplicate the role of SENCO”.   She sees her role as very simple: “I avoid pointless meetings, unnecessary paperwork and attending timewasting conferences as much as possible. Instead, I teach, organise interventions, spend lots of time with the pupils, and make sure teachers and support staff have everything they need to teach their kids really, really well.

Her approach sounds very sensible.  But she doesn’t say what the interventions are. Or what the teachers and support staff need to teach their kids really, really well. Or what meetings, paperwork and conferences she thinks are pointless, unnecessary and timewasting. Katie doesn’t say how many children at Michaela have statements of special needs or EHCPs – presumably some children have arrived there with these in place. Or what she does about the meetings and paperwork involved. Or how she tracks individual children’s progress. (I’m not suggesting that statements and EHCPs are the way to go – just that currently they’re part of the system and SENCOs have to deal with them).

What puzzled me most about Katie’s interventions was that they bore little resemblance to those I’ve seen other SENCOs implement in mainstream schools. It’s possible that they’ve overcomplicated their role.   It could be that the SENCOs I’ve watched at work are in primary schools and that at secondary level it’s different. Another explanation is that they’ve identified the root causes of children’s learning difficulties and have addressed them.

They’ve introduced visual timetables, taught all pupils Makaton, brought in speech and language therapists to train staff, installed the same flooring throughout the building to improve the mobility of children with cerebral palsy or epilepsy and integrated music, movement and drama into the curriculum. They’ve developed assistive technology for children with sensory impairments and built up an extensive, accessible school library that includes easy-to-read books with content suitable for older kids for poor readers and more challenging texts with content suitable for younger kids for good readers. They’ve planted gardens and attended forest schools regularly to develop motor and sensory skills.

As a child who read avidly  –  including Dickens – I can see how many hours of reading and working through chapters of Dickens could improve the reading ability of many children. But I’m still struggling to see how that would work for a kid whose epilepsy results in frequent ‘absences’ of attention or who has weak control of eye movements, an auditory processing impairment or very limited working memory capacity.

I’m aware that ‘special educational needs’ is a contentious label and that it’s often only applied because children aren’t being taught well, or taught appropriately. I’m utterly committed to the idea of every child being given the best possible education. I just don’t see any evidence to support the idea that catching up with one’s peers is a measure of educational excellence, or that practicing what you’re not good at is a better use of time than doing what you are good at.

Section 7 of the Education Act 1996 (based on the 1944 Education Act) frames a suitable education in terms of an individual child’s age, ability and aptitude, and any special educational needs they may have. The education system appears to have recently lost sight of the aptitude element. I fear that an emphasis on ‘catching up’ with one’s peers and on addressing weaknesses rather than developing strengths will inevitably result in many children seeing themselves as failing to jump an arbitrary hurdle, rather than as individuals with unique sets of talents and aptitudes who can play a useful and fulfilling role in society.

I’d be interested in Katie’s comments.

standardised testing: what’s it good for?

A campaign by parents to keep their children off school on Tuesday 3rd May as a protest against SATs prompted a Twitter discussion about the pros and cons of standardised tests. One teacher claimed that they’re important because they hold schools to account. I think that’s a misuse of standardised tests. First, because test results are a poor proxy measure of teaching quality. Second, good teaching (and hard work on the part of the student) are necessary but not sufficient conditions for good test performance. Third, using test results to hold schools to account overlooks the natural variation inherent in large populations.

test results as a measure of teaching quality

Tests such as the National Curriculum Tests (commonly known as SATs) GCSEs and A levels sample students’ recall and understanding of a particular body of knowledge – the KS2 curriculum, GCSE/A level course. The knowledge is sampled because testing the student’s knowledge of all the material in the course would be very time consuming and unwieldy. In other words, test results are a proxy for the student’s knowledge of the course material.

But the course material itself is a proxy for all that’s known about a particular topic. KS2 students learn basic principles about how atoms and molecules behave, GCSE and A level students learn about atomic theory in more detail, but Chemistry undergraduates complain that they have to then unlearn much of what they were taught earlier because it was the simplified version.  So test results are actually a second order proxy for the student’s knowledge of a particular topic.

Then factors other than the student’s knowledge impact on test results. The student might be unwell on the day of the test, or might have slept badly the night before. In the months before the test they might have been absent from school for weeks with glandular fever or their parents might have split up. In other words, test results are affected by factors other than teaching and learning; factors beyond the control of either the school or the student.  In other words, test results are a weak proxy for both the quality of teaching and the student’s knowledge.

good teaching and hard work are necessary but not sufficient for good test performance

There’s an asymmetry between the causes of high and low test results. It’s difficult to get a high test score without hard work on the part of the student and good teaching on the part of the school.   But there are many reasons why a student might get a low score despite hard work and good teaching.

That’s at the individual level. Similarly at the school level it’s safe to conclude that a school with consistently good results in national tests is doing its job properly, but it’s not safe to conclude that a school that doesn’t get consistently good results isn’t.

The education system has been plagued over the years by two false assumptions about student potential. Either that all students have the potential to get good test scores and that good teaching is the key determining factor, or that students from certain demographic groups won’t get good test scores however well they’re taught. In reality it’s more complicated than that, of course. Students from leafy suburbs are more likely to do well in tests for many reasons; even if they are taught badly, they have access to resources that can sometimes compensate for that. Students from the kind of housing estate that motivates Iain Duncan Smith are at a higher risk of adverse life events scuppering their chances of getting good test results no matter how good the teaching at their school. And the older they get, the more adverse life events they are likely to encounter.

So, test results are a pretty good first order proxy for a student’s knowledge of course material. They are a not-so-good second order proxy for a student’s knowledge of the topic the course material represents. And only a weak proxy for quality of teaching.

life is just one damn thing after another*

Those in favour of standardised testing often cite cases of particular schools in deprived areas that have achieved amazing outcomes against the odds. Every child can read by the age of six, or is fluent in French, or whatever.   The implication is that if one school can do it, all schools can. In principle, that’s true. In principle, all head teachers can be visionaries, all teachers can be excellent and all families can buy in to what the school wants to achieve.

But in practice life doesn’t work like that. Head teachers get sick, senior staff have to work part-time because of family commitments, local housing is unaffordable making recruitment a nightmare, or for many families school is just one more thing they can’t quite keep up with.

On top of that, human beings are biological organisms. Like all populations of biological organisms we show considerable variation due to our genes, our environment and interactions between the two. It might be possible to improve test performance across the education system, but there are limits to the improvement that’s possible. Clean water and good sanitation increase life expectancy, but life expectancy doesn’t go on increasing indefinitely once communities have access to clean water and sanitation. Expecting more than 50% of children in primary schools to perform above average simply shows a poor grasp of natural variation – and statistics.

standardised testing: what is it good for?

Standardised testing in primary schools makes sense. It samples children’s knowledge of key material. It allows schools to benchmark attainment. Standardised testing as a performance measure can alert schools to problems that are impacting on children’s learning.

However, the reasons for differences in students’ performance in standardised tests are many and varied. Performance will not improve unless the reasons for poor performance are addressed. Sometimes those reasons are complex and not within the schools’ remit. To address them local families might need better public services, better jobs or better housing – arguably not the core responsibility of a school. Poor teaching might not be involved at all.

However, successive governments haven’t used test results simply as broad indicators of whether a school is on track or whether there are problems that need to be addressed (not necessarily by the school), but as a proxy for teaching quality.  Test results have been used to set performance targets and determine funding, regardless of whether schools can control the factors involved.

This shows a poor understanding of performance management§, and it’s hardly surprising that the huge amounts of money and incessant policy changes thrown at the education system over recent decades have had little impact on the quality of education of the population as a whole.

Notes

*A quotation attributed to Elbert Hubbard, an American  writer who died when the Lusitania was sunk in 1915.

§ The best book I’ve read on performance management is a slim volume by Donald Wheeler called Understanding variation: The key to managing chaos.  A clearly written, step-by-step guide to figuring out if the variation you’ve spotted is within natural limits or not.  Lots of references to things like iron smelting and lumber yards, but still very relevant to schools.