magic beans, magic bullets and crypto-pathologies

In the previous post, I took issue with a TES article that opened with fidget-spinners and closed with describing dyslexia and ADHD as ‘crypto-pathologies’. Presumably as an analogy with cryptozoology – the study of animals that exist only in folklore. But dyslexia and ADHD are not the equivalent of bigfoot and unicorns.

To understand why, you have to unpack what’s involved in diagnosis.

diagnosis, diagnosis, diagnosis

Accurate diagnosis of health problems has always been a challenge because:

  • Some disorders* are difficult to diagnose. A broken femur, Bell’s palsy or measles are easier to figure out than hypothyroidism, inflammatory bowel disease or Alzheimer’s.
  • It’s often not clear what’s causing the disorder. Fortunately, you don’t have to know the immediate or root causes for successful treatment to be possible. Doctors have made the reasonable assumption that patients presenting with the same signs and symptoms§ are likely to have the same disorder.

Unfortunately, listing the signs and symptoms isn’t foolproof because;

  • some disorders produce different signs and symptoms in different patients
  • different disorders can have very similar signs and symptoms.

some of these disorders are not like the others…

To complicate the picture even further, some signs and symptoms are qualitatively different from the aches, pains, rashes or lumps that indicate disorders obviously located in the body;  they involve thoughts, feelings and behaviours instead. Traditionally, human beings have been assumed to consist of a physical body and non-physical parts such as mind and spirit, which is why disorders of thoughts, feelings and behaviours were originally – and still are – described as mental disorders.

Doctors have always been aware that mind can affect body and vice versa. They’ve also long known that brain damage and disease can affect thoughts, feelings, behaviours and physical health. In the early 19th century, mental disorders were usually identified by key symptoms. The problem was that the symptoms of different disorders often overlapped. A German psychiatrist, Emil Kraepelin, proposed instead classifying mental disorders according to syndromes, or patterns of co-occurring signs and symptoms. Kraepelin hoped this approach would pave the way for finding the biological causes of disorders. (In 1906, Alois Alzheimer found the plaques that caused the dementia named after him, while he was working in Kraepelin’s lab.)

Kraepelin’s approach laid the foundations for two widely used modern classification systems for mental disorders; the Diagnostic and Statistical Manual of Mental Disorders, published by the American Psychiatric Association, currently in its 5th edition (DSM V), and the International Classification of Diseases Classification of Mental and Behavioural Disorders published by the World Health Organisation, currently in its 10th edition (ICD-10).

Kraepelin’s hopes for his classification system have yet to be realised. That’s mainly because the brain is a difficult organ to study. You can’t poke around in it without putting your patient at risk. It’s only in the last few decades that scanning techniques have enabled researchers to look more closely at the structure and function of the brain, and the scans require interpretation –  brain imaging is still in its infancy.

you say medical, I say experiential

Kraepelin’s assumptions about distinctive patterns of signs and symptoms, and about their biological origins, were reasonable ones. His ideas, however, were almost the polar opposite to those of his famous contemporary, Sigmund Freud, who located the root causes of mental disorders in childhood experience. The debate has raged ever since. The dispute is due to the plasticity of the brain.  Brains change in structure and function over time and several factors contribute to the changes;

  • genes – determine underlying structure and function
  • physical environment e.g. biochemistry, nutrients, toxins – affects structure and function
  • experience – the brain processes information, and information changes the brain’s physical structure and biochemical function.

On one side of the debate is the medical model; in essence, it assumes that the causes of mental disorders are primarily biological, often due to a ‘chemical imbalance’. There’s evidence to support this view; medication can improve a patient’s symptoms. The problem with the medical model is that it tends to assume;

  • a ‘norm’ for human thought, feelings and behaviours – disorders are seen as departures from that norm
  • the cause of mental disorders is biochemical and the chemical ‘imbalance’ is identified (or not) through trial-and-error – errors can be catastrophic for the patient.
  • the cause is located in the individual.

On the other side of the debate is what I’ll call the experiential model (often referred to as anti-psychiatry or critical psychiatry). In essence it assumes the causes of unwanted thoughts, feelings or behaviours are primarily experiential, often due to adverse experiences in childhood. The problem with that model is that it tends to assume;

  • the root causes are experiential and not biochemical
  • the causes are due to the individual’s response to adverse experiences
  • first-hand reports of early adverse experiences are always reliable, which they’re not.

labels

Kraepelin’s classification system wasn’t definitive – it couldn’t be, because no one knew what was causing the disorders. But it offered the best chance of identifying distinct mental health problems – and thence their causes and treatments. The disorders identified in Kraepelin’s system, the DSM and ICD, were – and most still are – merely labels given to clusters of co-occurring signs and symptoms.  People showing a particular cluster are likely to share the same underlying biological causes, but that doesn’t mean they do share the same underlying causes or that the origin of the disorder is biological.

This is especially true for signs and symptoms that could have many causes. There could be any number of reasons for someone hallucinating, withdrawing, feeling depressed or anxious – or having difficulty learning to read or maintain attention.  They might not have a medical ‘disorder’ as such. But you wouldn’t know that to read through the disorders listed in the DSM or ICD. They all look like bona fide, well-established medical conditions, not like labels for bunches of symptoms that sometimes co-occur and sometimes don’t, and that have a tendency to appear or disappear with each new edition of the classification system.  That brings us to the so-called ‘crypto-pathologies’ referred to in the TES article.

Originally, terms like dyslexia were convenient and legitimate shorthand labels for specific clusters of signs or symptoms. Dyslexia means difficulty with reading, as distinct from alexia which means not being able to read at all; both problems can result from stroke or brain damage. Similarly, autism was originally a shorthand term for the withdrawn state that was one of the signs of schizophrenia – itself a label.  Delusional parasitosis is also a descriptive label (the parasites being what’s delusional, not the itching).

reification

What’s happened over time is that many of these labels have become reified – they’ve transformed from mere labels into disorders widely perceived as having an existence independent of the label. Note that I’m not saying the signs and symptoms don’t exist. There are definitely children who struggle with reading regardless of how they’ve been taught; with social interaction regardless of how they’ve been brought up; and with maintaining focus regardless of their environment. What I am saying is that there might be different causes, or multiple causes, for clusters of very similar signs and symptoms.  Similar signs and symptoms don’t mean that everybody manifesting those signs and symptoms has the same underlying medical disorder –  or even that they have a medical disorder at all.

The reification of labels has caused havoc for decades with research. If you’ve got a bunch of children with different causes for their problems with reading, but you don’t know what the different causes are so you lump all the children together according to their DSM label; or another bunch with different causes for their problems with social interaction but lump them all together; or a third bunch with different causes for their problems maintaining focus, but you lump them all together; you are not likely to find common causes in each group for the signs and symptoms.  It’s this failure to find distinctive features at the group level that has been largely responsible for claims that dyslexia, autism or ADHD ‘don’t exist’, or that treatments that have evidently worked for some individuals must be spurious because they don’t work for other individuals or for the heterogeneous group as a whole.

crypto-pathologies

Oddly, in his TES article, Tom refers to autism as an ‘identifiable condition’ but to dyslexia and ADHD as ‘crypto-pathologies’ even though the diagnostic status of autism in the DSM and ICD is on a par with that of ADHD, and with ‘specific learning disorder with impairment in reading‘ with dyslexia recognised as an alternative term (DSM), or ‘dyslexia and alexia‘ (ICD).  Delusional parasitosis, despite having the same diagnostic status and a plausible biological mechanism for its existence, is dismissed as ‘a condition that never was’.

Tom is entitled to take a view on diagnosis, obviously. He’s right to point out that reading difficulties can be due to lack of robust instruction, and inattention can be due to the absence of clear routines. He’s right to dismiss faddish simplistic (but often costly) remedies. But the research is clear that children can have difficulties with reading due to auditory and/or visual processing impairments (search Google scholar for ‘dyslexia visual auditory’), that they can have difficulties maintaining attention due to low dopamine levels – exactly what Ritalin addresses (Iversen, 2006), or that they can experience intolerable itching that feels as if it’s caused by parasites.

But Tom doesn’t refer to the research, and despite provisos such as acknowledging that some children suffer from ‘real and grave difficulties’ he effectively dismisses some of those difficulties as crypto-pathologies and implies they can be fixed by robust teaching and clear routines  –  or that they are just imaginary.  There’s a real risk, if the research is by-passed, of ‘robust teaching’ and ‘clear routines’ becoming the magic bullets and magic beans he rightly despises.

Notes

*Disorder implies a departure from the norm.  At one time, it was assumed the norm for each species was an optimal set of characteristics.  Now, the norm is statistically derived, based on 95% of the population.

§ Technically, symptoms are indicators of a disorder experienced only by the patient and signs are detectable by others.  ‘Symptoms’ is often used to include both.

Reference

Iversen, L (2006).  Speed, Ecstasy, Ritalin: The science of amphetamines.  Oxford University Press.

white knights and imaginary dragons: Tom Bennett on fidget-spinners

I’ve crossed swords – or more accurately, keyboards – with Tom Bennett, the government’s behaviour guru tsar adviser, a few times, mainly about learning styles. And about Ken Robinson. Ironic really, because broadly speaking we’re in agreement. Ken Robinson’s ideas about education are woolly and often appear to be based on opinion rather than evidence, and there’s clear evidence that teachers who use learning styles, thinking hats and brain gym probably are wasting their time. Synthetic phonics helps children read and whole school behaviour policies are essential for an effective school and so on…

My beef with Tom has been his tendency to push his conclusions further than the evidence warrants. Ken Robinson is ‘the butcher given a ticker tape parade by the National Union of Pigs‘.  Learning Styles are ‘the ouija board of serious educational research‘.  What raised red flags for me this time is a recent TES article by Tom prompted by the latest school-toy fad ‘fidget-spinners’.

fidget-spinners

Tom begins with claims that fidget-spinners can help children concentrate. He says “I await the peer-reviewed papers from the University of Mickey Mouse confirming these claims“, assuming that he knows what the evidence will be before he’s even seen it.  He then introduces the idea that ‘such things’ as fidget-spinners might help children with an ‘identifiable condition such as autism or sensory difficulties’, and goes on to cite comments from several experts about fidget-spinners in particular and sensory toys in general. We’re told “…if children habitually fidget, the correct path is for the teacher to help the child to learn better behaviour habits, unless you’ve worked with the SENCO and the family to agree on their use. The alternative is to enable and deepen the unhelpful behaviour. Our job is to support children in becoming independent, not cripple them with their own ticks [sic]”.

If a child’s fidgeting is problematic, I completely agree that a teacher’s first course of action should be to help them stop fidgeting, although Tom offers no advice about how to do this. I’d also agree that the first course of action in helping a fidgety child shouldn’t be to give them a fidget-toy.

There’s no question that children who just can’t seem to sit still, keep their hands still, or who incessantly chew their sleeves, are seeking sensory stimulation, because that’s what those activities are – by definition. It doesn’t follow that allowing children to walk about, or use fidget or chew toys will ‘cripple them with their own ticks’. These behaviours are not tics, and usually extinguish spontaneously over time. If they’re causing disruption in the classroom, questions need to be asked about school expectations and the suitability of the school provision for the child, not about learning unspecified ‘better behaviour habits’.

mouthwash

Tom then devotes an entire paragraph to, bizarrely, Listerine. His thesis is that sales of antiseptic mouthwash soared due to an advertising campaign persuading Americans that halitosis was a serious social problem. His evidence is a blogpost by Sarah Zhang, a science journalist.  Sarah’s focus is advertising that essentially invented problems to be cured by mouthwash or soap. Neither she nor Tom mention the pre-existing obsession with cleanliness that arose from the discovery – prior to the discovery of antibiotics – that a primary cause of death and debility was bacterial infections that could be significantly reduced by the use of alcohol rubs, boiling and soap.

itchy and scratchy

The Listerine advertising campaign leads Tom to consider ‘fake or misunderstood illnesses’ that he describes as ‘charlatan’. His examples are delusional parasitosis (people believe their skin is itching because it’s infested with parasites) and Morgellon’s (belief that the itching is caused by fibres). Tom says “But there are no fibres or parasites. It’s an entirely psycho-somatic condition. Pseudo sufferers turn up at their doctors scratching like mad, some even cutting themselves to dig out the imaginary threads and crypto-bugs. Some doctors even wearily prescribe placebos and creams that will relieve the “symptoms”. A condition that never was, dealt with by a cure that won’t work. Spread as much by belief as anything else, like fairies.”

Here, Tom is pushing the evidence way beyond its limits. The fact that the bugs or fibres are imaginary doesn’t mean the itching is imaginary. The skin contains several different types of tactile receptor that send information to various parts of the brain. The tactile sensory system is complex so there are several points at which a ‘malfunction’ could occur.  The fact that busy GPs – who for obvious reasons don’t have the time or resources to examine the functioning of a patient’s neural pathways at molecular level – wearily prescribe a placebo, says as much about the transmission of medical knowledge in the healthcare system as it does about patients’ beliefs.

crypto-pathologies

Tom refers to delusional parasitosis and Morgellon’s as ‘crypto-pathologies’ – whatever that means – and then introduces us to some crypto-pathologies he claims are encountered in school; dyslexia and ADHD. As he points out dyslexia and ADHD are indeed labels for ‘a collection of observed symptoms’. He’s right that some children with difficulty reading might simply need good reading tuition, and those with attention problems might simply need a good relationship with their teacher and clear routines. As he points out “…our diagnostic protocol is often blunt. Because we’re unsure what it is we’re diagnosing, and it becomes an ontological problem“.  He then says “This matters when we pump children with drugs like Ritalin to stun them still.

Again, some of Tom’s claims are correct but others are not warranted by the evidence. In the UK, Ritalin is usually prescribed by a paediatrician or psychiatrist after an extensive assessment of the child, and its effects should be carefully monitored. It’s a stimulant that increases available levels of dopamine and norepinephrine and it often enhances the ability to concentrate. It isn’t ‘pumped into’ children and it doesn’t ‘stun them still’, In the UK at least, NICE guidelines indicate it should be used as a last resort. The fact that its use has doubled in the last decade is a worrying trend. This is more likely to be due to the crisis in child and adolescent mental health services, than to an assumption that all attention problems in children are caused by a supposed medical condition we call ADHD.

Tom, rightly, targets bullshit. He says it matters because “many children suffer from very real and very grave difficulties, and it behoves us as their academic and social guardians to offer support and remedy when we can”. Understandably he wants to drive his point home. But superficial analysis and use of hyperbole risk real and grave difficulties being marginalised at best and ridiculed at worst by teachers who don’t have the time/energy/inclination to check out the detail of what he claims.

Specialist education, health and care services for children have been in dire straits for many years and the situation isn’t getting any better. This means teachers are likely to have little information about the underlying causes of children’s difficulties in school. If teachers take what Tom says at face value, there’s a real risk that children with real difficulties, whether they need to move their fingers or chew in order to concentrate, experience unbearable itching, struggle to read because of auditory, visual or working memory impairments, or have levels of dopamine that prevent them from concentrating, will be seen by some as having ‘crypto-conditions’ that can be resolved by good teaching and clear routines. If they’re not resolved, then the condition must be ‘psycho-somatic’.  Using evidence to make some points, but ignoring it to make others means the slings and arrows Tom hurls at the snake-oil salesmen and white knights galloping to save us from imaginary dragons are quite likely to be used as ammunition against the very children he seeks to help.

a modern day trivium

In the two previous posts, I’ve criticised Martin Robinson’s argument that traditional and progressive education are mutually exclusive approaches characterised by single core values; subject centred and child centred, respectively.

Martin describes himself as an “educationalist with an interest in culture, politics, creativity, and the Liberal Arts (especially grammar, dialectic and rhetoric)”. Grammar, logic and rhetoric are the three strands of the mediaeval trivium and Martin’s educational consultancy and his blog are called Trivium 21C. In response to my comments, he suggested I produce a graphical representation of my understanding of the trivium.

liberal arts, trivium and quadrivium

In Ancient Greece and Rome, the liberal arts were the knowledge and skills it was considered essential for a free man to learn in order to participate in civic society. The liberal arts were revived during the reign of Charlemagne in the 8th century, in an effort to improve educational and cultural standards across Western Europe. Seven subjects were studied; grammar, logic and rhetoric made up the foundational trivium, and the quadrivium consisted of arithmetic, geometry, astronomy and music.

The trivium essentially trained students to think, and the quadrivium gave them the opportunity to apply their thinking to mathematical concepts (considered fundamental to all knowledge by the Greek philosophers). The seven liberal arts formed the foundation that enabled students to proceed to study theology, medicine or law.

Up until the early 19th century, the body of collective human knowledge was relatively small.   It was possible for a well-educated person to master all of it.   In order to acquire the knowledge, and to understand and apply it, you’d have to learn Latin and probably Greek, and also how scholars (who would have written in Latin) reasoned. The trivium made explicit the structure of language, how language was used to reason, and how to explain and persuade using language.

Since the early 19th century our collective knowledge has expanded enormously and much of that knowledge is recorded in English. There are good reasons why English-speaking students should learn the structure of their native language, how to reason in it, and how to use it to explain and persuade. But those skills wouldn’t be much use without the knowledge to apply them to.

I can see how those principles could be applied to our current body of knowledge, and that’s what I’ve mapped out below.Slide1

Grammar would make explicit the structure of the knowledge (including the structure of language). Logic would make reasoning explicit – and common errors and biases in thinking. (Martin replaces logic with dialectic, a process by which different parties seek to arrive at the truth through reasoned discussion with each other.) Rhetoric would make explicit the representation of knowledge, including how people conceptualise it. Incidentally, the body of knowledge has a fuzzy boundary because although much of it is reliable, some is still uncertain.

modern liberal arts and cultural literacy

Many modern colleges and universities offer liberal arts courses, although what’s entailed varies widely. Whatever the content, the focus of liberal arts is on preparing the student for participation in civic society, as distinct from professional, vocational or technical training.

So… I can see the point of the trivium in its original context. And how the principles of the trivium could be applied to the body of knowledge available in the 21st century. Those principles would provide a practical way of ensuring students had a thorough understanding of the knowledge they were applied to.

But… I do have some concerns about using the trivium to do that. The emphasis of the trivium and of liberal arts education, is on language. Language is the primary vehicle for ideas, so there are very good reasons for students mastering language and its uses. And the purpose of a liberal arts education is to prepare students for life, rather than just for work. There are good reasons for that too; human beings are obviously much more than economic units.

However, language and the ideas it conveys also appears to be the end-point of education for liberal arts advocates, rather than just a means to an end. The content of the education is frequently described as ‘the best which has been thought or said’ (Arnold, 1869), and the purpose to enable students to participate in the ‘conversation of mankind’ (Oakeshott, 1962).

The privileging of words and abstract ideas over the nitty-gritty of everyday life is a characteristic of liberal arts education that runs from Plato through the mediaeval period to the modern day. Plato was primarily concerned with the philosopher king and the philosophers who debated with him, not with people who grew vegetables, made copper pots or traded olive oil.   Charlemagne’s focus was on making sure priests could read the Vulgate and that there were enough skilled scribes to keep records, not in improving technology, or the fortunes of the wool industry.

This dualistic rift still permeates thinking about education as evidenced by the ongoing debate about academic v vocational education. Modern-day liberal arts advocates favour the academic approach because, rightly, they see education as more than preparation for work.   Their emphasis, instead, is on cultural literacy. Cultural literacy is important for everybody because it gives access to ideas. However, the flow of information needs to be in two directions, not just one.

Recent events suggest that policy-makers who attended even ‘the best’ private schools, where cultural literacy was highly valued, have struggled to generate workable solutions to the main challenges facing the human race; the four identified by Capra and Luisi (2014) are globalisation, climate change, agriculture, and sustainable design. The root causes and the main consequences of such challenges involve the lowest, very concrete levels that would be familiar to ancient Greek farmers, coppersmiths and merchants, to mediaeval carpenters and weavers, and to those who work in modern factories, but might be unfamiliar to philosophers, scholars or politicians who could rely on slaves or servants.

An education that equips people for life rather than work does not have to put language and ideas on a pedestal; we are embodied beings that live in a world that is uncompromisingly concrete and sometimes sordidly practical. An all-round education will involve practical science, technology and hands-on craft skills, not to prepare students for a job, but so they know how the world works.  It will not just prepare them for participating in conversations.

references

Arnold, M (1869).  Culture and Anarchy.  Accessed via Project Gutenberg http://www.gutenberg.org/cache/epub/4212/pg4212-images.html

Capra, F and Luisi, PL (2014).  The Systems View of Life, Cambridge University Press (p. 394)

Oakeshott, M (1962).”The Voice of Poetry in the Conversation of Mankind” in Rationalism in Politics and Other Essays. London: Methuen, 197-247. Accessed here http://english.ttu.edu/kairos/2.1/features/brent/oakeshot.htm

The Tiger Teachers and cognitive science

Cognitive science is a key plank in the Tiger Teachers’ model of knowledge. If I’ve understood it properly the model looks something like this:

Cognitive science has discovered that working memory has limited capacity and duration, so pupils can’t process large amounts of novel information. If this information is secured in long-term memory via spaced, interleaved practice, students can recall it instantly whenever they need it, freeing up working memory for thinking.

What’s wrong with that? Nothing, as it stands. It’s what’s missing that’s the problem.

Subject knowledge

One of the Tiger Teachers’ beefs about the current education system is its emphasis on transferable skills. They point out that skills are not universally transferable, many are subject-specific, and in order to develop expertise in higher-level skills novices need a substantial amount of subject knowledge. Tiger Teachers’ pupils are expected to pay attention to experts (their teachers) and memorise a lot of facts before they can comprehend, apply, analyse, synthesise or evaluate. The model is broadly supported by cognitive science and the Tiger Teachers apply it rigorously to children. But not to themselves, it seems.

For most Tiger Teachers cognitive science will be an unfamiliar subject area. That makes them (like most of us) cognitive science novices. Obviously they don’t need to become experts in cognitive science to apply it to their educational practice, but they do need the key facts and concepts and a basic overview of the field. The overview is important because they need to know how the facts fit together and the limitations of how they can be applied.   But with a few honourable exceptions (Daisy Christodoulou, David Didau and Greg Ashman spring to mind – apologies if I’ve missed anyone out), many Tiger Teachers don’t appear to have even thought about acquiring expertise, key facts and concepts or an overview. As a consequence facts are misunderstood or overlooked, principles from other knowledge domains are applied inappropriately, and erroneous assumptions made about how science works. Here are some examples (page numbers refer to Battle Hymn of the Tiger Teachers):

It’s a fact…

“Teachers’ brains work exactly the same way as pupils’” (p.177). No they don’t. Cognitive science (ironically) thinks that children’s brains begin by forming trillions of connections (synapses). Then through to early adulthood, synapses that aren’t used get pruned, which makes information processing more efficient. (There’s a good summary here.)  Pupils’ brains are as different to teachers’ brains as children’s bodies are different to adults’ bodies. Similarities don’t mean they’re identical.

Then there’s working memory. “As the cognitive scientist Daniel Willingham explains, we learn by transferring knowledge from the short-term memory to the long term memory” (p177). Well, kind of – if you assume that what Willingham explicitly describes as “just about the simplest model of the mind possible”  is an exhaustive model of memory. If you think that, you might conclude, wrongly, “the more knowledge we have in long-term memory, the more space we have in our working memory to process new information” (p.177). Or that “information cannot accumulate into long-term memory while working memory is being used” (p.36).

Long-term memory takes centre stage in the Tiger Teachers’ model of cognition. The only downside attributed to it is our tendency to forget things if we don’t revisit them (p.22). Other well-established characteristics of long-term memory – its unreliability, errors and biases – are simply overlooked, despite Daisy Christodoulou’s frequent citation of Daniel Kahneman whose work focused on those flaws.

With regard to transferable skills we’re told “cognitive scientist Herb Simon and his colleagues have cast doubt on the idea that there are any general or transferable cognitive skills” (p.17), when what they actually cast doubt on is the ideas that all skills are transferable or that none are.

The Michaela cognitive model is distinctly reductionist; “all there is to intelligence is the simple accrual and tuning of many small units of knowledge that in total produce complex cognition” (p.19). Then there’s “skills are simply just a composite of sequential knowledge – all skills can be broken down to irreducible pieces of knowledge” (p.161).

The statement about intelligence is a direct quote from John Anderson’s paper ‘A Simple Theory of Complex Cognition’ but Anderson isn’t credited, so you might not know he was talking about simple encodings of objects and transformations, and that by ‘intelligence’ he means how ants behave rather than IQ. I’ve looked at Daisy Christodoulou’s interpretation of Anderson’s model here.

The idea that intelligence and skills consist ‘simply just’ of units of knowledge ignores Anderson’s procedural rules and marginalises the role of the schema – the way people configure their knowledge. Joe Kirby mentions “procedural and substantive schemata” (p. 17), but seems to see them only in terms of how units of knowledge are configured for teaching purposes; “subject content knowledge is best organised into the most memorable schemata … chronological, cumulative schemata help pupils remember subject knowledge in the long term” (p.21). The concept of schemata as the way individuals, groups or entire academic disciplines configure their knowledge, that the same knowledge can be configured in different ways resulting in different meanings, or that configurations sometimes turn out to be profoundly wrong, doesn’t appear to feature in the Tiger Teachers’ model.

Skills: to transfer or not to transfer?

Tiger Teachers see higher-level skills as subject-specific. That hasn’t stopped them applying higher-level skills from one domain inappropriately to another. In her critique of Bloom’s taxonomy, Daisy Christodoulou describes it as a ‘metaphor’ for the relationship between knowledge and skills. She refers to two other metaphors; ED Hirsch’s scrambled egg and Joe Kirby’s double helix (Seven Myths p.21).  Daisy, Joe and ED teach English, and metaphors are an important feature in English literature. Scientists do use metaphors, but they use analogies more often, because in the natural world patterns often repeat themselves at different levels of abstraction. Daisy, Joe and ED are right to complain about Bloom’s taxonomy being used to justify divorcing skills from knowledge. And the taxonomy itself might be wrong or misleading.   But it is a taxonomy and it is based on an important scientific concept – levels of abstraction – so should be critiqued as such, not as if it were a device used by a novelist.

Not all evidence is equal

A major challenge for novices is what criteria they can use to decide whether or not factual information is valid. They can’t use their overview of a subject area if they don’t have one. They can’t weigh up one set of facts against another if they don’t know enough facts. So Tiger Teachers who are cognitive science novices have to fall back on the criteria ED Hirsch uses to evaluate psychology – the reputation of researchers and consensus. Those might be key criteria in evaluating English literature, but they’re secondary issues for scientific research, and for good reason.

Novices then have to figure out how to evaluate the reputation of researchers and consensus. The Tiger Teachers struggle with reputation. Daniel Willingham and Paul Kirschner are cited more frequently than Herb Simon, but with all due respect to Willingham and Kirschner, they’re not quite in the same league. Other key figures don’t get a mention.  When asked what was missing from the Tiger Teachers’ presentations at ResearchEd, I suggested, for starters, Baddeley and Hitch’s model of working memory. It’s been a dominant model for 40 years and has the rare distinction of being supported by later biological research. But it’s mentioned only in an endnote in Willingham’s Why Don’t Students Like School and in Daisy’s Seven Myths about Education. I recommended inviting Alan Baddeley to speak at ResearchEd – he’s a leading authority on memory after all.   One of the teachers said he’d never even heard of him. So why was that teacher doing a presentation on memory at a national education conference?

The Tiger Teachers also struggle with consensus. Joe Kirby emphasises the length of time an idea has been around and the number of studies that support it (pp.22-3), overlooking the fact that some ideas can dominate a field for decades, be supported by hundreds of studies and then turn out to be profoundly wrong; theories about how brains work are a case in point.   Scientific theory doesn’t rely on the quantity of supporting evidence; it relies on an evaluation of all relevant evidence – supporting and contradictory – and takes into account the quality of that evidence as well.  That’s why you need a substantial body of knowledge before you can evaluate it.

The big picture

For me, Battle Hymn painted a clearer picture of the Michaela Community School than I’d been able to put together from blog posts and visitors’ descriptions. It persuaded me that Michaela’s approach to behaviour management is about being explicit and consistent, rather than simply being ‘strict’. I think having a week’s induction for new students and staff (‘bootcamp’) is a great idea. A systematic, rigorous approach to knowledge is vital and learning by rote can be jolly useful. But for me, those positives were all undermined by the Tiger Teachers’ approach to their own knowledge.  Omitting key issues in discussions of Rousseau’s ideas, professional qualifications or the special circumstances of schools in coastal and rural areas, is one thing. Pontificating about cognitive science and then ignoring what it says is quite another.

I can understand why Tiger Teachers want to share concepts like the limited capacity of working memory and skills not being divorced from knowledge.  Those concepts make sense of problems and have transformed their teaching.  But for many Tiger Teachers, their knowledge of cognitive science appears to be based on a handful of poorly understood factoids acquired second or third hand from other teachers who don’t have a good grasp of the field either. Most teachers aren’t going to know much about cognitive science; but that’s why most teachers don’t do presentations about it at national conferences or go into print to share their flimsy knowledge about it.  Failing to acquire a substantial body of knowledge about cognitive science makes its comprehension, application, analysis, synthesis and evaluation impossible.  The Tiger Teachers’ disregard for principles they claim are crucial is inconsistent, disingenuous, likely to lead to significant problems, and sets a really bad example for pupils. The Tiger Teachers need to re-write some of the lyrics of their Battle Hymn.

References

Birbalsingh, K (2016).  Battle Hymn of the Tiger Teachers: The Michaela Way.  John Catt Educational.

Christodoulou, D (2014).  Seven Myths about Education.  Routledge.

getting the PISA scores under control

The results of the OECD’s 2015 Programme for International Student Assessment (PISA) were published a couple of weeks ago. The PISA assessment has measured the performance of 15 year-olds in Reading, Maths and Science every three years since 2000. I got the impression that teachers and academics (at least those using social media) were interested mainly in various aspects of the analysis. The news media, in contrast, focussed on the rankings. So did the OECD and politicians according to the BBC website. Andreas Schleicher of the OECD mentions Singapore ‘getting further ahead’ and John King US Education Secretary referred to the US ‘losing ground’.

What they are talking about are some single-digit changes in scores of almost 500 points. Although the PISA analysis might be informative, the rankings tell us very little. No one will get promoted or relegated as a consequence of their position in the PISA league table. Education is not football. What educational performance measures do have in common with all other performance measures – from football to manufacturing – is that performance is an outcome of causal factors. Change the causal factors and the performance will change.

common causes vs special causes

Many factors impact on performance. Some fluctuations are inevitable because of the variation inherent in raw materials, climatic conditions, equipment, human beings etc. Other changes in performance occur because a key causal factor has changed significantly. The challenge is in figuring out whether fluctuations are due to variation inherent in the process, or whether they are due to a change in the process itself – referred to as common causes and special causes, respectively.

The difference between common causes and special causes is important because there’s no point spending time and effort investigating common causes. Your steel output might have suffered because of a batch of inferior iron ore, your team might have been relegated because two key players sustained injuries, or your PISA score might have fallen a couple of points  due to a flu epidemic just before the PISA tests. It’s impossible to prevent such eventualities and even if you could, some other variation would crop up instead. However, if performance has improved or deteriorated following a change in supplier, strategy or structure you’d want to know whether or not that special cause has had a real impact.

spotting the difference

This was the challenge facing Walter A Shewhart, a physicist, engineer and statistician working for the Western Electric Company in the 1920s. Shewhart figured out a way of representing variations in performance so that quality controllers could see at a glance whether the variation was due to common causes or special causes. The representation is generally known as a control chart. I thought it might be interesting to plot some PISA results as a control chart, to see if changes in scores represented a real change or whether they were the fluctuations you’d expect to see due to variation inherent in the process.

If I’ve understood Shewhart’s reasoning correctly, it goes like this: Even if you don’t change your process, fluctuations in performance will occur due to the many different factors that impact on the effectiveness of your process. In the case of the UK’s PISA scores, each year similar students have learned and been assessed on very similar material, so the process remains unchanged; what the PISA scores measure is student performance.   But student performance can be affected by a huge number of factors; health, family circumstances, teacher recruitment, changes to the curriculum a decade earlier etc.

For statistical purposes, the variation caused by those multiple factors can be treated as random. (It isn’t truly random, but for most intents and purposes can be treated as if it is.) This means that over time, UK scores will form a normal distribution – most will be close to the mean, a few will be higher and a few will be lower. And we know quite a bit about the features of normal distributions.

Shewhart came up with a formula for calculating the upper and lower limits of the variation you’d expect to see as a result of common causes. If a score falls outside those limits, it’s worth investigating because it probably indicates a special cause. If it doesn’t, it isn’t worth investigating, because it’s likely to be due to common causes rather than a change to the process. Shewhart’s method is also useful for finding out whether or not an intervention has made a real difference to performance.  Donald Wheeler, in Understanding Variation: The key to managing chaos, cites the story of a manager spotting a change in performance outside the control limits and discovering it was due to trucks being loaded differently without the supervisor’s knowledge.

getting the PISA scores under control

I found it surprisingly difficult, given the high profile of the PISA results, to track down historical data and I couldn’t access it via the PISA website – if anyone knows of an accessible source I’d be grateful. Same goes for any errors in my calculations.  I decided to use the UK’s overall scores for Mathematics as an example. In 2000 and 2003 the UK assessments didn’t meet the PISA criteria, so the 2000 score is open to question and the 2003 score was omitted from the tables.

I’ve followed the method set out in Donald Wheeler’s book, which is short, accessible and full of examples. At first glance the formulae might look a bit complicated, but the maths involved is very straightforward. Year 6s might enjoy applying it to previous years’ SATs results.

Step 1: Plot the scores and find the mean.

year 2000* 2003* 2006 2009 2012 2015 mean (Xbar§)
UK maths score 529 495 492 494 492 500.4

Table 1: UK maths scores 2000-2015

* In 2000 and 2003 the UK assessments didn’t meet the PISA criteria, so the 2000 score is open to question and the 2003 score was omitted from the results.

§  I was chuffed when I figured out how to type a bar over a letter (the symbol for mean) but it got lost in translation to the blog post.

pisa-fig-1Fig 1: UK Maths scores and mean score

Step 2: Find the moving range (mR) values and calculate the mean.

The moving range is the differences between consecutive scores, referred to as mR values.

year 2000 2003 2006 2009 2012 2015 mean

(R bar)

UK maths score 529 495 492 494 492
mR values 34 3 2 2 10.25

Table 2: moving range (mR values) 2000-2015

pisa-fig-2Fig 2: Differences between consecutive scores (mR values)

Step 3: Calculate the Upper Control Limit for the mR values (UCLR).

To do this we multiply the mean of the mR values (Rbar) by 3.27.

UCLR = 3.27 x Rbar = 3.27 x 10.25 = 33.52

pisa-fig-3Fig 3: Differences between scores (mR values) showing upper control limit (UCLR)

Step 4: Calculate the Upper Natural Process Limit (UNPL) for the individual scores using the formula UNPL = Xbar + (2.66 x Rbar )

UNPL = Xbar + (2.66 x Rbar ) = 500.4 + (2.66 x 10.25) = 500.4 + 27.27 = 527.67

Step 5: Calculate the Lower Natural Process Limit (LNPL) for the individual scores using the formula LNPL = Xbar – (2.66 x Rbar )

LNPL = Xbar – (2.66 x Rbar) = 500.4 – (2.66 x 10.25) = 500.4 – 27.27 = 473.13

We can now plot the UK’s Maths scores showing the upper and lower natural process limits – the limits of the variation you’d expect to see as a result of common causes.

pisa-fig-4Fig 4: UK Maths scores showing upper and lower natural process limits

What Fig 4 shows is that the UK’s 2000 Maths score falls just outside the upper natural process limit, so even if the OECD hadn’t told us it was an anomalous result, we’d know that something different happened to the process in that year. You might think this is pretty obvious because there’s such a big difference between the 2000 score and all the others. But what if the score had been just a bit lower?  I put in some other numbers:

score  Xbar  Rbar UCLR UNPL LNPL
529 (actual) 500.4 10.25 33.52 527.67 473.13
520 498.6 8 26.16 519.88 477.32
510 496.6 5.5 17.99 511.23 481.97
500 494.6 3 9.81 502.58 486.62

Table 3: outcomes of alternative scores for year 2000

Table 3 shows if the score had been 520, it would still have been outside the natural process limits, but a score of 510 would have been within them.

pisa-fig-5 Fig 5: UK Maths scores showing upper and lower natural process limits for a year 2000 score of 510

ups, downs and targets

The ups and downs of test results are often viewed as more important than they really are; up two points good, down two points bad – even though a two-point fluctuation might be due to random variation.

The process control model has significant implications for target-setting too. Want to improve your score?  Then you need to work harder or smarter. Never mind the fact that students and teachers can work their socks off only to find that their performance is undermined by a crisis in recruiting maths teachers or a whole swathe of schools converting to academies. Working harder or smarter but ignoring natural variation supports what’s been called Ackoff’s proposition – that “almost every problem confronting our society is a result of the fact that our public policy makers are doing the wrong things and are trying to do them righter”.

To get tough on PISA scores we need to get tough on the causes of PISA scores.

 Reference

Wheeler, DJ (1993).  Understanding variation: The key to managing chaos.  SPC Press Inc, Knoxville, Tennessee.

Reforming the SEND system – for good

In the previous post, I claimed that teacher training and targets were two factors that explained why the current SEND system couldn’t work  –  and why it has never worked effectively.  In this post, I’ll explain my claims about teacher training and targets and suggest how the SEND system could become both effective and sustainable.

 Teacher training

For any system – education, health or social care – to meet the needs of a varied population, two ingredients are vital; expertise and flexibility. Practitioners need the knowledge and experience to deal with any needs they might encounter and the system has to be able to adapt to whatever needs arise.

Bizarrely, teachers have always been expected to teach the 98% or so of children who attend mainstream schools, but have only ever been trained to teach the 80% who don’t have SEN, not the 20% who do. And since funding was withdrawn for special education Master’s degrees in the mid-1980s, SEN expertise has gradually leached out of the education system as a whole as special education teachers have retired. It’s only since 2009 that new SENCOs (special educational needs co-ordinators) have been required to be qualified teachers, and only recent appointees are required to have SEN training. There is still a massive gap in SEND expertise within the education system. How can teachers teach children if they don’t know how to meet their educational needs?

Targets

Setting targets sounds like an obvious way to improve performance. You set the target, expect someone to meet it whatever that takes, and provide some sticks and carrots for their encouragement. Targets, accompanied by sticks and carrots, were part and parcel of the early education system but were abandoned because they didn’t work.  And as quality control researchers have been telling us since at least the 1920s, performance depends on the factors that contribute to it. In the current education system, the measure of school performance is actually pupil performance in SATs or GCSEs. But how children perform in tests is influenced by many factors; their health, family circumstances, life events, quality of teaching, their own learning etc. Schools have little or no control over most of those factors, so to measure school performance by pupil performance in tests is pointless.

Despite the evidence, the current education system still sets targets.  And the sticks and carrots expected to encourage schools to raise their (pupil) performance mean that there are no incentives for a school to invest resources in the education of students who are unlikely to improve the school’s test results. If students aren’t going to meet the ‘expected standard’ however hard they or the school try, why invest resources in them? Why not focus on the children likely to meet the ‘expected standard’ with a bit of extra effort.

So, teacher training and targets have been major factors in marginalising the education of children with SEND. But even if the government had a forehead-slapping moment, cried ‘How foolish we’ve been!’, required all teachers to be trained to teach all the children in their classes, and abandoned its ‘expected standards’ criteria, it would take years to transform the system into a SEND-friendly one. Children with SEND don’t have years to spare and their parents have to deal with the here and now. So what needs to be done?

Parents can’t police the system

This post was prompted by a recent conversation I had with a parent carer forum. The parent carer forum was of the opinion that parents with good knowledge of the national framework and their local offer can use that knowledge to get commissioners and providers to make suitable educational provision for children.

It’s certainly true that knowledge of the national framework and the local offer (however incomplete) can help. How effective it is at getting commissioners and providers to meet their statutory obligations is another matter. Since the new system was introduced, I’ve been told repeatedly that it’s improved outcomes for parents and children. Maybe – but I have yet to see any. What I have seen is parents who know the national framework backwards having to resort to mediation, tribunal, formal complaint, the Local Government Ombudsman and in some cases being advised that their only option is Judicial Review – exactly the kind of problems that prompted the revision of the SEN system in 2014.

Until I had the conversation with the parent carer forum, I’d assumed these hurdles were the unwanted and unintended consequences of flaws in legislation that had been rushed through (the pilot study didn’t finish until after the legislation came into force). Then the penny dropped. The only explanation that made sense was that individual parents challenging commissioners and providers is the government’s chosen method of enforcing the new legislation.

That’s a terrible way of enforcing legislation.  For many parents of children with SEND, it’s as much as they can do to hold the family together. To expect parents in already challenging circumstances to police a flawed system that was rushed through at a time when LAs are struggling with huge budget cuts is, to put vulnerable families in harm’s way. Not only is that strategy likely to fail to bring about compliance on the part of commissioners and providers, it’s morally reprehensible.  For 150 years, if a school failed a child parents have been able to appeal to school boards, independent governors or their LEA for support. Not any more. Parents (and children with SEND) are on their own.

What needs changing and who can change it?

The system still needs to change and if parents don’t change it no one else will, so what to do? Since my family entered the SEN ‘world’ 14 years ago, I’ve seen parents fighting lone battles with their LA; the same battles replicated hundreds, if not thousands, of times. I’ve seen parents new to the system set up support or campaign groups only to discover they are just one in a long line of support or campaign groups that have either burned out or at best brought about change that hasn’t actually made much difference on the ground.

What the individual parents and campaign groups have lacked is focus and organisation. I don’t mean they’ve been unfocussed or disorganised; some of them could focus and organise for England. And there’s no doubt that parent groups were instrumental in getting the SEND system changed. It’s rather that there’s been a lot of duplication of effort and the focus has been on single issues or fighting on all fronts at once rather than on the key points in the system that are causing the problems.

I think the key points are these;

  • Mainstream teachers should know how to teach all the children in mainstream schools.
  • Each child needs an education suitable for them as an individual rather than for the average child in their age group, as the law already requires.
  • Assessment and funding should be the responsibility of separate bodies – the new legislation didn’t do away with the LAs’ conflict of interest.
  • There should be an independent body (with teeth) responsible for implementation and compliance that should support parents in their dealings with commissioners and providers. Parents should not have to resort to legal action except in extreme cases.
  • Parents struggling with the system need more support than they are currently offered. A buddying system matching up parents in similar positions dealing with the same local authority might help. As would training in negotiation.

Much of the negotiation undertaken by individual parents and parent groups is with schools, LA officers or the DfE. And problems with the SEND system are generally seen not as being with the structure of the education system or the SEND legislation, but with implementation. But the problem runs deeper than implementation, and deeper than the SEND legislation. It lies with the structure of the education system as a whole, and with the market model espoused by successive governments. Instead of lobbying LA officers and DfE officials who are trying to implement the law as it stands, groups of parents should be lobbying their local councillors and MPs to ensure that teachers are suitably trained, arbitrary targets are abandoned, and responsibility for implementing the system is distributed more widely. These changes won’t require significant new legislation, but they might require a big shift in thinking.

 

 

 

A short history of special education

In 2006 a House of Commons Select Committee described the special educational needs system as ‘no longer fit for purpose’. By September 2014 a new system was in place. Two years on, it’s safe to say it hasn’t been an unmitigated success. To understand why the new system hasn’t worked – and indeed can’t work – it might help to take a look at the history of special educational needs and disability (SEND).

A short history of SEND

Education became compulsory in England in 1880. Some local school boards set up special schools or special classes within mainstream schools for physically ‘handicapped’* children, but provision was patchy. What took people by surprise was the number of mentally handicapped children who turned up to school.

At the time, teachers were expected to teach according to the official Code – essentially a core curriculum – and many schools in the fledgling national educational system were seriously under-resourced. Teachers were often untrained, paper was very expensive (hence the use of rote learning and slates) and many schools operated in conditions like the one below – with several classes in one room. They just weren’t equipped to cope with children with learning difficulties or disabilities.

Shepherd Street School in Preston in 1902

Shepherd Street School in Preston in 1902§

Two Royal Commissions were set up to investigate the education of handicapped children, and reported in 1889 and 1896 respectively. Both recommended the integration of the children in mainstream schools where possible and that special provision (classes or schools) should be made by school boards. The emphasis was on children acquiring vocational skills so they could earn a living. Those with the most severe mental handicap were deemed ‘ineducable’.

The Royal Commissions’ recommendations, and many others made over the next few decades, were clearly well-intentioned. Everybody wanted the best outcomes for the children. The challenge was how to get there. After WW2, concerns about the special education system increased. Parents felt they had little control, the number of pupils in special schools was rising, and children were still being institutionalised or marginalised from society. In 1973 Margaret Thatcher, then Education Secretary, commissioned a review of the education of handicapped children, led by Mary Warnock, whose Committee of Enquiry reported in 1978. A year later Margaret Thatcher became Prime Minister and some of the Warnock recommendations came into force in the Education Act 1981.

The Warnock report introduced a very different way of understanding ‘handicapped’ children. They were no longer seen as being different, but as having special educational needs – as did up to 20% of the school population. Special educational needs were defined in terms of the support children needed, rather than in terms of their physical or mental impairments. What was envisaged was that many children in special schools would gradually migrate to mainstream, supported by Statements of Special Educational Need. And mainstream schools would gradually become more inclusive, adapting their buildings, equipment and teaching methods to meet an ever wider range of educational needs. The new system might have worked well if the rest of the education system hadn’t changed around it.

Context is crucial; one size doesn’t fit all

The Warnock recommendations were made in the context of a very flexible education system. In 1981 Local Education Authorities (LEAs), schools and teachers had a great deal of autonomy in what was taught, how it was taught and when. That all changed with the 1988 Education Reform Act that heralded a compulsory National Curriculum, SATS and Ofsted. Central government essentially wrested control of education from local bodies, something that had been actively opposed for the previous 100 years – few people wanted education to become a political football.

The new education system was at heart a one-size-fits-all affair. Governments find one-size-fits-all systems very appealing. They look as if they are going to be cheaper to run because professional training, equipment and resources can be standardised and performance can be easily measured. Unfortunately for governments, human populations are not one-size, but are very varied. If a universal service is to meet the needs of a whole population, it won’t do that if it’s designed to meet only the needs of the average person. A stark choice faces those designing universal systems; either they can design a system that meets everybody’s needs and resource it properly, or they can design a system that doesn’t meet everybody’s needs and then spend years trying to sort out the ensuing muddle.

The 1880 education system was one-size-fits-all and the next century was spent sorting out the problems that resulted for handicapped children. There was a brief period after 1981 when the education system took a big step towards meeting the needs of all children, but seven years later it flipped back to one-size-fits-all. The last 30 years have been spent trying unsuccessfully to limit the damage for children with SEND.

So what’s the alternative? The answer isn’t further reform of the SEND system, because the causes of the problems don’t lie within the SEND system, but with the broader education system. Two key causes are teacher training and targets – the subjects of the next post.

*I’ve used the term ‘handicapped’ because it was in widespread use in the education system until the Warnock Committee changed the terminology.
§ © Harris Museum and Art Gallery http://www.mylearning.org/victorian-school-and-work-in-preston/images/1-3215/