standardised testing: what’s it good for?

A campaign by parents to keep their children off school on Tuesday 3rd May as a protest against SATs prompted a Twitter discussion about the pros and cons of standardised tests. One teacher claimed that they’re important because they hold schools to account. I think that’s a misuse of standardised tests. First, because test results are a poor proxy measure of teaching quality. Second, good teaching (and hard work on the part of the student) are necessary but not sufficient conditions for good test performance. Third, using test results to hold schools to account overlooks the natural variation inherent in large populations.

test results as a measure of teaching quality

Tests such as the National Curriculum Tests (commonly known as SATs) GCSEs and A levels sample students’ recall and understanding of a particular body of knowledge – the KS2 curriculum, GCSE/A level course. The knowledge is sampled because testing the student’s knowledge of all the material in the course would be very time consuming and unwieldy. In other words, test results are a proxy for the student’s knowledge of the course material.

But the course material itself is a proxy for all that’s known about a particular topic. KS2 students learn basic principles about how atoms and molecules behave, GCSE and A level students learn about atomic theory in more detail, but Chemistry undergraduates complain that they have to then unlearn much of what they were taught earlier because it was the simplified version.  So test results are actually a second order proxy for the student’s knowledge of a particular topic.

Then factors other than the student’s knowledge impact on test results. The student might be unwell on the day of the test, or might have slept badly the night before. In the months before the test they might have been absent from school for weeks with glandular fever or their parents might have split up. In other words, test results are affected by factors other than teaching and learning; factors beyond the control of either the school or the student.  In other words, test results are a weak proxy for both the quality of teaching and the student’s knowledge.

good teaching and hard work are necessary but not sufficient for good test performance

There’s an asymmetry between the causes of high and low test results. It’s difficult to get a high test score without hard work on the part of the student and good teaching on the part of the school.   But there are many reasons why a student might get a low score despite hard work and good teaching.

That’s at the individual level. Similarly at the school level it’s safe to conclude that a school with consistently good results in national tests is doing its job properly, but it’s not safe to conclude that a school that doesn’t get consistently good results isn’t.

The education system has been plagued over the years by two false assumptions about student potential. Either that all students have the potential to get good test scores and that good teaching is the key determining factor, or that students from certain demographic groups won’t get good test scores however well they’re taught. In reality it’s more complicated than that, of course. Students from leafy suburbs are more likely to do well in tests for many reasons; even if they are taught badly, they have access to resources that can sometimes compensate for that. Students from the kind of housing estate that motivates Iain Duncan Smith are at a higher risk of adverse life events scuppering their chances of getting good test results no matter how good the teaching at their school. And the older they get, the more adverse life events they are likely to encounter.

So, test results are a pretty good first order proxy for a student’s knowledge of course material. They are a not-so-good second order proxy for a student’s knowledge of the topic the course material represents. And only a weak proxy for quality of teaching.

life is just one damn thing after another*

Those in favour of standardised testing often cite cases of particular schools in deprived areas that have achieved amazing outcomes against the odds. Every child can read by the age of six, or is fluent in French, or whatever.   The implication is that if one school can do it, all schools can. In principle, that’s true. In principle, all head teachers can be visionaries, all teachers can be excellent and all families can buy in to what the school wants to achieve.

But in practice life doesn’t work like that. Head teachers get sick, senior staff have to work part-time because of family commitments, local housing is unaffordable making recruitment a nightmare, or for many families school is just one more thing they can’t quite keep up with.

On top of that, human beings are biological organisms. Like all populations of biological organisms we show considerable variation due to our genes, our environment and interactions between the two. It might be possible to improve test performance across the education system, but there are limits to the improvement that’s possible. Clean water and good sanitation increase life expectancy, but life expectancy doesn’t go on increasing indefinitely once communities have access to clean water and sanitation. Expecting more than 50% of children in primary schools to perform above average simply shows a poor grasp of natural variation – and statistics.

standardised testing: what is it good for?

Standardised testing in primary schools makes sense. It samples children’s knowledge of key material. It allows schools to benchmark attainment. Standardised testing as a performance measure can alert schools to problems that are impacting on children’s learning.

However, the reasons for differences in students’ performance in standardised tests are many and varied. Performance will not improve unless the reasons for poor performance are addressed. Sometimes those reasons are complex and not within the schools’ remit. To address them local families might need better public services, better jobs or better housing – arguably not the core responsibility of a school. Poor teaching might not be involved at all.

However, successive governments haven’t used test results simply as broad indicators of whether a school is on track or whether there are problems that need to be addressed (not necessarily by the school), but as a proxy for teaching quality.  Test results have been used to set performance targets and determine funding, regardless of whether schools can control the factors involved.

This shows a poor understanding of performance management§, and it’s hardly surprising that the huge amounts of money and incessant policy changes thrown at the education system over recent decades have had little impact on the quality of education of the population as a whole.

Notes

*A quotation attributed to Elbert Hubbard, an American  writer who died when the Lusitania was sunk in 1915.

§ The best book I’ve read on performance management is a slim volume by Donald Wheeler called Understanding variation: The key to managing chaos.  A clearly written, step-by-step guide to figuring out if the variation you’ve spotted is within natural limits or not.  Lots of references to things like iron smelting and lumber yards, but still very relevant to schools.

 

 

 

 

Advertisements

joining the dots and seeing the big picture

I’m a tad cynical about charitable bodies these days, especially if they’re associated with academies. Whilst reading their ostensibly ‘independent’ reports I’m on the lookout for phrasing calculated to improve their chances of doing well in the next funding round, or for ‘product placement’ for their services. So a report from the Driver Youth Trust – Joining the Dots: Have recent reforms worked for those with SEND? was a welcome surprise.

The Driver Youth Trust (DYT) is a charity focused on the needs of dyslexic students. Its programme Drive for Literacy is used in ARK schools. I’m well aware of the issues around ‘dyslexia’ and haven’t investigated the Drive for Literacy; in this post I want to focus on Joining the Dots, commissioned by DYT and written by LKMco.

Joining the Dots one of the clearest, most perceptive overviews of the new SEND system that I’ve read. Some of the findings and explanations for the findings are counterintuitive, often a sign of report driven by the evidence rather than what the report writers think they are expected to say. The take-home message is that the new SEND system has had mixed outcomes to date, but the additional autonomy schools now have should allow them to improve outcomes for children regardless, and it presents some inspiring case studies to prove the point.

Here are some of the findings that stood out for me.

SEND reforms interact with the rest of the education system

Reforms to the school system since 2010 have had an even greater impact on young people with SEND than the 2014 Act itself…we find that changes have often enabled those previously succeeding to achieve even better outcomes, while things have only got tougher for those already struggling. As a result unacceptable levels of inequity have merely been reinforced. It is also clear that changes have been inadequately communicated and that many stakeholders (including parents in particular) are struggling to navigate the new landscape.” (p.7)

Fragmentation

“I think that what we did is picked up all the fragments, dropped them on the floor and made them even more fragmented… and now it’s a question of putting them back together in the right order…” – LA service delivery manager (p.15)

SEND pupils and their families have therefore found themselves lost in a system that has yet to reform or regroup.” (p.17)

Funding

Three levels of funding are available for schools: Element 1 is basic funding for all pupil, Element 2 is a notional SEND budget based on a range of factors, and Element 3 is high needs block funding mainly for pupils with EHC plans. The lack of ring-fencing around of the notional SEND budget means that schools can spend this money however they want. (p.20)

Admissions

Pupils with SEND require additional resources and their often lower attainment can impact on the school’s standing in league tables. Parents and teachers reported concerns about admissions policies being stacked against students with SEND.

The local offer

The DfE Final Impact Report for the Pathfinder LAs trialling the new SEND framework found that only 12% of Pathfinder families had looked at their Local Offer and only half of those had found it useful. That picture doesn’t seem to have changed. An FOI request revealed that the number of LA staff with responsibility for SEND varies between 0-382.8 full time equivalent.

Schools

Schools often don’t know what information to give to the LA about their SEND pupils, and the information LAs give schools is sometimes inaccurate. The Plumcroft Primary case study illustrates the point. Plumcroft’s new headteacher tried to improve LA support for pupils with SEND but realised that services available commercially and privately were not only often better, but were actually affordable. As he put it; “If a local authority says ‘no you can’t’ most people just go ‘alright then’ and carry on with the service and whinge about it. Whereas the reality is, you can… there’s no constraint at all.” (p.35)

Categories

The new SEND system does away with the School Action and School Action Plus categories, partly because of concerns that children identified as having SEN were stuck with the label even when it was no longer applicable. The number of children identified with SEN has dropped substantially since, but concerns have been voiced about how children with additional needs are being identified and supported.

Brian Lamb highlights another concern that emerged in the early stages of the legislation, that pupils who would previously have had a Statement, would, under the new system, find it ‘difficult to impossible’ to qualify for an EHCP unless they also have health difficulties or are in care (p.39). This fear doesn’t seem to have materialised, since LAs are now transferring pupils from statements to EHC plans en masse, and it’s in the interest of service providers to ask for an EHC plan to be in place in order to resource any substantial support a child needs.

All teachers are teachers of children with special educational needs

Even though the DfE itself said in 2001 that ‘all teachers are teachers of children with special educational needs’ teacher training funding has consistently failed to recognise this. The new system hasn’t introduced significant improvements.

Exam reform

A shift to making public examinations more demanding in terms of literacy automatically puts students with literacy difficulties at a disadvantage. A student might have an excellent knowledge and understanding of the subject matter, but be unable to get it down on paper. The distribution of assistive technology varies widely between schools.

Reinventing the wheel

LA bureaucracy has been seen as a significant factor in the move over recent years to give schools increased autonomy. This has resulted, predictably, in increased concerns over transparency, accountability, expertise and resources. Many schools are now forming federations in order to pool resources and share expertise. There is clearly a need for an additional tier of organisation at the local level suggesting that it might have been more sensible to improve local authority practice rather than marginalise it.

The content of the report might not be especially cheering, but it makes a change to find a report that’s so readable, informative and insightful.

how to run a government – or not

A decade ago, the then government heard the cries of overworked teachers and implemented an initiative intended to reduce their workload. Statutory planning, preparation and assessment (PPA) time was introduced; teachers were to be freed up from at least 10% of their contact time with students. My children’s primary school sent a letter home about it. Cover was to be provided by teaching assistants.

Since time immemorial, teachers’ working days have been relatively short- if somewhat intense – and they’ve had long vacations (not ‘holidays’, note), giving them ample time for PPA. So why the sudden need to resort to statutory guidance in order to give teachers time to carry out an integral part of their job?

The answer, of course, is that since the Education Reform Act 1988 an entire performance industry had sprouted from the education system, and teachers spent so much time servicing its bureaucratic demands they didn’t have time to do their jobs. But the reasons why PPA time needed safeguarding didn’t appear to have been considered by government.

Baffled, and concerned about both the need for and consequences of this initiative, I wrote to the school, our LA and the Department for Education and Skills (DES). Rather to my surprise, I got replies from each of them, the gist of which was along these lines:

School: *Sigh*. Yes, quite, but we’ll do our best (with yet another initiative that requires yet more reorganisation).

LA: (Phone call from primary HT seconded to LA.) You’ve summed up our concerns exactly. Do we have permission to quote you?

DES: (A month later). We are required to reply to your letter within 20 working days. This constitutes a reply. We’ll respond to the points you made as soon as we can.
(Another month later). We are required to reply to your letter within 20 working days. This constitutes a reply. We’ll respond to your concerns as soon as we can.
(Another month later). We’ve addressed the problem of teachers’ workload. If it doesn’t work, it’s the schools’ and LAs’ fault.

I also wrote to Boots’ Health & Beauty Magazine, which had featured an article about an over-worked teacher. It recommended products she could use to help her sleep, relax, boost her immune system and look less tired. There was no mention of the possibility of reducing her working hours. Their response? Essentially “teachers’ working hours are nothing to do with us”.

Taken at face value, the statutory guidance was successful. Teachers did get additional PPA time. But ten years on it doesn’t appear to have made a significant dent in their workload. That’s because PPA time itself wasn’t the cause of the problem.

How to Run a Government so that Citizens Benefit and Taxpayers Don’t Go Crazy

The impact (positive and negative) of the PPA initiative illustrates the flaw at the heart of Sir Michael Barber’s ‘delivery science’ (formerly known as ‘deliverology’) described in detail in his latest book, How to Run a Government so that Citizens Benefit and Taxpayers Don’t Go Crazy. Barber, a former history teacher, Hackney councillor, NUT official and education policy adviser, was head of Tony Blair’s Prime Minister’s Delivery Unit (PMDU) from 2001-2005. He’s since worked for McKinsey and is currently chief education adviser to educational publishers Pearson.

The Delivery Unit’s job was to ensure that policies were actually implemented. Part of the Prime Ministerial role is to co-ordinate government so having a team dedicated to tracking the progress of particular policies and removing obstacles from their path makes sense. In How to Run a Government, Barber discusses prioritisation, organisation, strategy, planning, routines, problem-solving, ensuring that policies are well-established and that the outcomes are what the electorate wants. He describes the tools and processes the PMDU used to get the job done. On the face of it, his book could be viewed as a practical handbook for anyone who, with little experience, has been tasked with chasing up the progress of particular policies. Or as a project management primer that doesn’t refer directly to the substantial existing literature on the subject. It also offers intriguing insights into how the Blair government functioned.

But scratch the surface and underneath you find something rather different. Barber calls his book How to Run a Government but it’s actually about how to run a Delivery Unit – not quite the same thing. Barber appears to think that the narrow focus required of his Delivery Unit means it’s OK to adopt an equally narrow frame of reference. And for Barber a government and a Delivery Unit amount to the same thing because he attributes inefficiencies and ineffectiveness in modern governments to the failure to implement promised policies. He twice cites Margaret Thatcher’s famous exasperated; “Don’t tell me what. I know what. Tell me how”. For Delivery Units the ‘what’ might be secondary. But for government it’s of primary importance.

what governments do

Traditionally, the focus of central government has been national security; the defence of the realm and the maintenance of law and order. But in recent years governments have gradually taken on what were once local responsibilities – utilities, transport, education, health and welfare. And governments are not just making sure the services are functioning and are properly resourced, they are attempting to manage them. There are good reasons why they shouldn’t. One is that government ministers or civil servants are unlikely to have the necessary specialist domain knowledge. Another is the big risk of services being used for political ends rather than for the benefit of the people. Nonetheless, rightly or wrongly, modern government involves co-ordinating the functions of several complex interconnected systems.

Complex interconnected systems are challenging things. Because they are complex and interconnected, what looks like a minor tweak to one bit can have a massive – sometimes catastrophic – impact on another. For example, removing ‘spare’ hospital beds, increasing GPs’ salaries and reducing social care budgets – all measures that in isolation seem perfectly reasonable – have resulted in the current A&E crisis. It’s imperative that people in government have a good understanding of how systems work. Barber’s book shows that they often don’t. Take for example, the origin of government policies.

where policies come from; a systems perspective

In his book The Open Society and Its Enemies Karl Popper suggests that the focus of a democratic government should be on resolving problems; he illustrates the point by highlighting the disasters that have resulted from attempts to instantiate utopian visions. Suppose for a moment that the next UK government agrees with Popper about its problem-solving role. Suppose also that it has accurately identified the most serious problems besetting the nation.

From a systems perspective, the next task would be to check out the causes of those problems. Causes can be complex, but it’s crucial to pinpoint them accurately in order to get the problems sorted. Sometimes a relatively inexpensive tweak to one system can lead to major improvements in several others – effective sanitation and access to clean water lead to big improvements in health and economic prosperity, for example. The specific policies would consist of whatever action was needed in order to address the causes of the problems. Before implementing the policies, government would need to pilot them to check out their actual impact. This is an essential phase because the complexity of system interactions makes it very difficult to predict the exact outcomes of any given change.

In other words, specific policies would emerge from an analysis of the systems within which the problems are embedded. Policies would be evidence-based and their consequences would be fairly predictable. Pilots would allow unintended or unwanted outcomes to be addressed before implementation.

But that’s not how Sir Michael sees policies. For him, they lie ‘somewhere’ between strategy and implementation (p.101). It doesn’t matter if they originate in ideology (p.62) or in the personal preferences of the PM or President (p.182). Barber recognises that policies can have unintended or unwanted outcomes, but attributes those to choosing the wrong targets (e.g. ones beyond the control of the PM – p.11) or to a failure of a target to ‘tell a good story’ or have sufficient ‘moral purpose’ (p.24). And for Barber, pilots and evidence-based policymaking represent an over-cautious approach (p.7).

One advantage of adopting a narrow frame of reference is that you can just focus on getting the job done. Another advantage is that you can overlook the problems that might emerge as a consequence.

looking past the problems

Barber talks with approval about Calvin Coolidge’s use of routine to implement budget cuts (p.164) but refers only in passing to ‘America’ (never mind anywhere else) being ‘plunged into depression’ even though the 1929 economic crash occurred shortly after Coolidge’s term of office ended and was arguably caused by factors he failed to address. Barber mentions the Iraq war only in passing too, in passages that deal in detail with a 5-hour meeting with Barber about asylum seekers (p.182) and Blair’s political capital leaking away (p.209).

In short, Barber’s narrow focus and narrow frame of reference allow him to get on with the job regardless of the consequences. Impressive results for delivery science; maybe not so impressive for the rest of us.

the emerging science of delivery

Barber claims How to Run a Government is about the ‘emerging science’ of delivery (p.xvii). To be sure he sets out 57(!) rules of ‘delivery science’ throughout the text – also gathered together in a convenient appendix. There’s a nine-page bibliography. And almost 300 pages on what Barber has learned about policy implementation. But that doesn’t make delivery a new ‘emerging science’.

Barber thinks it’s new because there are ‘countless books and manuals’ on various aspects of government, “but on how to get things done in government there is almost nothing. No manuals. Virtually no academic literature.” He goes on to say “Surveying the academic literature on the subject of political science…” (p.xvii). Maybe the clue to the missing manuals lies in where Barber looked for them. Maybe what politicians need to know about policy implementation isn’t filed under ‘political science’ because it’s already well-established in other domains.

Much of Barber’s bibliography is drawn from the literature on management; he doesn’t draw on (highly relevant) project management research; and organisational theory dating back at least to Weber (if not to Thucydides) and systems theory (tried and tested across multiple domains) don’t get a mention. In short, delivery isn’t an emerging science at all. What governments need to know about policy development and implementation is already out there. It just needs to be applied. The fact that recent governments have failed to apply it is worrying.

In his introduction, Barber quotes Charles I, who made a bit of a hash of his period as monarch; “There’s more to the doing than bidding it be done”*. There is indeed, but I’m not convinced Michael Barber has realised quite how much more there is.

Bibliography
Barber, M (2008). Instruction to Deliver: Fighting to Transform Britain’s Public Services. Methuen.
Barber, M (2015). How to Run a Government so that Citizens Benefit and Taxpayers Don’t Go Crazy. Allen Lane.
Popper, K (1945/2003). The Open Society and Its Enemies Vol 1: The Spell of Plato. Routledge.

* The source for this quote is cited as Barber’s Instruction to Deliver (2008). In it, the quote is cited without a source. A Google search appears to attribute it to a 2013 RSA lecture by Stein Ringen, mentioned in How to Run a Government. https://twitter.com/hashtag/RSARingen?src=hash I haven’t watched the Ringen lecture yet, but wouldn’t be surprised if he turns out to cite Barber. Did Charles I actually say it? Maybe we’ll never know.

the new SEN legislation and the Dunkirk spirit

In less than a week an event will take place that’s been awaited with excitement, apprehension, or in some cases with something approaching the Dunkirk spirit. On 1 September part 3 of the Children and Families Act 2014 comes into force. It’s been described as the biggest change to special educational needs in 30 years.

It won’t work
. If I were a betting sort of person, I’d put money on the next government having to review the system again in a couple of years. How can I be so sure? Or so pessimistic? It’s because the ‘problem’ with special educational needs and disabilities (SEND) isn’t the special educational needs and disabilities, it’s the education system. And not just the SEN bit of it – it’s the education system as a whole. To find out why we need to go back in time…

we have a history

Education became compulsory in England in 1870. The new education system was essentially a one-size-fits-all affair focusing on reading, writing and arithmetic. Or more accurately one-size-fits-most; what took the government by surprise was the number of children turning up to school who didn’t fit the education system. Government essentially saw these ‘handicapped’ children as a problem, and its solution was to provide special schools for them. Although the solution made perfect sense, it wasn’t entirely successful. Handicapped children often ended up socially marginalised and sometimes institutionalised, and there were still children in mainstream schools who were struggling.

By the 1970s, the education system had changed considerably. There was more emphasis on an individualised education and local education authorities (LEAs), schools and teachers had a good deal of flexibility in the education they provided. The time was right for Margaret Thatcher as Secretary of State for Education to commission a review of the education of handicapped children, headed by Mary Warnock. The Warnock Committee reported in 1978. It defined special education as ‘provision not generally available in normal schools’ (p.45). In other words it saw the ‘problem’ of special education not as the children but as the educational provision available in mainstream schools. The committee’s recommendations fed into the 1981 Education Act that:

• assumed children would attend mainstream schools where possible
• did away with the old categories of handicap
• introduced the concept of ‘special educational needs’
• gave LEAs a duty to assess children’s special educational needs and to fund the additional provision required for their education.

The Act had the potential to transform the lives of children marginalised by the education system, but it clearly hasn’t done so – not in a good way, anyway. In the last 20 years we’ve had three SEN Codes of Practice, numerous inquiries, reports and tinkerings with SEN legislation and regulations. One select committee described the system as not fit for purpose. So…

what went wrong?

The Warnock recommendations were made in the context of a highly flexible education system. A contemporary account describes a fruitful collaboration between a school for children with visual impairment (VI) and a mainstream junior school, pioneered by a keen LEA officer (Hegarty & Pocklington, 1981). Children with VI were gradually integrated into the mainstream school and teachers trained each other. Everybody won.

In order to undertake such a project, LEAs, schools and teachers needed a fair amount of control over their time and budgets. Projects like this might have eventually been rolled out nationwide, except that within a decade the introduction of a compulsory national curriculum and standardised testing had begun to steer the education system back towards a one-size-fits-all approach. Within a few short years central government had essentially wrested the responsibility for education and its funding from local authorities and education had become a serious ‘political football’. Successive governments have focused on raising educational attainment as an indicator of their own effectiveness as a government and ironically that’s what’s resulted in SEN becoming a problem again in recent years.

Essentially, if you want an efficient one-size-fits-all education system and world-beating exam results it makes perfect sense to remove from the equation children who don’t fit into the system and are unlikely to do well in exams however hard everyone tries. That’s what the government did in the 1890s. If you want an education system that provides all children with an education suitable to their individual needs, you can forget about one-size-fits-all and world-beating exam results; you’ll need a lot of flexibility. That’s what the education system had developed into by the time of the Warnock committee. If you want both you’re likely to end up where we are now.

"Relativity" by MC Escher

“Relativity” by MC Escher

The Warnock committee defined special educational needs in terms of the educational provision ‘generally available in normal schools’. By definition, the better the provision in normal schools, the smaller the number of children who would be deemed to have special educational needs. The committee couldn’t have emphasised the need for SEN training for all teachers more strongly if it had tried, but perversely, the education system appears to have taken a step in the opposite direction.

teacher training

The Warnock committee recommended the inclusion of SEN training in the initial teacher training (ITT) for all teachers. Following the 1981 Education Act, the assumption that many children with SEN would be taught in mainstream schools and that all teachers would be trained in SEN led to the cessation of many special needs teacher training courses. They obviously haven’t been replaced with comparable training in ITT. This, coupled with the retirement of special education teachers and a reduction of the number of children in special schools, has meant that the education system as a whole has suffered a considerable loss of SEN expertise.

Reviews of SEN provision have repeatedly reported concerns about there being insufficient emphasis on SEN in ITT. But it’s only been since 2009 that Special Educational Needs Co-ordinators (SENCOs) have been required to be trained teachers, and only new SENCOs have been required to have SEN training. The current government has allocated additional funding for SEN qualifications (para 53) but only up until last year. This isn’t going to touch the problem. DfE figures for 2011 show that only around 7% of the total education workforce has SEN experience and/or training, and most of those people are concentrated in special schools. And special schools report ongoing difficulties recruiting suitably trained staff. This, despite the fact that the Warnock report 35 years ago pointed out that based on historical data, around 20% of the school population could be expected to need additional educational provision at some time during their school career. The report made it clear that all teachers are teachers of children with special educational needs.

Teachers’ expertise, or lack of it, will have a big impact on the attainment of children with SEN, but that hasn’t prevented government from developing unrealistic targets for all children under the guise of raising aspirations.

expectations of attainment

I mentioned earlier that over the last three decades education has become a ‘political football’. Concern is often expressed over the proportion of young people who leave school functionally illiterate or innumerate or without qualifications, despite evidence that this proportion has remained pretty constant for many years. In the case of literacy, it’s remained stubbornly at around 17%, by bizarre coincidence not far from the equally stubborn 20% figure for children with SEN.

But the possibility that some of those young people might be in the position they’re in because of lack of expertise in the education system – or even because they are never going to meet government’s arbitrary attainment targets and that that might actually be OK – doesn’t seem to have occurred to successive governments. In her keynote address to the inaugural national conference of the Autism Education Trust in 2009 the then Minister for Schools and Learning Sarah McCarthy-Fry, saw no reason why young people with autism shouldn’t achieve 5 A-C grade GCSEs. Some of course might do just that. For others such an aspiration bears no relation to their ability or aptitude, part of the definition for the ‘suitable education’ each child is required, by law, to receive.

Currently, funding for post-16 education requires young people to have or be studying for A-C grade GCSEs in both English and Maths. Post-16 providers are rolling their eyes. Although I can understand the reasoning behind this requirement, it’s an arbitrary target bearing no relation to the legal definition of a suitable education.

it’s the system

Currently, local authorities, schools and teachers are under pressure from the SEN system to make personalised, specialised educational provision for a small group of children, whilst at the same time the education system as a whole is pushing them in the opposite direction, towards a one-size-fits-all approach. This is a daft way to design a system and no matter how much effort individual professionals put in, it can’t work. But it isn’t the SEN system itself that needs changing, it’s teacher expertise and government expectations.

Over recent decades, successive governments have approached education legislation (and legislation in general, for that matter) not by careful consideration of the historical data and ensuring that the whole system is designed to produce the desired outcomes, but essentially by edict. A bit of the education system is wrong, so government has decreed that it should be put right, regardless of what’s causing the problem or the impact of changing part of the system without considering the likely consequences elsewhere.

In systems theory terms, this is known as sub-system optimization at the expense of systems optimization. That mouthful basically means that because all the parts of a system are connected, if you tweak one bit of it another bit will change, but not necessarily in a good way. Policy-makers refer to the not-in-a-good-way changes as unintended and unwanted outcomes.

The new SEN legislation is a classic case of an attempt at sub-system optimization that’s doomed to fail. It requires the education, health and social care sectors to do some joined up thinking and extend the support offered to children with SEND for a further decade – until they are 25 – at a time when all three sectors are undergoing massive organisational change and simultaneously having their budgets cut. It introduces personal budgets at a time when all three sectors are changing their commissioning arrangements. It fails to address the lack of expertise in all three systems. (Recent reports have pointed out that teachers aren’t trained in SEN, GPs don’t have paediatric training and children’s social workers don’t know about child development.) It fails to address the fundamental systems design problems inherent in all three sectors; a one-size-fits-all education system, and health and social care sectors that focus on cure rather than prevention.

This approach to systems design isn’t just daft, it’s incompetent and reprehensively irresponsible. People who have made hopeful noises about the new SEN system have tended to focus on the good intentions behind the legislation. I have no doubt about the good intentions or the integrity of the ministers responsible – Sarah Teather and Edward Timpson – but they have been swimming against a strong tide. Getting through the next few years will be tough. Fortunately, in the world of SEN there’s a lot of Dunkirk spirit – we’re going to need it.

References
Hegarty, S & Pocklington, K (1981). A junior school resource area for the visually impaired. In Swann, W (ed.) The Practice of Special Education. Open University Press/Basil Blackwell.
Warnock, H M (1978). Report of the Committee of Enquiry into the Education of Handicapped Children and Young People. HMSO.

progressively worse

‘Let the data speak for themselves’ is a principle applied by researchers in a wide range of knowledge domains, from particle physics through molecular biology to sociology and economics. The converse would be ‘make the data say what you want them to say’, a human tendency that different knowledge domains have developed various ways of counteracting, such as experimental design, statistical analysis, peer review and being explicit about one’s own epistemological framework.

Cognitive science has explored several of the ways in which our evaluation of data can be flawed; Kahneman, Slovic & Tversky (1982) for example, examine in detail some of the errors and biases inherent in human reasoning. Findings from cognitive science have been embraced with enthusiasm by the new traditionalists, but they appear to have applied the findings only to teaching and learning, not to the thinking of the people who design education systems or pedagogical methods – or those who write books about those things. In Progressively Worse Robert Peal succumbs to some of those errors and biases – notably the oversimplification of complex phenomena, confirmation bias and attribution errors – and as a consequence he draws conclusions that are open to question.

The ‘furious debate’

Peal opens Progressively Worse with a question he says has been the subject of half a century of ‘furious debate’; ‘how should children learn?’ He exemplifies the debate as a series of dichotomies – an authoritative teacher vs independent learning, knowledge vs skills etc. representing differences between traditional and progressive educational approaches. He then provides an historical overview of changes to the British (or, more accurately English – they do things differently in Scotland) education system between 1960 and 2010, notes their impact on pedagogy and concludes that it’s only freedom to innovate that will rescue the country from the ‘damaging doctrine’ of progressive education to which the educational establishment is firmly wedded. (p.1)

Progressive or traditional

For Peal, progressive education has four core themes;

• education should be child-centred
• knowledge is not central to education
• strict discipline and moral education are oppressive and
• socio-economic background dictates success (pp.5-7).

He’s not explicit about the core themes of traditional education, but the features he mentions include;

• learning from the wisdom of an authoritative teacher
• an academic curriculum
• a structure of rewards and examinations
• sanctions for misbehaving and not working (p.1).

He also gives favourable mention to;

subject divisions
the house system
smart blazers, badges and ties
lots of sport
academic streaming
prize-giving
prefects
pupil duties
short hair
silent study
homework
testing
times tables
grammar, spelling and punctuation
school song, colours and motto
whole-class teaching, explanation and questioning
the difference between right and wrong, good and evil
class rankings

I claimed that Peal’s analysis of the English education system is subject to three principle cognitive errors or biases. Here are some examples:

Oversimplification

For the new traditionalists, cognitive load theory – derived from the fact that working memory has limited capacity – has important implications for pedagogy. But people don’t seek to minimise cognitive load only when learning new concepts in school. We also do it when handling complex ideas. On a day-to-day level, oversimplification can be advantageous because it enables rapid, flexible thinking; when devising public policy it can be catastrophic because the detail of policy is often as important as the overarching principle.

Education is a relatively simple idea in principle, but in practice it’s fiendishly complex, involving political and philosophical frameworks, socio-economic factors, systems pressures, teacher recruitment, training and practice and children’s health and development. Categorising education as ‘progressive’ or ‘traditional’ doesn’t make it any simpler. Each of Peal’s four core themes of progressive education is complex and could be decomposed into many elements. In classrooms, the elements that make up progressive education are frequently interspersed with elements of traditional education, so although I agree with him that some elements of progressive education taken to extreme have had a damaging influence, it’s by no means clear that they have been the only causes of damage, nor that other elements of progressive education have not been beneficial.

Peal backs up with numbers his claim that the British education system is experiencing ‘enduring educational failure’ (p. 4). He says the ‘bare figures are hard to ignore’. Indeed they are; what he doesn’t seem to realise is that ‘bare figures’ are also sometimes ambiguous. For example, the UK coming a third of the way down the PISA rankings is not an indication of educational ‘failure’ – unless your definition of success is a pretty narrow one. And the fact that in all countries except the UK literacy and numeracy levels of 16-24 year-olds are better than those of 55-65 year-olds might be telling us more about the resilience of the UK education system in the post-war period than about current literacy standards in other countries. ‘Bare figures’ rarely tell the whole story.

Confirmation bias

Another concept from cognitive science important to the new traditionalists is the schema – the way related information is organised in long-term memory. Schemata are seen as useful because they aid recall. But our own schemata aren’t always an accurate representation of the real world. Peal overlooks the role schemata play in confirmation bias; we tend to construe evidence that confirms the structure of one of our own existing schemata as having higher validity than evidence that contradicts it, even if the evidence overall shows that our schema is inaccurate.

Research usually begins with a carefully worded research question; the question has to be one that can have an answer, and the way the question is framed will determine what data are gathered and how they are analysed to provide an answer. The data don’t always confirm researchers’ expectations; what the data say is sometimes surprising and occasionally counterintuitive. Peal opens with the question; ‘how should children learn?’ but it’s not a question that could be answered using data as it’s framed in terms of an imperative. That’s not an issue for Peal, because he doesn’t use his data to answer the question, but starts with his answer and marshals the data to support it. He’s entitled to do this of course. Whether it’s an appropriate way to tackle an important area of public policy is another matter. The big pitfall in using this approach is that it’s all too easy to overlook data that doesn’t confirm one’s thesis, and Peal overlooks data relating to the effectiveness of traditional educational methods.

Peal’s focus on the history of progressive education during the last 50 years means he doesn’t cover the history of traditional education in the preceding centuries. If Peal’s account of British education is the only one you’ve read, you could be forgiven for thinking that traditional education was getting along just fine until the pesky progressives arrived with their political ideology that happened to gain traction because of the counter-cultural zeitgeist in the 1960s and 1970s. But other accounts paint a different picture.

Traditional education has had plenty of opportunities to demonstrate its effectiveness; Prussia had introduced a centralised, compulsory education system by the late 18th century – one that was widely emulated. But traditional methods weren’t without their critics. It wasn’t uncommon for a school to consist of one class with one teacher in charge. Children (sometimes hundreds) were seated in order of age on benches (‘forms’) and learned by rote not just multiplication tables and the alphabet, but entire lessons, which they then recited to older children or ‘monitors’ (Cubbereley, 1920). This was an approach derived from the catechetical method used for centuries by religious groups and was understandable if funding was tight and pupils didn’t have access to books. But a common complaint about rote learning was that children might memorise the lessons but they often didn’t understand them.

Another problem was the children with learning difficulties and disabilities enrolled in schools when education became compulsory. The Warnock committee reports teachers being surprised by the numbers. In England, such children were often hived off into special schools where those deemed ‘educable’ were trained for work. In France, by contrast, Braille, Itard and Seguin developed ways of supporting the learning of children with sensory impairments and Binet was commissioned to develop an assessment for learning difficulties that eventually transformed into the Stanford-Binet Intelligence Scale.

Corporal punishment for misdemeanours or failure to learn ‘lessons’ wasn’t uncommon either, especially after payment by results was introduced through ‘Lowe’s code’ in 1862. In The Lost Elementary Schools of Victorian England Philip Gardner draws attention to the reasons why ‘dame schools’- small schools in private houses – persisted up until WW2; these included meeting the needs of children terrified of corporal punishment and parents sceptical of the quality of teaching in state schools – often the result of their own experiences.

Not all schools were like this of course, and I don’t imagine for a moment that that’s what the new traditionalists would advocate. But it’s important to bear in mind that just as progressive methods taken to extremes can damage children’s educational prospects, traditional methods taken to extremes can do the same. It’s difficult to make an objective comparison of the outcomes of traditional and progressive education in the early days of the English state education system because comparable data aren’t available for the period prior to WW2, but it’s clear that the drawbacks of rote learning, whole class teaching and teacher authority made a significant contribution to progressive educational ideas being well-received by a generation of adults whose personal experience of school was often negative.

Attribution errors

Not only is the structure of some things complex, but their causes can be too. Confirmation bias can lead to some causes being considered but others being prematurely dismissed – in other words, to wrong causal attributions being made. One common attribution error is to assume that a positive correlation between two factors indicates that one causes another.

Peal attributes the origins of progressive education to Rousseau and the Romantic movement, presumably following ED Hirsch, a former professor of English literature whose specialism was the Romantic poets and who re-frames the nature/nurture debate as Romantic/Classical. Peal also claims that “progressive education seeks to apply political principles such as individual freedom and an aversion to authority to the realm of education” (p.4) supporting the new traditionalists’ view of progressive education as ideologically motivated. Although the pedagogical methods advocated by Pestalozzi, Froebel, Montessori and Dewey resemble Rousseau’s philosophy, a closer look at their ideas suggests his influence was limited. Pestalozzi became involved in developing Rousseau’s ideas when Rousseau’s books were banned in Switzerland. Pestalozzi was also influenced by Herbart, a philosopher intrigued by perception and consciousness, topics that preoccupied early psychologists such as William James, a significant influence on John Dewey. Froebel was a pupil of Pestalozzi interested in early learning who set up the original Kindergärten. Maria Montessori trained as a doctor. She applied the findings of Itard and Seguin who worked with deaf-mute children, to education in general. The founders of progressive education were influenced as much by psychology and medicine as by the Romantics.

Peal doesn’t appear to have considered the possibility of convergence – that people with very different worldviews, including Romantics, Marxists, social reformers, educators and those working with children with disabilities – might espouse similar educational approaches for very different reasons; or of divergence – that they might adopt some aspects of progressive education but not others.

Peal and traditional education

Peal’s model of the education system certainly fits his data, but that’s not surprising since he explicitly begins with a model and selects data to fit it. Although he implies that he would like to see a return to traditional approaches, he doesn’t say exactly what they would look like. Several of the characteristics of traditional education Peal refers to are the superficial trappings of long-established independent schools – bells, blazers and haircuts, for example. Although some of the other features he mentions might have educational impacts he doesn’t cite any evidence to show what they might be.

I suspect that Peal has fallen into the trap of assuming that because long-established independent schools have a good track record of providing a high quality academic education, it follows that if all schools emulated them in all respects, all students would get a good education. What this view overlooks is that independent schools are, and have always been, selective, even those set up specifically to provide an education for children from poor families. Providing a good academic education to an intellectually able, academically-inclined child from a family motivated enough to take on additional work to be able to afford the school uniform is a relatively straightforward task. Providing the same for a child with learning difficulties, interested only in football and motor mechanics whose dysfunctional family lives in poverty in a neighbourhood with a high crime rate is significantly more challenging, and might not be appropriate.

The way forward

The new traditionalists argue that the problems with the education system are the result of a ‘hands off’ approach by government and the educational establishment being allowed to get on with it. Peal depicts government, from Jim Callaghan’s administration onward, as struggling (and failing) to mitigate the worst excesses of progressive education propagated by the educational establishment. That’s a popular view, but not necessarily an accurate one and Peal’s data don’t support that conclusion. The data could equally well indicate that the more government intervenes in education, the worse things get. The post-war period has witnessed a long series of expensive disasters since government got more ‘hands on’ with education; the social divisiveness of the 11+, pressure on schools to adopt particular pedagogical approaches, enforced comprehensivisation, change to a three-tier system followed by a change back to a two-tier one, a constantly changing compulsory national curriculum, standardised testing focused on short-term rather than long-term outcomes, a local inspectorate replaced by a centralised one, accountability to local people replaced by accountability to central government, a constant stream of ‘initiatives’, constantly changing legislation and regulation and increasing micro-management.

A state education system has to be able to provide a suitable education for all children, a challenging task for teachers. The most effective approach found to date for occupations required to apply expertise to highly variable situations is the professional one. Although ‘professional’ is often used simply to denote good practice, it has a more specific meaning for occupations – professionals are practitioners who have acquired high-level expertise to the point where they are authorised to practice without supervision. Regulation and accountability comes via professional bodies and independent adjudicators. This model, used in occupations ranging from doctors, lawyers and architects to builders and landscape gardeners, although not foolproof, has worked well for centuries.

Teaching is an obvious candidate for professional status, but teachers in England have never been treated as true professionals. Initial teacher training has often been shortened or set aside entirely in times of economic downturn or shortages of teachers in specific subject areas, and it’s debatable whether a PGCE provides a sufficient grounding for subject-specialist secondary teachers, never mind for the range of skills required in primary education. Increasing micromanagement by local authorities and more recently by central government has undermined the professional status of teachers further.

I see no evidence to suggest that the university lecturers and researchers, civil servants, local authorities, school inspectors, teaching unions, educational psychologists and teachers themselves that make up the so-called ‘educational establishment’ are any less able than government to design a workable and effective education system – indeed by Peal’s own reckoning, during the period when they actually did that the education system functioned much better.

Despite providing some useful information about recent educational policy, Peal’s strategy of starting with a belief and using evidence to support it is unhelpful and possibly counterproductive because it overlooks alternative explanations for why there might be problems with the English education system. This isn’t the kind of evidence-based approach to policy that government needs to use. Let the data speak for themselves.

References
Cubberley, EP (1920). The History of Education. Cambridge, MA: Riverside Press
Gardner, P (1984). The Lost Elementary Schools of Victorian England: The People’s Education. Routledge.
Kahneman, D., Slovic, P & Tversky A (1982). Judgement under Uncertainty: Heuristics and Biases. Cambridge University Press.
Peal, R (2014). Progressively Worse: The Burden of Bad Ideas in British Schools. Civitas.

not enough jam: select committee report on SEN legislation

Sad person that I am, I love reading Parliamentary Select Committee reports. Select Committees don’t always get it right, but they are an example of democracy at its most transparent. Evidence, written and verbal, is presented verbatim so anyone who cares to can see how the Committee has taken evidence into account in its recommendations – and anyone can learn from the expertise and insights of witnesses. And because government responses to Select Committee reports are also published, anyone can see how much notice the government has taken of the Select Committee – and therefore of the evidence presented. Just before Christmas, the UK’s House of Commons Education Select Committee produced a report on its pre-legislative scrutiny of the draft special educational needs legislation published in September this year. I want to comment on the report in the light of my previous post about upstream and downstream factors in the education system.

Evidence

The first thing that struck me about this report is that it is firmly grounded in the evidence submitted by individuals and organizations involved with special educational needs; almost all the recommendations are based on information from the frontline. The second thing was that it brings a systems perspective to the draft legislation. And the third thing (I have mixed feelings about this) is that I’m not the only Cassandra out there. The impression that the report as a whole conveys is that although the government’s intention and direction of travel in reforming the SEN system is heartily welcomed, that welcome is accompanied by long list of misgivings.

In this post, I want to list some of the key misgivings that emerged from the evidence presented to the Select Committee and then look at the upstream factors that might have prompted them.

Misgivings

Joined-up thinking:
• no statutory duty for health or care services to provide the support specified in the Education, Health and Care (EHC) plans
• questions over how EHC plans will fit in with adult Care and Support plans.

Assessments:
• doubts about the capacity within the system to carry out assessments – without enough people with sufficient expertise, young people will continue to need multiple assessments from different agencies as is currently the case
• a conflict of interest if assessment and service provision are carried out by the same parties.

Accountability:
• lack of clarity about who is accountable to whom for what and how that accountability can be enforced.

SEN Code of practice:
• to be revised, but not as a statutory document laid before Parliament.

Children and young people falling through the net:
• concern about children who have non-educational needs (e.g. pre-schoolers, children with disabilities but not SEN, young people in supervised work placements, apprenticeships)
• concern about children currently on School Action, School Action Plus or lower Statement funding ‘bands’ levels – SA and SA+ categories will disappear.

The Local Offer:
• no minimum standard required – concern that LAs will simply provide a service directory
• no minimum requirement regarding parent participation – a risk that parent participation will be tokenistic

The task of government

As I see it, the primary task of government is to ensure the maintenance of an infrastructure that allows the community it serves to go about its lawful business without let or hindrance. That doesn’t mean government has to design the infrastructure – the evidence suggests that design is far better left to people with relevant expertise. But government does need to maintain an overview – to make sure the different parts of the infrastructure interact effectively, to legislate in order to resolve conflict and to ensure the community’s cash isn’t wasted. Government departments have different areas of responsibility and one of the tasks of the Prime Minister or his/her office should be to ensure that those departments interact effectively. This is a thankless and difficult task and conflict between government departments is unlikely ever to be eradicated, but someone, somewhere needs to have oversight of what’s going on in different departments to ensure that government policy is coherent – that legislation drawn up by one department isn’t going to conflict with legislation drawn up by another, or that budgets aren’t going to scupper policy. Unfortunately, in the case of the draft SEN legislation, this doesn’t appear to have happened.

The biggest reform in SEN legislation for 30 years is being introduced at the same time as the NHS is undergoing the biggest structural change in its history, the school leaving age is being raised to 18, school funding is changing to reflect the increasing autonomy of schools and public sector budgets are being cut year-on-year for the foreseeable future. The SEN legislation rests on several assumptions about the way other public sector services will be working. But no one actually knows how they’ll be working. Witness after witness drew the Committee’s attention to the large number of ‘unknowns’ in the proposed SEN equation.

Sub-system optimization

The SEN legislation is a perfect example of what’s known as sub-system optimization at the expense of whole system optimization. In other words, the proposed SEN sub-system on its own might be great; but the SEN sub-system doesn’t exist on its own, it interacts with several other systems many of which are also undergoing change. Re-designing a service so that it works effectively is a challenging task and one that’s best undertaken by a team of people who have expertise in different aspects of the service, in consultation with a wide range of those working at the front-line – including service users. The reason for this is not to ensure that all parties feel they have been consulted, but to avoid the unforeseen and unwanted outcomes of poorly designed legislation that often end up as part of the judiciary’s caseload. Large-scale or rapid structural changes should be undertaken only when absolutely necessary otherwise there is a big risk of costly knock-on outcomes elsewhere. Over recent decades, the speed with which legislation is introduced seems to have gathered pace. This is certainly true for special educational needs legislation.

The Warnock Committee responsible for the previous re-design of SEN provision was set up in 1974 and consisted of 27 members. Its terms of reference were as follows;

To review educational provision in England, Scotland and Wales for children and
young people handicapped by disabilities of body or mind, taking account of the medical
aspects of their needs, together with arrangements to prepare them for entry into
employment; to consider the most effective use of resources for these purposes; and to
make recommendations
”.

The Committee took nearly four years to report and legislation based on its recommendations wasn’t enacted until 1981. The recent equivalent was the Lamb Inquiry. Its Expert Advisers Group had six members (although it had a larger Reference Group). It was commissioned in 2008 in response to Select Committee reports critical of SEN provision published in 2006 and 2007, reported in 2009 and its recommendations have prompted legislation that has been drafted before pathfinder local authorities’ pilot studies are complete. Its terms of reference are very different from those of the Warnock Committee, focusing on parental confidence in the SEN system:-

In formulating their advice, the Inquiry would:
●● consider whether increasing parental confidence could be best achieved by:
–– making the provision of educational psychology advice ‘arm’s length’ from
local authorities;
–– sharing best practice in developing good relationships between the
authority and parents, through effective Parent Partnership Services and
other local mechanisms;
–– effective practice by schools and local authorities in meeting the needs of
children at School Action Plus;
–– developing the ‘team around the child’ approach in the school stages;
–– other innovative proposals;
●● commission and evaluate innovative projects, in the areas identified, that can
demonstrate the impact on parental confidence of a particular approach;
●● draw on the evidence of other work currently commissioned by the
Department;
●● take into account the evidence of the submissions to the two Select
Committee Reports in 2006 and 2007.

In 1981, the changes resulting from the Warnock report would have been applied to a fairly flexible education system – it would have been up to individual schools or local authorities how implementation took place. A decade later, a compulsory national curriculum and standardized testing had completely transformed that educational landscape. Ironically, the SEN reforms had been both introduced and undermined by changes to the wider education system by the same person – Margaret Thatcher. The constraints imposed on schools and local authorities by performance indicators have led to unforeseen and unwanted outcomes for children with SEN.

Unforseen and unwanted outcomes

The recent Select Committee report draws attention, for example, to the disincentives in the education system for schools to educate children with special needs. The NASUWT cites the case of the flagship Mossbourne Academy in Hackney (founding principal Sir Michael Wilshaw, currently Chief Inspector of Schools) where parents have successfully challenged the school in relation to admission of pupils with SEN. My attempts to find a reference to ‘special educational needs’ on Mossbourne’s website met with failure – as they did on a number of websites for secondary schools in my local area. This might be because the search function on the websites doesn’t work – but frankly, I doubt that’s the cause.

In addition, giving schools increased autonomy and removing them from local authority control has resulted in a lack of clarity about who’s responsible for what and to whom. Edward Timpson, Parliamentary Under-Secretary of State for Education assured the Committee that

all schools will have a vested interest in ensuring that the services that they have available are part of the local offer. Parents will be able to hold them to account for whether they do or they do not” (para.138)

I suspect the Committee wasn’t assured, since this means that the only way for parents to ultimately hold schools to account will involve taking legal action against them – which many parents will be unable or unwilling to do.

In short, making sure that a suitable education is available to all children and that schools actually provide that education is no longer safeguarded in the design of the system – by, for example, ensuring that all education providers have ready access to relevant expertise and resources and that there’s a clear pathway of accountability that doesn’t require parents to resort to legal action. Instead, government appears to see its role as having good intentions.

In response to the Select Committee’s suggestion that the draft clauses in the legislation lacked substance the Minister stated;

“I am confident—and it is borne out in many of the conversations I have already had with many of those who played a part in bringing it together—that it does illustrate, very clearly, the ambition of this Government and many other people to ensure that the system we move to is a vast improvement on the previous system” (para.13)

That might be perfectly true, but ‘ambition’ isn’t all that’s required to design and run an education system, health or care service. As I see it, over recent decades governments have become increasingly involved in the design of public sector services for political reasons, but are reluctant to take responsibility for flaws in the design of those systems – flaws that are unsurprising given the unavoidable lack of relevant expertise of government ministers and their special advisers.

Upstream factors

I said I’d look at upstream and downstream issues. Not surprisingly, the factors I flagged in my previous post – lack of expertise, insufficient resources and capacity and inadequate needs analysis, cropped up in the evidence submitted to the Select Committee.

Expertise The NUT drew attention to the fact that schools were already reporting difficulties accessing specialist advice regarding children with School Action or School Action Plus support, implying that at least some teachers don’t currently have the expertise required to support children at these levels. Witnesses also asked for the legislation to require SENCOs to have appropriate training.

Resources and capacity The difficulties experienced in accessing specialist advice suggest some local authorities are already cutting back on support services. One headteacher had been told by her local authority that children currently with lower band Statement funding would not be eligible for EHC plans. Funding cuts across the public sector have significant implications for the viability of the SEN proposals.

Needs analysis The task of local authorities is, and always has been, to provide services that meet the needs of the local population. By now, LAs should have accumulated sufficient information about the needs of local children to have a reasonably accurate idea about what services those children need. But currently, many LAs prioritise the needs of children with severe difficulties, suggesting that services are not based on need, but on budgets. The NHS hasn’t been around for as long as local authorities, but 60 years is quite long enough to have formed a good awareness of what children’s needs are. But long waits for diagnoses, to see specialists or get wheelchairs suggest that again, children’s healthcare is based on budgetary considerations rather than needs.

Not enough jam

In a letter to the Education Select Committee, Sarah Teather, responsible for the Green Paper that initially set out the proposals for change to the SEN system, asked whether there was ‘a case for extending the scope of the integrated provision requirement to all children and young people, including those with SEN’ (para.73). The consensus amongst witnesses was that doing this would mean ‘spreading the jam too thinly’.

One can appreciate concerns about limited resources being diverted from those who need them most, but this response does beg a couple of questions: The first is ‘Why are children categorized as those who need jam or those who don’t?’ Difficulties that require educational, health or social support are distributed across the population and vary during the lifetime of the individual – some children need more support than others and some might need support at some times but not at others. In other words, all children need access to the jam, even if they never need the jam itself. The second question is ‘Is there enough jam in the pot?’ If service design is based on the outcomes of a needs analysis, there should be. If service design is based on budgets, then assessments determine children’s eligibility for support, not what their needs are. And if there isn’t enough support to go round, this means that there are likely to be children who need support but who aren’t getting it.

The saying ‘children are our future’ might sound trite, but it’s still true. Child abuse by individuals has, rightly, received a great deal of attention in recent years. But public sector systems that withhold support from children who need it is also abusive and needs to be addressed as a matter of urgency. Treating children with special educational needs and disabilities as second-class citizens is a self-fulfilling prophecy.

the dead sheep in the stream and new special needs legislation

Many years ago, on a walking holiday in the Lake District with friends, the conversation turned to how clean the water in the mountain streams might be. One of the more intrepid members of our party said; “So it would be OK for me to drink this?” “Probably,” replied an experienced fell-walker, “But not if there’s a dead sheep in the beck higher up.”

mountain stream

I was reminded of this incident by my local parent carer group newsletter. Not that there was anything wrong with the newsletter itself, but it included a couple of articles about the proposed legislative changes for the support of children with special educational needs and disabilities (SEND). The proposals include;

• joint planning and commissioning of services by local authorities and Clinical Commissioning Groups
• individual support specified in a single Education, Health and Care (EHC) plan
• support extended to age 25 and
• that families of children with EHC plans should have the option of a personal budget.

The proposals have, overall, been welcomed. However, concerns have also been expressed.

The changes were first put forward in March 2011 in a Green Paper entitled “Support and aspiration: A new approach to special educational needs and disability”. I was involved in the responses of several groups to the consultation that followed and the general feeling was that it was difficult to comment on the viability of the proposals because they hadn’t been set out in enough detail. The Department for Education’s response to the consultation, “Support and aspiration: A new approach to special educational needs and disability – progress and next steps” was published in May this year and draft legislation was published in September. The Department for Education appointed 20 pathfinder authorities to pilot and evaluate the proposed changes, with a final evaluation due in the summer of 2013 – almost a year later. Queries from interested parties about how the proposals would be implemented were generally greeted with advice to wait for the pathfinder reports. In the event, not surprisingly, the pathfinder evaluation has been extended and it is likely that the legislative programme will be delayed until after the final pathfinder evaluation is published.

In discussions about these changes, I’ve felt like a Cassandra, prophesying doom and gloom whilst many around me have remained relentlessly upbeat. After all, the fact that there’s a SEND Bill at all shows that the current government recognizes there are problems with the current system, and the proposed changes show that the DfE knows what the biggest ones are. Many children are likely to benefit from the changes. But in my view the proposals merely tweak problems caused by much more fundamental factors in the system, and that if these factors aren’t addressed, the current set of problems will simply be exchanged for another. One parent I sat next to in a meeting kept saying “At least it couldn’t be worse than the current system.” Well, actually it could. It could be like the situation prior to the Warnock report in 1978, which recognized that many able children were denied a suitable education because of a physical disability, and many less able children were considered ineducable. Or, as I suggested, the legislation might result in a set of problems that are simply different to the current ones.

Components of a service

A service – whatever it is and whoever it’s for – has to have several components. First, expertise. The people offering the service need to know how to accurately assess their clients’ needs and how best to meet them. Second, capacity and resources. An effective service will need enough people with the right expertise and sufficient equipment, materials, buildings etc. Thirdly, before designing the system the service will need to carry out a requirements analysis for all the people who need the service – usually described as a needs analysis in the case of children with SEND. No service would have an unlimited budget, so once planners and commissioners know what the needs are, they can then decide what expertise and resources are going to be most cost effective and what service users can probably manage without. This might seem self-evident and might appear to be what central and local government are doing already, but since the current system of support for children with SEND clearly isn’t working – and I would argue that it never has worked, in terms of ensuring that most children with SEND achieve their full potential – there must be something going wrong somewhere.

What’s going wrong?

The Department for Education seems to have decided that the problem lies in the way support services are planned, commissioned and delivered. Planning and commissioning aren’t joined up enough, despite local authorities having integrated children’s services for nearly a decade. The process of statutory assessment is too cumbersome and takes too long, even though in principle, assessments could be completed within weeks, rather than months. Support doesn’t go on for long enough, despite adult services being available. Local authorities aren’t allocating finance in the most effective way, even though it’s their job to do so. Consequently, the planning, commissioning and delivery of the system are being changed. Since the people who designed the current system presumably thought it would work, and viable processes for planning, commissioning and delivery are already in place, a key question does not appear to have been asked; what made the system go wrong in the first place?

The dead sheep in the stream

This is where the sheep in the stream analogy comes in. Imagine that you live in a farmhouse at the foot of a mountain. The farm is too remote for a mains water supply and for three hundred years the inhabitants have relied on water from a stream fed by a spring halfway up the mountainside. The purity of the water is renowned locally and the only problems ever reported have been that the stream flows sluggishly during extreme droughts. Then one day everyone at the farm gets sick. The illness is identified as a water-borne one and further investigation reveals the source – the body of a sheep lying in the stream just below the spring, hidden in a densely wooded area where sheep rarely stray. The farming family is advised to boil their drinking water or install a purification unit, but they might not need to do anything that involves that level of inconvenience or expense. It’s quite likely that simply removing the body of the sheep from the stream and letting the water flow for a couple of days would allow farmers to continue drinking the spring water for the next three hundred years without mishap – provided no more bodies end up in the stream.

Requiring local authorities to undertake joint planning and commissioning, implementing EHC plans, extending children’s services to 25 and providing personal budgets are all the equivalent of the farming family boiling their water in pots instead of kettles or installing a more sophisticated purification unit – while there’s still a dead sheep in the stream that’s contaminating the water. So what’s the equivalent of the sheep? I’d say it was a problem with each of the three components of service provision I mentioned earlier – expertise, capacity and resources, and requirements analysis – not downstream in the system near the point of delivery where most of the amendments are taking place, but further upstream.

Expertise

First, let’s look at expertise. Recent independent reports have indicated a lack of expertise with regard to children, in the education (Lamb, 2009), health (Kennedy, 2010) and social care (Munro, 2011) sectors. Despite the Warnock recommendation that children with SEND be taught in mainstream schools where possible being implemented since 1978, it’s only since 2009 that teachers have been required to have SEN training and that new special educational needs co-ordinators (SENCOs) have had to be qualified teachers. Teaching Assistants (TAs), who now make up around 25% of the mainstream school workforce, are generally not qualified teachers and don’t necessarily have any educational training, but are often the people who spend most time with children with SEN. A recent study (Webster & Blatchford, 2012) revealed that teachers aren’t usually trained to work with TAs, so many TAs are having to work ‘on the hoof’ in the classroom with little or no preparation with a child with learning or behavioural problems. The study found that when TAs worked with the rest of the class for part of the lesson so teachers could spend time with the children with SEN, the achievement of the pupils improved and teachers understood their learning difficulties better. What’s puzzling is how this situation arose in the first place. Here’s an extract from a piece about SENCO training published in the Times Educational Supplement in May 2009.

The [training] courses have been set up to address serious concerns about the perceived “low status” of Sencos and to raise the profile of special needs and disabilities in schools.”

I find it intriguing that although the professional status of SENCOs and poor awareness of special educational needs might be relevant issues, the TES reporter frames SENCO training in those terms of rather than in terms of the expertise required to help all children learn. What does this say about perceptions of SEN?

Capacity and resources

A second factor is capacity and resources; I’ll talk about capacity first. A recurring problem for parents of children with SEND is how long it takes to see professionals who can carry out assessments. Often all children get is repeated assessments; because of limited service capacity sometimes parents (and occasionally teachers) are expected to implement therapies even though they have no idea what might be causing the child’s problems or what outcomes to expect. Another recent report (Bercow, 2008) suggested that speech and language therapy in England was a postcode lottery, and there doesn’t seem to have been a significant improvement since then. The British Psychological Society has expressed concerns (not for the first time) about cuts in the number of educational psychologists employed by local authorities. Google ‘shortage occupational therapist’; and you’ll find reports from various parts of the globe. Then there’s resources. Parents report problems getting wheelchairs and nappies; even the NHS website says that there might be a waiting list for assessments (waits for the actual wheelchair aren’t even mentioned). My local occupational therapy service apologized for the delay in providing therapy for my son. One problem was that they hadn’t been able to access his school to show teachers how to integrate exercises into his school day. Another obstacle was that because their equipment takes an hour to put up and an hour to dismantle, the only time they were able to book a room large enough and available for long enough for them to treat several children in one day was during the school summer holidays.

Requirements analysis

And then there’s the requirements analysis. Under the 1989 Children Act, local authorities are required to keep a register of children with disabilities. This should provide the information they need to enable them to design support services. The register is a voluntary one in the sense that parents volunteer information about their children, and there are obviously questions over what qualifies as a disability, so at best such a register is only going to provide approximate information about the needs of children with disabilities in a given locality. But an approximation is all that’s required. In the past twenty years, it should have been possible to form a fairly accurate picture of local needs, trends over time and year-to-year fluctuations. But judging by recent reports, support for children with SEND has been getting worse, rather than better. So what’s gone wrong?

I suggest that because education, health and social care systems have been evolving piecemeal during this time, national government initiatives have cut across local authorities’ ability to use data to design effective services. For example, following the Warnock report in 1978, local authorities were encouraged to educate children with disabilities in mainstream schools where possible. An inspiring example of this is the collaboration between a mainstream junior school and a school for children with visual impairment described by Hegarty and Pocklington (1981). At that time, local authorities and individual schools had complete control over such initiatives. Then in 1988, the Education Reform Act introduced a compulsory national curriculum, followed in 1991 by national curriculum assessments, commonly known as SATs. Although there might have been good reasons for introducing both, they have each had an impact on the Warnock recommendation for the inclusion of SEND pupils in mainstream schools. If the performance of schools is assessed by pupils’ performance in standardized tests, systems pressures will inevitably lead to a tendency to marginalize pupils with SEND, either overtly – by schools discouraging admittance or by formal or informal exclusions – or covertly by simply not allocating sufficient resources to their education. Add to this the absence of SEN from initial teacher training and the reduction in SEN expertise within the education system as a whole due to a focus on children within the normal range and the closure of special schools, and no amount of tinkering with statutory assessments or who holds budgets will be able to compensate.

Failure demand

Overlooking shortcomings in factors that are upstream in a system means that whatever you do to problems downstream, they won’t get fixed. In fact the upstream issues create the need for further resources that wouldn’t be needed if the upstream problems were fixed. This phenomenon is what John Seddon calls failure demand – demand created solely by failures of system design. A common failure demand in the case of children with SEND is that avoiding early intervention in an attempt to avoid unnecessary costs often means that simple problems become complex ones, requiring expensive interventions later on. Not to mention the sometimes permanent damage done to a child’s self-esteem and the time wasted by teachers, parents and professionals trying to get problems resolved in the meantime. Providing sufficient resources to meet needs might not cost more; in fact, once failure demand is eliminated, costs can go down.

In short, until teachers, healthcare and social care professionals are trained to meet the needs of all children, not just those within the normal range, until there are enough people with that training working within the education, health and social care sectors, and until there are enough materials, equipment and space available to meet the needs of all children, the needs of all children will not, and cannot be met.

References

Hegarty S. and Pocklington K. (1981). “A junior school resource area for the visually impaired” in W. Swann (Ed.) The Practice of Special Education, Basil Blackwell/Open University Press.

Webster R. & Blatchford P. (2012). “Supporting learning?:.How effective are teaching assistants?” in P. Adey & J. Dillon (Eds) Bad Education: Debunking myths in education, McGraw Hill.

Acknowledgements

Photograph: Tullynaglack, Donegal, copyright Louise Price, used under Creative Commons http://commons.wikimedia.org/wiki/File:Mountain_stream,_Tullynaglack_-_geograph.org.uk_-_974248.jpg