Science, postmodernism and the real world

In a recent blogpost Postmodernism is killing the social sciences, Eoin Lenihan recommends that the social sciences rely on the scientific method “to produce useful and reliable evidence, or objective truths”.  Broadly, I agree with Eoin, but had reservations about the ‘objective truths’ he refers to. In response to a comment on Twitter I noted;postm quote 1

which was similar to a point made by Eoin, “postmodernism originally was a useful criticism of the Scientific Method or dominant narratives and a reminder of the importance of accounting for the subjective experiences of different people and groups.”

Ben Littlewood took issue with me;

quote 2

In the discussion that followed I said science couldn’t claim to know anything for sure. Ben took issue with that too. The test question he asked repeatedly was:

flat earth

simple question

For Ben,

facts

Twitter isn’t the best medium for a discussion of this kind, and I suspect Ben and I might have misunderstood each other. So here, I’m setting out what I think. I’d be interested in what he (and Eoin) has to say.

reason and observation

Something that has perplexed philosophers for millennia is what our senses can tell us about the world. Our senses tell us there’s a real world out there, that it’s knowable, and that we all experience it in more or less the same way. But our senses can deceive us, we can be mistaken in our reasoning, and different people can experience the same event in different ways. So how do we resolve the tension between figuring out what’s actually out there and what we perceive to be out there, between reason and observation, rationalism and empiricism?

Human beings (even philosophers) aren’t great at dealing with uncertainty, so philosophers have tended to favour one pole of the reason-observation axis over the other. As Karl Popper observes in his introduction to Conjectures and Refutations, some (e.g. Plato, Descartes, Spinoza, Leibnitz) have opted for the rationalist view, in contrast to, for example, Aristotle, Bacon, Locke, Berkeley, Hume and Mill’s empiricism.  (I refer to Popper throughout this post because of his focus on the context and outcomes of the scientific method.)

The difficulty with both perspectives, as Popper points out, is that philosophers have generally come down on one side or the other; either reason trumps observation or vice versa. But the real world isn’t like that; both our reason and our observations tend to be flawed, and both are needed to work out what’s actually out there, so there’s no point trying to decide which is superior. The scientific method developed largely to avoid the errors we tend to make in reasoning and observation.

hypotheses and observations

The scientific method tests hypotheses against observations. If the hypothesis doesn’t fit the observations, we can eliminate it from our enquiries.

It’s relatively easy to rule out a specific hypothesis – because we’re matching only one hypothesis at a time to observations.   It’s much more difficult to come up with an hypothesis that turns out to be a good fit with observations – because our existing knowledge is always incomplete; there might be observations about which we currently have no knowledge.

If  an hypothesis is a good fit with our observations, we can make a working assumption that the hypothesis is true – but it’s only a working assumption. So the conclusions science draws from hypotheses and observations have varying degrees of certainty. We have a high degree of certainty that the earth isn’t flat, we have very little certainty about what causes schizophrenia, and what will happen as a consequence of climate change falls somewhere between the two.

Given the high degree of certainty we have that the earth isn’t flat, why not just say, as Ben does, that we’re certain about it and call it an objective fact? Because doing so in a discussion about the scientific method and postmodernism, opens a can of pointless worms. Here are some of them.

-What level of certainty would make a conclusion ‘certain’? 100%, 75%, 51%?

-How would we determine the level of certainty? It would be feasible to put a number on an evaluation of the evidence (for and against) but that would get us into the kind of arguments about methodology that have surrounded p values. And would an hypothesis with 80% support be considered certain, whereas a competing hypothesis with only 75% support might be prematurely eliminated?

-Who would decide whether a conclusion was certain or not? You could bet your bottom dollar it wouldn’t be the people at the receiving end of a morally suspect idea that had nonetheless reached an arbitrary certainty threshold.  The same questions apply to deciding whether something is a ‘fact’ or not.

-Then there’s ‘objectivity’. Ironically, there’s a high degree of certainty that objectivity, in reasoning and observation, is challenging for us even when armed with the scientific method.

life in the real world

All these problematic worms can be avoided by not making claims about ‘100% certainty’ and ‘objective facts’ in the first place.  Because it’s so complex, and because our knowledge about it is incomplete, the real world isn’t a 100%-certain-objective-fact kind of a place. Scientists are accustomed to working with margins of error and probabilities that would likely give philosophers and pure mathematicians sleepless nights. As Popper implies in The Open Society and its Enemies the human craving for certainty has led to a great deal of knowledge of what’s actually out there, but also to a preoccupation with precise definitions and the worst excesses of scholasticism – “treating what is vague as if it were precise“.*

I declined to answer Ben’s ‘simple question’ because in the context of the discussion it’s the wrong kind of question. It begs further questions about what is meant by certainty, objectivity and facts, to which a yes/no answer can’t do justice. I suspect that if I’d said ‘yes, it is certain that the earth isn’t flat’, Ben would have said ‘there you are, science can be certain about things’ and the can of pointless worms would have been opened. Which brings me on to my comment about postmodernism, that the root cause of postmodernism was the belief that science can produce objective truth.

postmodernism, science and objective truth

The 19th and 20th centuries were characterised by movements in thinking that were in large part reactions against previous movements. The urbanisation and mechanisation of the industrial revolution prompted Romanticism. Positivism (belief in verification using the scientific method) was in part a reaction to Romanticism, as was Modernism (questioning and rejecting traditional assumptions). Postmodernism, with its emphasis on scepticism and relativism was, in turn, a reaction to Modernism and Positivism, which is why I think claims about objective truth (as distinct from the scientific method per se) are a root cause of postmodernism.

I would agree with Eoin that postmodernism, taken to its logical conclusion, has had a hugely detrimental impact on the social sciences. At the heart of the problem however, is not postmodernism as such, but the logical conclusion bit. That’s because the real world isn’t a logical-conclusion kind of a place either.   I can’t locate where he says it, but at one point Popper points out that the world of philosophy and mathematicians (and, I would add, many postmodernists) isn’t like the real world. Philosophy and mathematics are highly abstracted fields. Philosophers and mathematicians explore principles abstracted from the real world. That’s OK as far as it goes. Clearing away messy real-world complications and looking at abstracted principles has resulted in some very useful outcomes.

It’s when philosophers and mathematicians start inappropriately imposing on the real world ideas such as precise definitions, objective truths, facts, logical conclusions and pervasive scepticism and relativism that things go awry, because the real world isn’t a place where you can always define things precisely, be objective, discover true truths, follow things to their logical conclusion, nor be thoroughly sceptical and relativistic. Philosophy and mathematics have made some major contributions to the scientific method obviously, but they are not the scientific method. The job of the scientific method is to reduce the risk of errors, not to reveal objective truths about the world. It might do that, but if we can’t be sure whether it has or not, it’s pointless to make such claims. It’s equally pointless to conclude that if we can’t know anything for certain, everything must be equally uncertain, or that if everything is relative, everything has equal weight. It isn’t and it doesn’t.

My understanding of the scientific method is that it has to be fit for purpose; good enough to do its job. Not being able to define everything exactly, or arrive at conclusively objective truths, facts and logical conclusions doesn’t mean that we can be sure of nothing. Nor does it mean that anything goes. Nor that some sort of ‘balance’ between positivism and postmodernism is required.

We can instead, evaluate the evidence, work with what conclusions appear reasonably certain, and correct errors as they become apparent. The simple expedient of acknowledging that the real world is complex and messy but not intractably complex and messy, and the scientific method can, at best, produce a best guess at what’s actually out there, bypasses pointless arguments about exact definitions, objectivity, truth and logicality. I’d be interested to know what Ben thinks.

Note

* Popper is quoting FP Ramsay, a close friend of Wittgenstein (The Open Society and its Enemies, vol II, p. 11)

References

Popper K. (2003).  The Open Society and its Enemies vol. II: Hegel and Marx, Routledge (first published 1945).

Popper, K. (2002).  Conjectures and Refutations, Routledge (first published 1963).

 

 

 

 

 

 

 

 

 

Advertisements

evolved minds and education: intelligence

The second vigorously debated area that Geary refers to in Educating the Evolved Mind is intelligence. In the early 1900s statistician Charles Spearman developed a technique called factor analysis. When he applied it to measures of a range of cognitive abilities he found a strong correlation between them, and concluded that there must be some underlying common factor that he called general intelligence (g). General intelligence was later subdivided into crystallised intelligence (gC) resulting from experience, and fluid intelligence (gF) representing a ‘biologically-based ability to acquire skills and knowledge’ (p.25). The correlation has been replicated many times and is reliable –  at the population level, at least.  What’s also reliable is the finding that intelligence, as Robert Plomin puts it “is one of the best predictors of important life outcomes such as education, occupation, mental and physical health and illness, and mortality”.

The first practical assessment of intelligence was developed by French psychologist Alfred Binet, commissioned by his government to devise a way of identifying the additional needs of children in need of remedial education. Binet first published his methods in 1903, the year before Spearman’s famous paper on intelligence. The Binet-Simon scale (Theodore Simon was Binet’s assistant) was introduced to the US and translated into English by Henry H Goddard. Goddard had a special interest in ‘feeble-mindedness’ and used a version of Binet’s scale for a controversial screening test for would-be immigrants. The Binet-Simon scale was standardised for American children by Lewis Terman at Stanford University and published in 1916 as the Stanford-Binet test. Later, the concept of intelligence quotient (IQ – mental age divided by chronological age and multiplied by 100) was introduced, and the rest, as they say, is history.

what’s the correlation?

Binet’s original scale was used to identify specific cognitive difficulties in order to provide specific remedial education. Although it has been superseded by tests such as the Wechsler Intelligence Scale for Children (WISC), what all intelligence tests have in common is that they contain a number of sub-tests that test different abilities. The 1905 Stanford-Binet scale had 30 sub-tests and the WISC-IV has 15. Although the scores in sub-tests tend to be strongly correlated, Early Years teachers, Educational Psychologists and special education practitioners will be familiar with the child with the ‘spiky profile’ who has high scores on some sub-tests but low ones on others. Their overall IQ might be average, but that can mask considerable variation in cognitive sub-skills. Deidre Lovecky, who runs a resource centre in Providence Rhode Island for gifted children with learning difficulties, reports in her book Different Minds having to essentially pick ‘n’ mix sub-tests from different assessment instruments because children were scoring at ceiling on some sub-tests and at floor on others. In short, Spearman’s correlation might be true at the population level, but it doesn’t hold for some individuals. And education systems have to educate individuals.

is it valid?

A number of issues have been vigorously debated in relation to intelligence. One is its construct validity. There’s no doubt intelligence tests measure something – but whether that something is a single biologically determined entity is another matter. We could actually be measuring several biologically determined functions that are strongly dependent on each other. Or some biologically determined functions interacting with culturally determined ones. As the psychologist Edwin Boring famously put it way back in 1923 “intelligence is what the tests test”, ie intelligence is whatever the tests test.

is it cultural?

Another contentious issue is the cultural factors implicit in the tests.  Goddard attempted to measure the ‘intelligence’ of European immigrants using sub-tests that included items culturally specific to the USA.  Stephen Jay Gould goes into detail in his criticism of this and other aspects of intelligence research in his book The Mismeasure of Man.  (Gould himself has been widely criticised so be aware you’re venturing into a conceptual minefield.)  You could just about justify culture-specificity in tests for children who had grown up in a particular culture, on the grounds that understanding cultural features contributed to overall intelligence. But there are obvious problems with the conclusions that can be drawn about gF in the case of children whose cultural background might be different.

I’m not going to venture in to bell-curve territory because the vigorous debate in that area is due to how intelligence tests are applied, rather than the content of the tests. Suffice it to say that much of the controversy about application has arisen because of assumptions made about what intelligence tests tell us. The Wikipedia discussion of Herrnstein & Murray’s book is a good starting point if you’re interested in following this up.

multiple intelligences?

There’s little doubt that intelligence tests are valid and reliable measures of the core abilities required to successfully acquire the knowledge and skills taught in schools in the developed industrialised world; knowledge and skills that are taught in schools because they are valued in the developed industrialised world.

But as Howard Gardner points out in his (also vigorously debated) book Frames of mind: The theory of multiple intelligences, what’s considered to be intelligence in different cultures depends on what abilities are valued by different cultures. In the developed industrialised world, intelligence is what intelligence tests measure. If, on the other hand, you live on a remote Pacific Island and are reliant for your survival on your ability to catch fish and navigate across the ocean using only the sun, moon and stars for reference, you might value other abilities. What would those abilities tell you about someone’s ‘intelligence’? Many people place a high value on the ability to kick a football, sing in tune or play stringed instruments; what do those abilities tell you about ‘intelligence’?

it’s all about the constructs

If intelligence tests are a good measure of the abilities necessary for learning what’s taught in school, then fine, let’s use them for that purpose. What we shouldn’t be using them for is drawing conclusions about a speculative entity we’ve named ‘intelligence’. Or assuming, on the basis of those tests, that we can label some people more or less ‘intelligent’ than others, as Geary does e.g.

Intelligent individuals identify and apprehend bits of social and ecological information more easily and quickly than do other people” (p.26)

and

Individuals with high IQ scores learned the task more quickly than their less-
intelligent peers” (p.59)

 

What concerned me most about Geary’s discussion of intelligence wasn’t what he had to say about accuracy and speed of processing, or about the reliability and predictive validity of intelligence tests, which are pretty well supported. It was the fact that he appears to accept the concepts of g, gC and gF without question. And the ‘vigorous debate’ that’s raged for over a century is reduced to ‘details to be resolved’ (p.25) which doesn’t quite do justice to the furore over the concept, or the devastation resulting from the belief that intelligence is a ‘thing’.  Geary’s apparently unquestioning acceptance of intelligence brings me to the subject of the next post; his model of the education system.

 

References

Gardner, H (1983). Frames of Mind: The theory of multiple intelligences. Fontana (1993).

Geary, D (2007).  Educating the evolved mind: Conceptual foundations for an evolutionary educational psychology, in Educating the evolved mind: Conceptual foundations for an evolutionary educational psychology, JS Carlson & JR Levin (Eds). Information Age Publishing.

Gould, SJ (1996).  The Mismeasure of Man.  WW Norton.

Lovecky, D V (2004).  Different minds: Gifted children with AD/HD, Asperger Syndrome and other learning deficits.  Jessica Kingsley.

 

traditional vs progressive: mathematics, logic and philosophy meet the real world

For thousands of years, human beings have been trying to figure out why the world they live in works in the way it does. But it’s only been in the last five hundred or so that a coherent picture of those explanations has begun to emerge. It’s as if people have long had many of the pieces of the jigsaw, but there was no picture on the box. Because a few crucial pieces were missing, it was impossible to put the puzzle together so that the whole thing made sense.

Some of the puzzle pieces that began to make sense to the ancient Greeks involved mathematics – notably geometry. They assumed that if the consistent principles of geometry could be reliably applied to the real world, then it was likely other mathematical principles and the principles underlying mathematics (logic) could too. So philosophers started to use logic to study the fundamental nature of things.

Unfortunately for the mathematicians, logicians and philosophers the real world didn’t always behave in ways that mathematics, logic and philosophy predicted. And that’s why we developed science as we know it today. Scientific theories are tested against observations. If the observations fit the theory we can take the theory to be true for the time being. As soon as observations don’t fit the theory, it’s back to the drawing board. As far as science is concerned we can never be 100% sure of anything, but obviously we can be pretty sure of some things, otherwise we wouldn’t be able to cure diseases, build aircraft that fly, or land probes on Mars.

unknown unknowns

Mathematics, logic and philosophy provide useful tools for helping us make sense of the real world, but those tools have limitations. One of the limitations is that the real world contains unknowns. Not only that, but as Donald Rumsfeld famously pointed out, some unknowns are unknown – we don’t always know what we don’t know. You can work out the unknowns in a set of mathematical equations – but not if you don’t know how many unknowns there are.

Education theory is a case in point. It has, from what I’ve seen, always been a bit of a mess. That’s not surprising, given that education is a heavily derived field; it encompasses a wide range of disciplines from sociology and politics to linguistics and child development. Bringing together core concepts from all relevant disciplines to apply them to education is challenging. There’s a big risk of oversimplifying theory, particularly if you take mathematics, logic or philosophy as your starting point.

That’s because it’s tempting, if you are familiar with mathematics, logic or philosophy but don’t have much experience of messier sciences like genetics, geography or medicine, to assume that the real world will fit into the mathematical, logical or philosophical grand scheme of things. It won’t. It’s also tempting to take mathematics, logic or philosophy as your starting point for developing educational theory on the assumption that rational argument will cut a clear path through the real-world jungle. It won’t.

The underlying principles of mathematics, logic and philosophy are well-established, but once real-world unknowns get involved, those underlying principles, although still valid, can’t readily be applied if you don’t know what you’re applying them to. If you haven’t identified all the causes of low school attendance, say, or if you assume you’ve identified all the causes of low school attendance when you haven’t.

traditional vs progressive

Take, for example, the ongoing debate about the relative merits of traditional vs progressive education. Critics often point out that framing educational methods as either traditional or progressive is futile for several reasons. People have different views about which methods are traditional and which are progressive, teachers don’t usually stick to methods they think of as being one type or the other, and some methods could qualify as both traditional and progressive. In short, critics claim that the traditional/progressive dichotomy is a false one.

This criticism has been hotly contested, notably by self-styled proponents of traditional methods. In a recent post, Greg Ashman contended that Steve Watson, as an author of a study comparing ‘traditional or teacher-centred’ to ‘student-centred’ approaches to teaching mathematics, was inconsistent here in claiming that the traditional/progressive dichotomy was a false one.

Watson et al got dragged into the traditional/progressive debate because of the terminology they used in their study. First off, they used the terms ‘teacher-centred’ and ‘student-centred’. In their study, ‘teacher-centred’ and ‘student-centred’ approaches are defined quite clearly. In other words ‘teacher-centred’ and ‘student-centred’ are descriptive labels that, for the purposes of the study, are applied to two specific approaches to mathematics teaching. The researchers could have labelled the two types of approach anything they liked – ‘a & b’, ‘Laurel & Hardy’ or ‘bacon & eggs’- but giving them descriptive labels has obvious advantages for researcher and reader alike. It doesn’t follow that the researchers believe that all educational methods can legitimately be divided into two mutually exclusive categories either ‘teacher-centred’ or ‘student-centred’.

Their second slip-up was using the word ‘traditional’. It’s used three times in their paper, again descriptively, to refer to usual or common practice. And again, the use of ‘traditional’ as a descriptor doesn’t mean the authors subscribe to the idea of a traditional/progressive divide. It’s worth noting that they don’t use the word ‘progressive’ at all.

words are used in different ways

Essentially, the researchers use the terms ‘teacher-centred’, ‘student-centred’ and ‘traditional’ as convenient labels for particular educational approaches in a specific context. The approaches are so highly specified that other researchers would stand a good chance of accurately replicating the study if they chose to do so.

Proponents of the traditional/progressive dichotomy are using the terms in a different way – as labels for ideas. In this case, the ideas are broad, mutually exclusive categories to which all educational approaches, they assume, can be allocated; the approaches involved are loosely specified, if indeed they are specified at all.

Another dichotomy characterises the traditional/progressive divide; teacher-centred vs student-centred methods. In his post on the subject, Greg appears to make three assumptions about Watson et al’s use of the terms ‘teacher-centred’ and ‘student-centred’ to denote two specific types of educational method;

• because they use the same terms as the traditional/progressive dichotomy proponents, they must be using those terms in the same way as the traditional/progressive dichotomy proponents, therefore
• whatever they claim to the contrary, they evidently do subscribe to the traditional/progressive dichotomy, and
• if the researchers apply the terms to two distinct types of educational approach, all educational methods must fit into one of the two mutually exclusive categories.

Commenting on his post, Greg says “to prove that it is a false dichotomy then you would have to show that one can use child-centred or teacher-centred approaches at the same time or that there is a third alternative that is commonly used”.  I pointed out that whether child-centred and teacher-centred are mutually exclusive depends on what you mean by ‘at the same time’ (same moment? same lesson?) and suggested collaborative approaches as a third alternative. Greg obviously didn’t accept that but omitted to explain why.

Collaborative approaches to teaching and learning were used extensively at the primary school I attended in the 1960s, and I’ve found them very effective for educating my own children. Collaboration between teacher and student could be described as neither teacher-centred nor student-centred, or as both. By definition it isn’t either one or the other.

tired of talking about traditional/progressive?

Many teachers say they are tired of never-ending debates about traditional/progressive methods and of arguments about whether or not the traditional/progressive dichotomy is a false one. I can understand why; the debates often generate more heat than light whilst going round in the same well-worn circles. So why am I bothering to write about it?

The reason is that simple dichotomies have intuitive appeal and can be very persuasive to people who don’t have the time or energy to think about them in detail. It’s all too easy to frame our thinking in terms of left/right, black/white or traditional/progressive and to overlook the fact that the world doesn’t fit neatly into those simple categories and that the categories might not be mutually exclusive. Proponents of particular policies, worldviews or educational approaches can marshal a good deal of support by simplistic framing even if that completely overlooks the complex messiness of the real world and has significant negative outcomes for real people.

The effectiveness of education, in the English speaking world at least, has been undermined by the overuse for decades of the traditional/progressive dichotomy. When I was training as a teacher, if it wasn’t progressive (whatever that meant) it was bad; for some teachers now, if it isn’t traditional (whatever that means) it’s bad. What we all need is a range of educational methods that are effective in enabling students to learn. Whether those methods can be described as traditional or progressive is not only neither here nor there, trying to fit methods into those categories serves, as far as I can see, no useful purpose whatsoever for most of us.

learning styles: what does Tom Bennett* think?

Tom Bennett’s disdain for learning styles is almost palpable, reminiscent at times of Richard Dawkins commenting on a papal pronouncement, but it started off being relatively tame. In May 2013, in a post on the ResearchEd2013 website coinciding with the publication of his book Teacher Proof: Why research in education doesn’t always mean what it claims, and what you can do about it he asks ‘why are we still talking about learning styles?’ and claims “there is an overwhelming amount of evidence suggesting that learning styles do not exist, and that therefore we should not be instructing students according to these false preferences.

In August the same year for his New Scientist post Separating neuromyths from science in education, he tones down the claim a little, pointing out that learning styles models are “mostly not backed by credible evidence”.

But the following April, Tom’s back with a vitriologic vengeance in the TES with Zombie bølløcks: World War VAK isn’t over yet. He rightly – and colorfully – points out that time or resources shouldn’t be wasted on initiatives that have not been demonstrated to be effective. And he’s quite right to ask “where were the educationalists who read the papers, questioned the credentials and demanded the evidence?” But Bennett isn’t just questioning, he’s angry.

He’s thinking of putting on his “black Thinking Hat of reprobation and fury”. Why? Because “it’s all bølløcks, of course. It’s bølløcks squared, actually, because not only has recent and extensive investigation into learning styles shown absolutely no correlation between their use and any perceptible outcome in learning, not only has it been shown to have no connection to the latest ways we believe the mind works, but even investigation of the original research shows that it has no credible claim to be taken seriously. Learning Styles are the ouija board of serious educational research” and he includes a link to Pashler et al to prove it.

Six months later, Bennett teams up with Daniel Willingham for a TES piece entitled Classroom practice – Listen closely, learning styles are a lost cause in which Willingham reiterates his previous arguments and Tom contributes an opinion piece dismissing what he calls zombie theories, ranging from red ink negativity to Neuro-Linguistic Programming and Multiple Intelligences.

why learning styles are not a neuromyth

Tom’s anger would be justified if he were right. But he isn’t. In May 2013, in Teacher Proof: Why research in education doesn’t always mean what it claims, and what you can do about it he says of the VAK model “And yet there is no evidence for it whatsoever. None. Every major study done to see if using learning style strategies actually work has come back with totally negative results” (p.144). He goes on to dismiss Kolb’s Learning Style Inventory and Honey and Mumford’s Learning Styles Questionnaire, adding “there are others but I’m getting tired just typing all the categories and wondering why they’re all so different and why the researchers disagree” (p.146). That tells us more about Tom’s evaluation of the research than it does about the research itself.

Education and training research has long suffered from a serious lack of rigour. One reason for that is that they are both heavily derived fields of discourse; education and training theory draws on disciplines as diverse as psychology, sociology, philosophy, politics, architecture, economics and medicine. Education and training researchers need a good understanding of a wide range of fields. Taking all relevant factors into account is challenging, and in the meantime teachers and trainers have to get on with the job. So it’s tempting to get an apparently effective learning model out there ASAP, rather than make sure it’s rigorously tested and systematically compared to other learning models first.

Review paper after review paper has come to similar conclusions when evaluating the evidence for learning styles models:

• there are many different learning styles models, featuring many different learning styles
• it’s difficult to compare models because they use different constructs
• the evidence supporting learning styles models is weak, often because of methodological issues
• some models do have validity or reliability; others don’t
• people do have different aptitudes in different sensory modalities, but
• there’s no evidence that teaching/training all students in their ‘best’ modality improves performance.

If Tom hadn’t got tired typing he might have discovered that some learning styles models have more validity than the three he mentions. And if he’d read the Coffield review more carefully he would have found out that the reason models are so different is because they are based on different theories and use different (often poorly operationalized) constructs and that researchers disagree for a host of reasons, a phenomenon he’d do well to get his head round if he wants teachers to get involved in research.

evaluating the evidence

Reviewers of learning styles models have evaluated the evidence by looking in detail at its content and quality and have then drawn general conclusions. They’ve examined, for example, the validity and reliability of component constructs, what hypotheses have been tested, the methods used in evaluating the models and whether studies have been peer-reviewed.

What they’ve found is that people do have learning styles (depending on how learning style is defined), but there are considerable variations in validity and reliability between learning styles models, and that overall the quality of the evidence isn’t very good. As a consequence, reviewers have been in general agreement that there isn’t enough evidence to warrant teachers investing time or resources in a learning styles approach in the classroom.

But Tom’s reasoning appears to move in the opposite direction; to start with the conclusion that teachers shouldn’t waste time or resources on learning styles, and to infer that;

variable evidence means all learning styles models can be rejected
poor quality evidence means all learning styles models can be rejected
• if some learning styles models are invalid and unreliable they must all be invalid and unreliable
if the evidence is variable and poor and some learning styles models are invalid or unreliable, then
• learning styles don’t exist.

definitions of learning style

It’s Daniel Willingham’s video Learning styles don’t exist that sums it up for Tom. So why does Willingham say learning styles don’t exist? It all depends on definitions, it seems. On his learning styles FAQ page Willingham says;

I think that often when people believe that they observe obvious evidence for learning styles, they are mistaking it for abilityThe idea that people differ in ability is not controversial—everyone agrees with that. Some people are good at dealing with space, some people have a good ear for music, etc. So the idea of “style” really ought to mean something different. If it just means ability, there’s not much point in adding the new term.

This is where Willingham lost me. Obviously, a preference for learning in a particular way is not the same as an ability to learn in a particular way. And I agree that there’s no point talking about style if what you mean is ability. The VAK model claims that preference is an indicator of ability, and the evidence doesn’t support that hypothesis.

But not all learning styles models are about preference; most claim to identify patterns of ability. That’s why learning styles models have proliferated; employers want a quick overall assessment of employees’ strengths and weaknesses when it comes to learning. Because the models encompass factors other than ability – such as personality and ways of approaching problem-solving – referring to learning styles rather than ability seems reasonable.

So if the idea that people differ in ability is not controversial, many learning styles models claim to assess ability, and some are valid and/or reliable, how do Willingham and Bennett arrive at the conclusion that learning styles don’t exist?

The answer, I suspect, is that what they are equating learning styles with the VAK model, most widely used in primary education. It’s no accident that Coffield et al evaluated learning styles and pedagogy in post-16 learning; it’s the world outside the education system that’s the main habitat of learning styles models. It’s fair to say there’s no evidence to support the VAK model – and many others – and that it’s not worth teachers investing time and effort in them. But the evidence simply doesn’t warrant lumping together all learning styles models and dismissing them outright.

taking liberties with the evidence

I can understand that if you’re a teacher who’s been consistently told that learning styles are the way to go and then discover there’s insufficient evidence to warrant you using them, you might be a bit miffed. But Tom’s reprobation and fury doesn’t warrant him taking liberties with the evidence. This is where I think Tom’s thinking goes awry;

• If the evidence supporting learning styles models is variable it’s variable. It means some learning styles models are probably rubbish but some aren’t. Babies shouldn’t be thrown out with bathwater.

• If the evidence evaluating learning styles is of poor quality, it’s of poor quality. You can’t conclude from poor quality evidence that learning styles models are rubbish. You can’t conclude anything from poor quality evidence.

• If the evidence for learning styles models is variable and of poor quality, it isn’t safe to conclude that learning styles don’t exist. Especially if review paper after review paper has concluded that they do – depending on your definition of learning styles.

I can understand why Willingham and Bennett want to alert teachers to the lack of evidence for the VAK learning styles model. But I felt Daniel Willingham’s claim that learning styles don’t exist is misleading and that Tom Bennett’s vitriol was unjustified. There’s a real risk in the case of learning styles of one neuromyth being replaced by another.

*Tom appears to have responded to this post here and here. With yet another article two more articles about zombies.

References
Coffield F., Moseley D., Hall, E. & Ecclestone, K. (2004). Learning styles and pedagogy in post-16 learning: A systematic and critical review. Learning and Skills Research Council.

Pashler, H. McDaniel, M. Rohrer, D. and Bjork, R. (2008). Learning Styles: Concepts and Evidence. Psychological Science in the Public Interest, 9, 106-116.

memories are made of this

Education theory appears to be dominated by polarised debates. I’ve just come across another; minimal guidance vs direct instruction. Harry Webb has helpfully brought together what he calls the Kirschner, Sweller & Clark cycle of papers that seem to encapsulate it. The cycle consists of papers by these authors and responses to them, mostly published in Educational Psychologist during 2006-7.

Kirschner, Sweller & Clark are opposed to minimal guidance approaches in education and base their case on the structure of human cognitive architecture. As they rightly observe “Any instructional procedure that ignores the structures that constitute human cognitive architecture is not likely to be effective” (p.76). I agree completely, so let’s have a look at the structures of human cognitive architecture they’re referring to.

Older models

Kirschner, Sweller & Clark claim that “Most modern treatments of human cognitive architecture use the Atkinson and Shiffrin (1968) sensory memory–working memory–long-term memory model as their base” (p.76).

That depends on how you define ‘using a model as a base’. Atkinson and Shiffrin’s model is 45 years old. 45 years is a long time in the fast-developing field of brain research, so claiming that modern treatments use it as their base is a bit like claiming that modern treatments of blood circulation are based on William Harvey’s work (1628) or that modern biological classification is based on Carl Linnaeus’ system (1735). It would be true to say that modern treatments are derived from those models, but our understanding of circulation and biological classification has changed significantly since then, so the early models are almost invariably referred to only in an historical context. A modern treatment of cognitive architecture might mention Atkinson & Shiffrin if describing the history of memory research, but I couldn’t see why anyone would use it as a base for an educational theory – because the reality has turned out to be a lot more complicated than Atkinson and Shiffrin could have known at the time.

Atkinson and Shiffrin’s model was influential because it provided a coherent account of some apparently contradictory research findings about the characteristics of human memory. It was also based on the idea that features of information processing systems could be universally applied; that computers worked according to the same principles as did the nervous systems of sea slugs or the human brain. That idea wasn’t wrong, but the features of information processing systems have turned out to be a bit more complex than was first imagined.

The ups and downs of analogies

Theoretical models are rather like analogies; they are useful in explaining a concept that might otherwise be difficult for people to grasp. Atkinson and Shiffrin’s model essentially made the point that human memory wasn’t a single thing that behaved in puzzlingly different ways in different circumstances, but that it could have three components, each of which behaved consistently but differently.

But there’s a downside to analogies (and theoretical models); sometimes people forget that analogies are for illustrative purposes only, and that models show what hypotheses need to be tested. So they remember the analogy/model and forget what it’s illustrating, or they assume the analogy/model is an exact parallel of the reality, or, as I think has happened in this case, the analogy/model takes on a life of its own.

You can read most of Atkinson & Shiffrin’s chapter about their model here. There’s a diagram on p.113. Atkinson and Shiffrin’s model is depicted as consisting of three boxes. One box is the ‘sensory register’ – sensory memory that persists for a very short time and then fades away. The second box is a short-term store with a very limited capacity (5-9 bits of information) that can retain that information for a few seconds. The third box is a long-term store, where information is retained indefinitely. The short-term and long-term stores are connected to each other and information can be transferred between them in both directions. The model is based on what was known in 1968 about how memory behaved, but Atkinson and Shiffrin are quite explicit that there was a lot that wasn’t known.

Memories are made of this

Anyone looking at Atkinson & Shiffrin’s model for the first time could be forgiven for thinking that the long-term memory ‘store’ is like a library where memories are kept. That was certainly how many people thought about memory at the time. One of the problems with that way of thinking about memory is that the capacity required to store all the memories that people clearly do store, would exceed the number of cells in the brain and that accessing the memories by systematically searching through them would take a very long time – which it often doesn’t.

This puzzle was solved by the gradual realisation that the brain didn’t store individual memories in one place as if they were photographs in a huge album, but that ‘memories’ were activated via a vast network of interconnected neurons. A particular stimulus would activate a particular part of the neural network and that activation is the ‘memory’.

For example, if I see an apple, the pattern of light falling on my retina will trigger a chain of electrical impulses that activates all the neurons that have previously been activated in response to my seeing an apple. Or hearing about or reading about or eating apples. I will recall other apples I’ve seen, how they smell and taste, recipes that use apples, what the word ‘apple’ sounds like, how it’s spelled and written, ‘apple’ in other languages etc. That’s why memories can (usually) be retrieved so quickly. You don’t have to search through all memories to find the one you want. As Antonio Damasio puts it;

Images are not stored as facsimile pictures of things, or events or words, or sentences…In brief, there seem to be no permanently held pictures of anything, even miniaturized, no microfiches or microfilms, no hard copies… as the British psychologist Frederic Bartlett noted several decades ago, when he first proposed that memory is essentially reconstructive.” (p.100)

But Atkinson and Shiffrin don’t appear to have thought of memory in this way when they developed their model. Their references to ‘store’ and ‘search’ suggest they saw memory as more of a library than a network. That’s also how Kirschner, Sweller & Clark seem to view it. Although they say “our understanding of the role of long-term memory in human cognition has altered dramatically over the last few decades” (p.76), they repeatedly refer to long-term memory as a ‘store’ ‘containing huge amounts of information’. I think that description is misleading. Long-term memory is a property of neural networks – if any information is ‘stored’ it’s stored in the pattern and strength of the connections between neurons.

This is especially noticeable in the article the authors published in 2012 in American Educator from which it’s difficult not to draw the conclusion that long term memory is a store that contains many thousands of schemas, rather than a highly flexible network of connections that can be linked in an almost infinite number of ways.

Where did I put my memory?

In the first paper I mentioned, Kirschner, Sweller & Clark also refer to long-term memory and working memory as ‘structures’. Although they could mean ‘configurations’, the use of ‘structures’ does give the impression that there’s a bit of the brain dedicated to storing information long-term and another where it’s just passing through. Although some parts of the brain do have dedicated functions, those localities should be thought of as localities within a network of neurons. Information isn’t stored in particular locations in the brain, it’s distributed across it, although particular connections are located in particular places in the brain.

Theories having a life of their own

Atkinson and Shiffrin’s model isn’t exactly wrong; human memory does encompass short-lived sensory traces, short-term buffering and information that’s retained indefinitely. But implicit in their model are some assumptions about the way memory functions that have been superseded by later research.

At first I couldn’t figure out why anyone would base an educational theory on an out-dated conceptual model. Then it occurred to me that that’s exactly what’s happened in respect of theories about child development and autism. In both cases, someone has come up with a theory based on Freud’s ideas about children. Freud’s ideas in turn were based on his understanding of genetics and how the brain worked. Freud died in 1939, over a decade before the structure of DNA was discovered, and two decades before we began to get a detailed understanding of how brains process information. But what happened to the theories of child development and autism based on Freud’s understanding of genetics and brain function, is that they developed an independent existence and carried on regardless, instead of constantly being revised in the light of new understandings of genetics and brain function. Theories dominating autism research are finally being presented with a serious challenge from geneticists, but child development theories still have some way to go. Freud did a superb job with the knowledge available to him, but that doesn’t mean it’s a good idea to base a theory on his ideas as if new understandings of genetics and brain function haven’t happened.

Again I completely agree with Kirschner, Sweller & Clark that “any instructional procedure that ignores the structures that constitute human cognitive architecture is not likely to be effective”, but basing an educational theory on one aspect of human cognitive architecture – memory – and on an outdated concept of memory at that, is likely to be counterproductive.

A Twitter discussion of the Kirschner, Sweller & Clark model centred around the role of working memory, which is what I plan to tackle in my next post.

References

Atkinson, R, & Shiffrin, R (1968). Human memory: A proposed system and its control processes. In K. Spence & J. Spence (Eds.), The psychology of learning and motivation (Vol. 2, pp. 89–195). New York: Academic Press
Clark, RE, Kirschner, PA & Sweller, J (2012). Putting students on the path to learning: The case for fully guided instruction, American Educator, Spring.
Damasio, A (1994). Descartes’ Error, Vintage Books.
Kirschner, PA, Sweller, J & Clark, RE (2006). Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching Educational Psychologist, 41, 75-86.