A discussion on Twitter raised a lot of questions about working memory and the evidence supporting direct instruction cited by Kirschner, Sweller and Clark. I couldn’t answer in 140 characters, so here’s my response. I hope it covers all the questions.
Kirschner Sweller & Clark’s thesis is;
• working memory capacity is limited
• constructivist, discovery, problem-based, experiential, and inquiry-based teaching (minimal guidance) all overload working memory and
• evidence from studies investigating efficacy of different methods supports the superiority of direct instruction.
Therefore, “In so far as there is any evidence from controlled studies, it almost uniformly supports direct, strong instructional guidance rather than constructivist-based minimal guidance during the instruction of novice to intermediate learners.” (p.83)
Sounds pretty unambiguous – but it isn’t.
1. Working memory (WM) isn’t simple. It includes several ‘dissociable’ sensory buffers and a central executive that monitors, attends to and responds to sensory information, information from the body and information from long term memory (LTM) (Wagner, Bunge & Badre, 2004; Damasio, 2006).
2. Studies comparing minimal guidance with direct instruction are based on ‘pure’ methods. Sweller’s work on cognitive load theory (CLT) (Sweller, 1988) was based on problems involving use of single buffer/loop e.g. mazes, algebra. New items coming into the buffer displace older items, so buffer capacity would be limiting factor. But real-world problems tend to involve different buffers, so items in the buffers can be easily maintained while they are manipulated by the central executive. For example, I can’t write something complex and listen to Radio 4 at the same time because my phonological loop can’t cope. But I can write and listen to music, or listen to Radio 4 whilst I cook a new recipe because I’m using different buffers. Discovery, problem-based, experiential, and inquiry-based teaching in classrooms tends to more closely resemble real world situations than the single-buffer problems used by Sweller to demonstrate the concept of cognitive load, so the impact of the buffer limit would be lessened.
3. For example, Klahr & Nigam (2004) point out that because there’s no clear definition of discovery learning, in their experiment involving a scientific concept they ‘magnified the difference between the two instructional treatments’ – ie used an ‘extreme type’ of both methods – that’s unlikely to occur in any classroom. Essentially they disproved the hypothesis that children always learn better by discovering things for themselves; but children are unlikely to ‘discover things for themselves’ in circumstances like those in the Klahr & Nigam study.
It’s worth noting that 8 of the children in their study figured out what to do at the outset, so were excluded from the results. And 23% of the direct instruction children didn’t master the concept well enough to transfer it.
That finding – that some learners failed to learn even when direct instruction was used, and that some learners might benefit from less direct instruction, comes up time and again in the evidence cited by Kirschner, Sweller and Clark, but gets overlooked in their conclusion.
I can quite see why educational methods using ‘minimal instruction’ might fail, and agree that proponents of such methods don’t appear to have taken much notice of such research findings as there are. But the findings are not unambiguous. It might be true that the evidence ‘almost uniformly supports direct, strong instructional guidance rather than constructivist-based minimal guidance during the instruction of novice to intermediate learners’ [my emphasis] but teachers aren’t faced with that forced choice. Also the evidence doesn’t show that direct, strong instructional guidance is always effective for all learners. I’m still not convinced that Kirschner, Sweller & Clark’s conclusion is justified.
References
Damasio, A (2006) Descartes’ Error. Vintage Books
Klahr, D & Klahr, D, & Nigam, M. (2004). The equivalence of learning paths in early
science instruction: Effects of direct instruction and discovery learning.
Psychological Science, 15, 661–667.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning.
Cognitive Science, 12, 257–285.
Wagner, A.D., Bunge, S.A. & Badre, D. (2004). Cognitive control, semantic memory and priming: Contributions from prefontal cortex. In M. S. Gazzaniga (Ed.) The Cognitive Neurosciences (3rd edn.). Cambridge, MA: MIT Press.
logicalincrementalism 
Reblogged this on Primary Blogging.
Responding to your po9ints.
1. So what? What is the impact of this on Kirschner, Sweller and Clark’s recommendations? Does it invalidate any of their argument? Their argument is based on evidence e.g. the worked-example effect. They use various models to explain this evidence. If more complex models are available then these are only relevant if they change the argument in some way.
2. I am not sure about the multi-modality of real-world problems. Let’s accept, for the sake of argument, that there is a slightly larger capacity for processing information in multiple modes (e.g. visual and auditory together) than there is for a single mode (e.g. auditory alone). The total capacity is still limited and I suspect its not really much greater. In real world problems, much of these additional modes will effectively represent extraneous cognitive load. So, although there may be additional capacity, it is still not useful and has potential to overload. True, in real world situations, people may need to switch rapidly between modes e.g. a shop assistant doing a stock take who suddenly has to deal with a customer, but I struggle to think of examples that are intrinsically multi-modal. Ironically, one of the few that I can think of is taking notes from a lecture. Views differ on this but I personally tend to print out any notes from a presentation so that students can focus on the presentation and not the notes – in other words, I tend to view the note-taking as extraneous.
The examples that you give really do not help your case. I also listen to Radio 4 when I am cooking (often the Today programme!). However, I am usually cooking something that I have cooked many times before and therefore the cooking procedure is largely automatized. I can therefore focus my attention on the radio. When I do cook something new from a recipe book, I switch the radio off. Another example is that I listen to music or the radio whilst driving. Again, driving has been automatized. However, if I am driving somewhere new then when I get to any parts of the journey where I need to think about my route, I tend to turn the radio off. I think you need to demonstrate an intrinsically multi-modal real-world problem in order to support your claim that ‘Discovery, problem-based, experiential, and inquiry-based teaching in classrooms tends to more closely resemble real world situations than the single-buffer problems used by Sweller to demonstrate the concept of cognitive load, so the impact of the buffer limit would be lessened’. If all you are saying is that real-world problems contain extraneous cognitive load then we all agree. But you don’t get better at this by simply being exposed to it. You help novices to learn by initially limiting any extraneous load so that they can focus on automatizing the key skills / concepts / processes. Once they have developed these schema, they are better able to cope with more complex situations involving extraneous load.
3. You seem to have both missed the main point of the Klahr and Nigam study as well as committing a profound logical error. An argument that I often encounter from inquiry etc. advocates runs something like this, “Yes, direct instruction is better for teaching basic skills but inquiry develops deeper understanding.” This is what Klahr and Nigam were investigating. They presumably had no doubt that a greater number of students would learn the control of variables principle by direct instruction. However, the question was whether those few who did discover it for themselves somehow ‘learnt it better’. The results of the transfer experiment give a resounding ‘no’. In other words, not only do more students learn by DI but there are no costs in terms of a lesser form of understanding. Yes, it is plainly true that not all students will learn everything that is presented to them first time via direct instruction. However, this absolutely does not demonstrate that ‘some learners might benefit from less direct instruction’. This is a classic non sequitur. In order to justify this statement, you would need to demonstrate that other methods produce better results for these students. The Klahr and Nigam study does not.
Given your scepticism of Kirschner, Sweller and Clark and your views about testing ‘extreme’ forms of discovery and direct instruction, I am surprised that you have not used the Alfieri et. al. paper in your arguments.
Click to access edu-103-1-1.pdf
Many inquiry learning advocates refer to this meta-analysis because it seems to provide support for the notion that ‘guided discovery’ is an effective form of instruction.
However, before you get too excited, it is worth making a few points. The effect size for guided discovery learning is small (d=0.30). Given that this is a meta-analysis of studies that were not necessarily RCTs, this is well within the realms of placebo / hawthorn effects. Secondly, guided discovery is compared against ‘other types of instruction’ including explicit instruction but also unguided discovery. It is therefore hard to know what this shows. Thirdly, drawing on Marzano, the authors use a very strange definition of guided discovery that includes ‘feedback, worked examples, scaffolding and elicited explanations’. I would consider all of these to be part of good direct instruction. The worked example is pretty much the quintessence of direct instruction and scaffolding is there presumably to reduce cognitive load. I would go as far as to say that this is looking a lot like Hattie’s definition of direct instruction.
One difference between guided discovery and direct instruction may, however, be an inquiry-learning focus on providing information ‘just in time’ rather than in an ordered way derived from abstract principles. I believe that there are costs involved in this in terms of coherence.
Thanks for commenting.
1. So what? It’s possible that I have completely misunderstood what KSC are saying. But, as I read their paper they are;
a) lumping together a range of learning/teaching approaches as if they are essentially the same
b) extrapolating some domain-specific findings to all domains
c) using evidence to support their thesis, rather than ‘letting the data speak’ and drawing conclusions from it.
You mentioned the worked examples evidence. As I understand it, KSC are saying that in some domains, carefully structured worked examples make explicit the deep structure of problem-solving strategies, and students learn the deep structure more effectively than by problem-solving alone. The references they cite relate to a range of mathematical problems, particularly in relation to computer-aided learning.
So why don’t they just say that? It’s a useful thing to know. It doesn’t follow that worked examples are equally useful in history, geography or chemistry. They probably are, but the evidence KSC cite doesn’t say so. But you wouldn’t know that unless you’d checked out the references.
And why do they introduce the topic by saying “ A worked example constitutes the epitome of strongly guided instruction, whereas discovering the solution to a problem in an information-rich environment similarly constitutes the epitome of minimally guided discovery learning.”
Do they? This looks like KSC’s opinion, to which they are entitled, but why open with a comment like that? To me, that’s a prime example of using evidence to support a thesis, rather than ‘letting the data speak’ and drawing conclusions from it. The worked example section is an example of my points a, b & c.
The more complex WM model is relevant here. All the references in the worked example section relate to mathematical problems. They all involved what Baddeley calls the ‘visuo-spatial sketchpad’ – a WM buffer in the dorsal stream of the visual system – that processes information related to where objects are, as distinct from what they are. Sensory information isn’t all the same, and it isn’t processed in exactly the same way. One obvious difference is that we can see several objects simultaneously, whereas we process speech linearly, meaning that cognitive load issues will be different for the phonological loop and the visuo-spatial sketchpad – they are situated in different neural networks.
It’s quite possible that the worked examples effect does apply to all knowledge domains, and to information extracted from all sensory modalities, but we can’t assume that. But because KSC use a very early model of WM that doesn’t take into account the different sensory buffers, they do seem to be assuming that.
2. Multi-modal problems. In the real world, most situations we encounter are multi-modal. Usually they involve visual, auditory and tactile information. Educational environments rely heavily on the auditory mode – lots of talking. All I’m saying is that the Sweller cognitive load research involved problem-solving using a single buffer and that distributing information across the buffers by using multi-modal approaches might lessen the load e.g. documentaries rather than lectures, practicals rather than talks.
I’m not disputing the underlying principle that cognitive load can be reduced by developing schemata.
3. I do get the point of the Klahr & Nigam study. I am not ‘an inquiry etc advocate’. The point I’m making is that all we can conclude from their data is what the data tell us. You’re absolutely right that their results disprove the hypothesis that problem-solving results in children learning better. My comment about ‘some learners might benefit from less direct instruction’ was based on the sections in KSC about individual differences and knowing less after instruction, Klahr & Nigram’s comment on specific learner characteristics favouring different approaches and their finding that direct instruction isn’t a failsafe. What about the 23% who didn’t master the concept using direct instruction? I’m not saying they need to use ‘discovery learning’ instead, just that they shouldn’t be swept under the carpet just because direct instruction might be more effective than no instruction at all. (Does that actually ever happen?)
Why would I cite Alfieri et al? I’m not trying to make a case for the superiority of ‘minimal guidance’ methods; but I am concerned that they are being prematurely dismissed. And that direct instruction is being viewed as a magic bullet.
One comment about RCTs. They are very susceptible to the ‘garbage-in-garbage-out’ effect. Your paragraph on Alfieri shows how challenging it is to define constructs precisely. The problems with precise definitions suggest that direct instruction vs minimal guidance is actually a false dichotomy. RCTs don’t help if we’re comparing apples with a fruit salad. It probably be more useful to compare the efficacy of different educational methods taking into account different knowledge domains and learner characteristics.
My scepticism isn’t about the efficacy of direct instruction; it’s about the way scientific evidence is being handled and what conclusions are being drawn about direct instruction. Taking liberties with theory and with conclusions from the data is just as unhelpful as constructivist speculation about discovery learning. If educational reformers are basing their case on the existence of scientific evidence, I think it’s incumbent on them to base their case on the scientific evidence, not on what they think the evidence is saying, what they would like it to say, or unjustified extrapolations from it.
Nice debate. In summary, the evidence we have is narrow and, generally, doesn’t warrant many of the conclusions drawn from it?
In my view, yes.