Double-Barrelled Learning for Young & Old

Paul A. Kirschner & Mirjam Neelen

When accompanying text with images, a learner learns better. However, this only works when they are combined properly! 

A good combination of words and images facilitates learning. This is based on the dual coding theory (Paivio, 1971), which uses the idea that humans need either verbal associations or visual imagery to increase learning effectiveness. The theory assumes that we have two specific yet connected cognitive subsystems. One subsystem is specialized in representing and processing nonverbal objects/events (i.e., imagery), and the other subsystem is specialized in dealing with language. To put it simply, the theory discusses how we process verbal and visual information in our brains.

While the theory is not new, it’s receiving quite a bit of attention. As a matter of fact, it actually informs how we think about and design multimedia learning environments; but this will be discussed later. An example of this interest can be seen in the work of Céleste Meijs and her colleagues (2016) who investigated how to best teach vocabulary to elementary school students. The question was if it’s most effective to teach them through images, speech, or written text? The results showed that children aged 7 to 16, learned better through images and recognised and remembered the words better than their peers who were taught through written or spoken words and that this effect grew stronger as students got older.

Now what’s that all about? In our brain written and spoken words only get coded once, however images of words get coded TWICE, first visually, then verbally. Celeste and her colleagues explained the results in terms of single or dual coding of information in which the phonological loop is involved.

Conclusion: Images leave double and thus stronger traces in our brains. You can watch this video and/or the image below from Rich Mayer’s Cambridge Handbook of Multimedia Learning (2005) to get your head around it.

Integrative model of text and picture comprehension

However, dual coding theory takes it even further. If the same information is properly offered to you in two different ways, it enables you to access more working memory capacity. This means that you can benefit from access to both visual and verbal memory capacity. It’s known as the working memory model from Baddeley and Hitch (you can also watch this video on the model).

Schematic of Baddeley and Hitch’s working memory model

In addition, you not only boost the information traces in your long-term memory (as two connected traces are stronger than one single trace) – it also allows you to remember or recognise the information in two different ways. In order to achieve this effect, you need to actually use both types of processes (i.e., verbal and visual). In other words: you need to use your double-barrelled shotgun effectively. By combining an image with a complementary word (written or – preferably – spoken), you’re using both capacities effectively as you’re using both a verbal/semantic process (i.e., you’re deciphering spoken/written words) and an iconic process (i.e., you’re deciphering images). In other words, you need to make use of the two working memory ‘compartments’; one for each process. They can partly complement each other and that way result in more processing capacity[1].

An example of ‘complementing’ working memory capacity that way comes to the surface in a study conducted by Simone Herrlinger and her colleagues (2016). They investigated whether written learning materials for biology in combination with complementary images for children aged 9 to 11 facilitated learning. They also studied to what extent the modality of the text (i.e., written or spoken) influenced learning effectiveness. The complementary imagery had a positive effect in general, however far and foremost when added to spoken text. Furthermore, the effect was larger when children are younger because reading a text themselves in combination with ‘reading’ the images was actually beyond their cognitive capacities, and especially their visual channel. In this case, text and pictures would not be integrated into one coherent mental model, and effective learning wouldn’t take place. This is called the modality effect / modality principle (see this video) and in this case it means that the visual modality got overloaded because young children’s reading skills are still developing big time so they have to work hard to decode the words. In combination with processing images, it simply overloads their visual memory capacities.

i.ytimg.com

Years earlier, Paul Ginns (2004) did a meta-study and also found that learners learned indisputably better from instructional materials that combined images with spoken text than from combining images with written text.

Let’s go back to our statement that the combination of words and images needs to be right. A bad combination is for example when someone reads a text out loud in parallel to the same text in writing. Who hasn’t attended a presentation where the presenter was reading their PowerPoint slides out loud? The audience listening to the presenter (and in schools of course the audience being the student) then needs to conduct one and the same verbal/semantic decoding process in two different ways.  This is when working memory actually gets overloaded and it’s proven over and over again that have trouble learning that way. This is called the redundancy effect.

image by onderzoekonderwijs

These effects/principles and many more are based upon Rich Mayer and Roxana Moreno’s Cognitive Theory of Multimedia Learning (CTMML)[2] which is based upon three assumptions, namely: there are two separate channels (auditory and visual) for processing information; there is limited channel capacity; and that learning is an active process of filtering, selecting, organizing, and integrating information.

Cognitive Theory of Multimedia Learning 

And here you can see two nice videos about the theory and his five principles for reducing extraneous processing (i.e., processing that interferes with learning).

Last but not least, thanks to The Learning Scientists Megan Smith (@DrSmithRIC) and Yana Weinstein (@doctorwhy), some recommendations to effectively use double-barrelled learning (they also have a fantastic poster on dual coding that you can download). Although Smith and Weinstein approach dual coding from an instruction approach, we have added some comments in italics below when using double-barrelled learning for self-directed learning purposes.

• Slow down. Take your time presenting with words and images and divide them in small parts (chunks). The same goes when you’re learning independently. Divide the ‘learning load’ in digestible chunks and present it to yourself with words and images.
• When explaining a diagram, do it verbally and not through text in or around the imagery. In a workplace context, you might need to explain a diagram to your peers or manager.  Use signalling (e.g. animations or highlighting) when you talk through the diagram. Don’t explain it in writing on the slide.
• Offer images and text at the same time so that the learner doesn’t have to remember the one part while processing the other, like the image below. This of course is an excellent way of putting learning content together for yourself as well.

Learn How to Study Using… Dual Coding

• Prevent working memory from becoming overloaded and therefore don’t offer a spoken text in parallel with the same text in writing. Like the PowerPoint example!
• Only use ‘useful’ information. Skip images or sounds for the sake of engagement or fun. These are only distracting and increase cognitive load.

Are you ready to implement double-barrelled learning?

References

Ginns, P., (2005). Meta-analysis of the modality effect. Learning and Instruction,15, 313-331.

Herrlinger, S., Höffler, T.N., Opferman, M. & Leutner, D., (2016). When do pictures help learning from expository text? Multimedia and modality effects in primary schools. Research in Science Education. Retrieved from https://link.springer.com/article/10.1007/s11165-016-9525-y.

Mayer, R.E., (2005). The Cambridge Handbook of Multimedia Learning. Cambridge: Cambridge University Press.

Meijs, C., Hurks, P., Wassenberg, R., Feron, F. J. M. & Jolles, J. (2016), Inter-individual differences in how presentation modality affects verbal learning performance in children aged 5 to 16. Child Neuropsychology, 22, 818-836.

Paivio, Allan, (1971). Imagery and Verbal Processes. New York, NY: Holt, Rinehart and Winston.

[1]     Note: The analogy ‘double-barrelled’ doesn’t mean that you actually double your working memory from 4±1 to 8±2!

[2]     This theory originally was about nine ways to improve multimedia learning. Mayer has since expanded this to include five principles for reducing extraneous processing. Three principles for managing essential processing and two principles for fostering generative processing.