Designing Winning Worked Examples 1 – Intra Example Features

Mirjam Neelen & Paul A. Kirschner

Most of us agree that the workplace is changing rapidly and that this has consequences for our jobs and for what and how we need to learn. One of the likely consequences is that cognitive (thinking) tasks that must be carried out by people will become increasingly complex (also see our blog on complex skills). We can also expect that these cognitively-complex task-focused skills will include problem-solving, critical thinking, and decision-making.

Between the 1950s and the 1980s, researchers believed that problem-solving was the best instructional approach to solve problems. However, starting with John Sweller’s research program (e.g., Cooper & Sweller, 1987 and Sweller & Cooper, 1985), the evidence has become overwhelming that learning how to solve problems through problem-solving practice is far inferior compared to instruction that also teaches us how to solve problems like worked examples (either paired with practice problems or not) (Atkinson et al, 2000).

At this point, it’s well established that studying worked examples is both more effective (resulting in better performance) and more efficient (higher performance is often attained with equal or less effort invested in the learning and test phases) than practicing problem-solving only (Van Harsel, Hoogerheide, Verkoeijen, & Van Gog, 2020). As a side note, unfortunately, it’s still quite popular to think that solving problems (as a method) leads to effective problem-solving (the goal). Jeroen van Merriënboer calls it out as a ‘meta myth’ in this video interview (around 7:20).

The research on worked examples is particularly relevant to skills acquisition and to learning to carry out complex cognitive tasks and therefore extremely useful in workplace contexts. Learning from worked examples is critical in the initial stages of learning when people are still new to what they need to learn and do; in other words when they’re novices. Note that a novice is not just someone who know little or nothing about a specific domain or topic. You’re also a novice when the task at hand is new for you (e.g., you might have a lot of experience as a learning designer but have never designed worked examples – in that case, you’re a novice for the task ‘designing worked examples’).

We’ve written about worked examples before, but in this blog series we provide a bit more detail. In order for a worked example to be effective, you need to apply some instructional principles which have been helpfully listed by Atkinson and colleagues (2000).

We briefly discuss what a worked example is (but again, for a bit more detail, see our previous blog) and then we dive straight into the first principle.

What’s a worked example?

Jeroen van Merriënboer and Paul Kirschner (2017) define a worked example as “A learning task describing a given begin state, a desired goal state, and a chosen solution; also called a case study if it reflects a real-life problem situation. A process-oriented worked example also pays attention to the problem-solving processes necessary to reach the goal and is called a modeling example” (p. 376).

For learning, the process-oriented part is key. After all, knowing how to do something is not enough to understand it. You also need to know why something is being done. This understanding is necessary for all transfer, both near (to seemingly similar tasks or problems) and far transfer (learning applied in real life situations that may be are different from the learning context) (Van Gog, Paas, & van Merriënboer, 2008; Van Merriënboer & Kirschner, 2017).

Worked examples come in many shapes and forms but what they all have in common is that they’re not only usable for simple, well-structured tasks or problems, but also for complex, ill-structured tasks or problems. In other words, tasks or problems with unknown elements and multiple acceptable solutions. They also possess multiple criteria for evaluating solutions and often require learners to make all kinds of judgments.

This image by Daniel Braith sums the worked example effect up nicely:

Although the early research by Sweller’s group clearly demonstrated that worked examples were very effective for (complex) skills acquisition and tasks that involve problem-solving, Atkinson et al.’s review shows that there are three factors that moderate their effectiveness:

• Intra-example features – how the example is designed and the way the example’s solution is presented.
• Inter-example features – certain relationships among multiple examples and practice problems within one ‘session’ (or lesson).
• Individual differences – the way learners process examples.

As often, the devil is in the detail! Over the next three weeks, we’ll look at all three. This week, we start with intra-example features.

Intra-example features

There are three important intra-example features to consider when designing worked examples: integrating text and diagrams, integrating aural and visual information, and integrating steps and subgoals.

1. Integrate text and diagrams

Tramizi and Sweller (1988) demonstrated that, when learners have to integrate information from diagrammed problems with textual explanations referring to the same concepts, it requires them to split their attention among multiple sources of information and this imposed heavy cognitive load (see left side of the eyeball example image). The solution to the split attention effect (also known as the spatial contiguity principle) is to make sure that the examples integrate textual explanations into the accompanying auxiliary representation (see right hand side of the eyeball example image).

Split attention effect: Learners need to integrate the information in the text with the letters aligned to the image of the eyeball.

Integrated format, following the spatial contiguity principle, preventing the split attention effect

Here’s another example by Borgþór Ásgeirsson, showing the integrated format on the right hand side.

And one more, by Ann Nemesh, just because :).

The split attention effect, doesn’t only occur across space, it also happens over time. This is called the temporal contiguity principle. If instructional materials include both words and pictures, the words and the visual(s) necessary for comprehension need to be presented together and not in consecutive order. If, for example, an animation plays first and the narration explaining the animation happens before or after, then the temporal contiguity effect may hinder learning (Ginns, 2006).

In other words, the visuals and the audio need to occur at the same time. Here’s a nice video by Mike Tyler, demonstrating the principle.

Integrate aural and visual information

We can keep this short and sweet. Research consistently shows that a dual presentation mode (a visual/auditory format, where, for example, a diagram is explained by a narrator) is more effective than a visual/visual format (image combined with text, like the eyeball example above). This is also called the modality principle – the recommendation to use spoken words rather than written text if presenting words and images simultaneously (Clark & Mayer, 2016 and here’s a nice video by Kristin Brown), based on the dual coding theory by Paivio. So, in the case of the eyeball example above, the first arrow would appear, pointing at  the lens and then the narrator would explain that the light passes through the lens of the eye. Next the arrow pointing at the retina would appear, and so forth. Make sure that the text is removed to avoid the redundancy effect (Sweller, Ayres, & Kalyuga, 2011).

The redundancy effect

The redundancy effect may initially appear to be related to the split-attention effect but isn’t really. There are similarities because both effects deal with multiple sources of information such as visuals and text and for both effects goes that we need to pay attention when we use combinations of visual and verbal information. However, the redundancy effect is quite different to the split-attention effect.

The redundancy effect may occur when the learner can understand multiple sources of verbal and visual information separately without the need for mental integration. For example, when we use the same written and spoken text to re-describe a diagram. In the evaporation – condensation – precipitation example (displayed here to prevent that you have to scroll, which would lead to split attention effect), we should either use the text OR remove the text and use narration (while timing the display of the arrow, sun, arrow, cloud in concurrence with the explanation, following the temporal contiguity principle).

Integrating steps and subgoals

This is about how to best structure worked examples to emphasise conceptually meaningful chunks of a problem’s solution or subgoals. Atkinson and colleagues point out that various studies (for example by Catrambone between 1994 and 1998), showed that a clear emphasis on subgoals improves learning. To be more specific,

… structuring worked examples so that they include cues designed to highlight meaningful chunks of information reflecting a problem’s underlying conceptual meaning can enhance a learner’s ability to learn from them and can help learners to be more successful solving novel problems (Atkinson et al., 2000, p 191).

Worked examples with (left) and without (right) subgoal labels, from Margulieux & Catrambone, 2016

Now we’ve seen some examples of how the actual design of the example and its presentation – the three intra-example features – moderate the effectiveness of a worked example. Next week, we’ll discuss the role of inter-example features, or how relationships among multiple examples and practice problems within one ‘session’ (or lesson).

References

Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of educational research, 70(2), 181-214.

Clark, R. C., & Mayer, R. E. (2016). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. Hoboken, NJ: John Wiley & sons.

Cooper, G., & Sweller, J. (1987). Effects of schema acquisition and rule automation on mathematical problem-solving transfer. Journal of educational psychology, 79(4), 347-362.

Ginns, P. (2006). Integrating information: A meta-analysis of the spatial contiguity and temporal contiguity effects. Learning and instruction, 16(6), 511-525.

Margulieux, L. E., & Catrambone, R. (2016). Improving problem solving with subgoal labels in expository text and worked examples. Learning and Instruction, 42, 58-71.

Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and instruction, 2(1), 59-89.

Sweller, J., Ayres, P., & Kalyuga, S. (2011). The redundancy effect. In Cognitive load theory (pp. 141-154). Springer, New York, NY.

Tarmizi, R. A., & Sweller, J. (1988). Guidance during mathematical problem solving. Journal of educational psychology, 80(4), 424-436.

Van Gog, T., Kester, L., & Paas, F. (2011). Effects of worked examples, example-problem, and problem-example pairs on novices’ learning. Contemporary Educational Psychology, 36(3), 212-218.

Van Gog, T., Paas, F., & Van Merriënboer, J. J. (2008). Effects of studying sequences of process-oriented and product-oriented worked examples on troubleshooting transfer efficiency. Learning and Instruction, 18, 211-222. Retrieved from http://dspace.ou.nl/bitstream/1820/1667/1/VanGog-etal_LI_2008.pdf

Van Harsel, M., Hoogerheide, V., Verkoeijen, P., & van Gog, T. (2020). Examples, practice problems, or both? Effects on motivation and learning in shorter and longer sequences. Applied Cognitive Psychology, 34(4), 793-812.

Van Merriënboer, J. J., & Kirschner, P. A. (2017). Ten steps to complex learning: A systematic approach to four-component instructional design. New York, NY: Routledge.