Designing Winning Worked Examples 2 – Inter Example Features

Mirjam Neelen & Paul A. Kirschner

Worked examples come in many shapes and forms. They’re not only useful for simple, well-structured tasks or problems, but also for complex, ill-structured tasks or problems. In other words, tasks or problems with unknown elements and multiple acceptable solutions. They also possess multiple criteria for evaluating solutions and often require learners to make all kinds of judgments.

Although we know that in general worked examples were very effective for (complex) skills acquisition and tasks that involve problem-solving, we also know that we need to consider the three factors that moderate their effectiveness:

• Intra-example features – how the example is designed and the way the example’s solution is presented.
• Inter-example features – certain relationships among multiple examples and practice problems within one ‘session’ (or lesson).
• Individual differences – the way learners process examples.

As often, the devil is in the detail!

Last week, we discussed the role of intra-example features (how the example is designed and the way the example’s solution is presented) in worked examples. In addition to designing the worked examples themselves, we also need to consider how to sequence and arrange the examples during instruction. This is where the lesson design and the ‘inter example features’ come in.

Inter Example Features: Lesson Design

Multiple examples

When people need to learn to carry out complex tasks, they need multiple examples to increase chances of transfer (Atkinson et al., 2000). It’s now widely accepted that studying multiple examples increases people’s ability to use information flexibly from the various examples, which supports transfer (e.g., to the job).

Of course, the million dollar question is how many examples are needed to support problem solving effectively. Providing each problem with an appropriate example would simply require too many examples. Another approach would be to combine teaching rules and procedures with demonstrating and explaining prototypical examples (Sweller & Cooper, 1985).

This is where the variability in surface stories comes in.

Variability in surface stories

What if there’s a way to select the ‘right’ or ‘best’ examples (Weber, 1996)? Weber and Bögelsack (1995) observed that novices use examples that were presented to them for solving an initial task to solve subsequent tasks. However, it’s clear that they don’t usually really understand the dissimilarities between the different tasks. They often simplify the analogical mapping process, which then results in an erroneous solution. They also interpret problems by looking at ‘surface structure characteristics’ (Chi, Feltovich, & Glaser, 1981). For example, when a previous problem also dealt with ‘a moving object’ and then a novice uses the wrong formula (e.g., velocity instead of accelareation).

This is why novice learners also need support to retrieve and select the best examples to help them solve new problems as they don’t necessarily know which example is best to guide the problem-solving process for a new task – the so called ‘retrieval problem’ (Weber, 1996).

A human or a ‘machine tutor’ needs to explain “what the relevant similarities and dissimilarities are between different tasks, how a previous solution can be mapped properly onto a solution of the new task, and how the old solution has to be adapted to fit the new problem”. Weber calls this the ‘explanation problem’ (p 3). This is known as a modelling example. In other words, in addition to ‘just’ presenting the example itself, the underlying procedures or rules need to be made explicit as well.

After all, we need learners to discriminate between two or more types of problems and solve each correctly. We know that novices tend to “pay too much attention to problem context and too little to problems’ deeper conceptual structures” (Atkinson et al., 2000, p. 193).

Research by Quilici and Mayer (1996) deals with this (you can access the article here if you want to take a closer look). This is what they did:

---

Structure-emphasising example set: They created three examples of t-test problems on one sheet, three examples of chi-square problems on a second, and three examples of correlation on a third. In the structure-emphasising set, three different surface stories were used for the f-test problems (i.e., years of experience and number of days of absence by employees, personal income of subscribers to different magazines, and serving time in a business and amount of money earned), the same three surface stories were used for the chi-square problems, and the same three surface stories were used for the correlation problems. In short, the structure-emphasising example set presented the same surface stories between problem types.

Surface-emphasising set: For the three f-test problems they used the same surface story, but it was different from those used on the other sheets (e.g., years of work experience and number of days of absence by employees); the three chi-square problems used the same surface story, but it was different from those used on the other sheets (e.g., personal income of readers of different magazines); and the three correlation problems used the same surface story, but it was different from those used on the other sheets (e.g., length of employment and amount of money earned). In short, the surface-emphasising example set presented the same surface stories within each problem type.

---

The results show that it’s more effective to design structure-emphasising examples (focusing on the underlying problem structure) than surface-emphasising examples (focusing on the surface story), because they demonstrate to learners that a reliance on surface structures doesn’t work.

In addition to providing multiple examples (with the appropriate underlying rules), variability of problems affect learning as well.

Varying problems within sessions/lessons

Variability of practice, also known as interleaving, has proven to be a strong way to help learners practice. What it means is that, when you design practice activities, you need to make sure that “a sequence of exercises or problems that the learner needs to work through can’t be solved using one and the same strategy, rule, concept …” (Neelen & Kirschner, 2020, p 229).

Paas and Van Merriënboer (1994) compared four groups (all participants received general instruction first and they all had to solve six problems):

• • low variability/practice – solved six problems, all the same sub type
  • high variability/practice – solved six problems, consisting of two problem sub types
  • low variability/example – studied one worked example, then solved six problems
  • high variability/example) – studied two worked examples (each explaining one problem sub type), then solved six problems

Next, all participants completed the same test, in which they had to combine what they learned in novel ways.

First of all, training with a worked example wins from just solving the problem (the practice condition) and, to be more precise, the results suggested that variability produces transfer benefits, but only in combination with a worked example. Worked examples for the win, so much is clear!

Example-Problem pairs

We’ve already seen that good instruction for novices should include both worked examples and practice problems. Some learners rely heavily on examples during problem-solving. However, to what extent should examples and practice be explicitly paired?

For example, Trafton and Reiser (1993) found that the most efficient way for a learner to acquire a skill is to study an example first and then solve a problem (instead of studying a blocked series of examples followed by a blocked series of practice problems).

Interestingly, more recent research by Van Gog, Kester, and Paas (2011) demonstrates that worked examples don’t necessarily need to be followed by a practice problem in order to be effective but when doing so, the right sequence is worked example followed by practice problem is the right sequence (instead of practice problem followed by worked example). Although their study didn’t look at blocked versus varied, it’s interesting that the practice problems aren’t necessarily required in order for the learner to learn effectively and efficiently.

So far, we’ve looked at intra-example features (how the example is designed and the way the example’s solution is presented, discussed in the first blog of this series) and this week’s inter-example features.

Next week, we’ll look at the ways examples are actually used by the learner, in particular when learners are explaining examples to themselves or to others.

References

Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of educational research, 70(2), 181-214.

Chi, M. T., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive science, 5(2), 121-152.

Paas, F. G., & Van Merriënboer, J. J. (1994). Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of educational psychology, 86(1), 122-133.

Quilici, J. L., & Mayer, R. E. (1996). Role of examples in how students learn to categorize statistics word problems. Journal of Educational Psychology, 88(1), 144–161.

Reed, S. K., & Bolstad, C. A. (1991). Use of examples and procedures in problem solving. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17(4), 753-766.

Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and instruction, 2(1), 59-89.

Trafton, J. G., & Reiser, B. J. (1993). Studying examples and solving problems: Contributions to skill acquisition. In Proceedings of the 15th conference of the Cognitive Science Society (pp. 1017-1022).

Van Gog, T., Kester, L., & Paas, F. (2011). Effects of worked examples, example-problem, and problem-example pairs on novices’ learning. Contemporary Educational Psychology, 36(3), 212-218.

Weber, G. & Bögelsack, A. (1995). Representation of programming episodes in the ELM model. In K. F. Wender, F. Schmalhofer, & H.-D. Böcker (Eds.), Cognition and computer programming (pp. 1-26). Norwood, NJ: Ablex Publishing Corporation.

Weber, G. (1996). Individual selection of examples in an intelligent learning environment. Journal of Artificial Intelligence in Education, 7(1), 3-31.

Advertisement