And the winner is… testing!

Paul A. Kirschner & Mirjam Neelen

When you ask learners how they study or how they prepare for a test, it’s likely that they’ll tell you that they reread and/or highlight and/or underline important paragraphs or sentences in their textbooks, workbooks, prints, and so forth. Another thing they might tell you is that they study/restudy their class notes if they made any (Karpicke, Butler, & Roediger, 2009). The problems with all of these so-called learning strategies is that they’re not really effective. In fact, most of the time they’re not effective AT ALL! As a consequence, learners waste their time and energy (e.g., they don’t learn that much or their grade on the exam is somewhat disappointing).

We’ve written about effective and ineffective learning strategies before, but thought it wouldn’t hurt to emphasise one of most effective learning strategies we know of in a separate blog (in fact, our doing this could be viewed as a form of spaced learning, which is an incredibly effective learning strategy!). And the winner is… testing!

Huh? Don’t you test either to check if something has been learned/can be applied or how the learning process is progressing? Yes, you do; the former is called summative and the latter formative testing or assessment. However, testing is also one of best and most robust ways to learn something. This type of testing is known as retrieval practice. Retrieval practice basically means ‘practice through remembering’ (i.e., retrieving something from your long-term memory) and what’s more, it can be designed and implemented either by the learner or the teacher.

Testing: The What

When using retrieval practice, learners are ‘forced’ to retrieve information that they’ve previously learned from their memory. By actively remembering this information (by retrieving it), it will be remembered better and longer. This is called the testing effect.

Recently, Adesope, Trevisan, and Sundarajan (2017) concluded in their meta-analysis that retrieval practice works way better than repeatedly rereading what needs to be learned (the most often used learning strategy: RRR – read, reread, reread again) but even better, they also found that it works better than ANY OTHER learning strategy they compared it with, such as restudying (e.g., reviewing notes and homework) and filler activities such as resting, solving puzzles, and so forth. The average effect size was 0.61[1], which is considered high to very high.

This effect was found for all types of retrieval practice; cued-recall (e.g., teachers ask learners questions to encourage them to recall stuff), free-recall (learners create their own key words or questions to help them remember), recognition tasks, multiple choice questions, and so forth, as well as throughout all ages, education types (primary, secondary, higher ed), school subjects, and learner characteristics. There are simply no other strategies that work as well as retrieval practice for learning. Their conclusion:

An overwhelming amount of evidence reviewed in this meta-analysis suggests that retrieval practice increases achievement. The benefits of retrieval practice persist across a wide array of educational levels, settings, and testing formats and procedures. Therefore, students should be encouraged and taught how to use retrieval practice during self-directed learning activities, and teachers may incorporate retrieval practice into structured classroom activities. (p. 690)

Christine Bae and her colleagues (2018) studied various types of self-testing and practice tests. They found that testing repeatedly works better than just one time testing and that free-recall and small practice test (e.g., short quizzes) were most effective. They also found that combining various test formats (e.g., self-generation of tests by learners plus practice tests and/or free-recall by the teacher) worked best.

In other words, what we’re dealing with here is an approach that has proven to work and is also easy to apply. Of course the question is: how?

Testing: The How

Making use of retrieval practice (testing) as a study strategy doesn’t have to be hard. Teachers can use each technique that encourage learners to recall information. It doesn’t matter if it’s done through short quizzes, taking practice tests, asking repetition questions, etc. You can also call this ‘low-stakes testing’, which refers to the fact that there are no or minimal consequences based on the test results.

Learners can also be encouraged to use various forms of self-testing. They can learn this by …ta-daaaah… doing it! Think flashcards, summarising texts (Alert!!! Research done by Paul and his colleagues at the Open Universiteit shows that even high school students make really poor summaries. Learners first have to learn how to make good effective summaries, otherwise it doesn’t work!), designing their own tests and quizzes or perhaps making a concept map of what they need to recall. Remember: Feedback, either from the teacher or having the learner check her/his own answers is imperative!

There is a special technique called Cornell notes, that is worth a mention. With this technique, the learner first divides a piece of paper into two columns: A recall column (usually on the left hand side) and a note taking column (usually on the right hand side and usually twice the size of the recall column). When (s)he reads a piece of text or listens to a teacher (or someone else explaining something) in class, (s)he jots down key words or questions in the left column and takes notes around the main ideas in the right. The trick is for the learner to avoid long sentences and to use symbols or abbreviations instead. When done reading or listening, the learner then writes a summary at the bottom of the page. At a later stage, the learner can cover the notes column on the right and recall them by using the key words and questions from the left column (see the example below).

Finally, Blake Harvard (The Effortful Educator) recently blogged about a novel approach namely the Swiss Cheese Summary Method.

Testing: The When

The timing of the testing is extremely important. It’s more likely that learners will remember the information if it’s tested the evening of or the day after it is learned rather than a week after learning. This is related to what is known as the Forgetting Curve (Hermann Ebbinghaus) which shows that information is lost over time when there is no attempt to retain it and that this information is most rapidly lost within the first few days after learning. In the figure below you see the original curve (‘First learned’) which represents the loss of information after originally learning it. If you test or review after one day (the first ‘Reviewed’ curve), then the learning is restored to its original level and the curve of loss is less steep, etc. Here we see the combination of retrieval practice (testing effect) and spaced practice.

Here are some rules of thumb using testing as a learning strategy:

• Production (come up with the answer yourself works better than recognising answers)
• Multiple tests work better than just one test
• A mix of formats seems to work better than using only one format
• Spacing the tests over time (e.g., every other day or every couple of days) is more fruitful then ‘cramming’ a couple of tests in one go; this is known as spaced practice, another strategy that has been proven to work (see our blog on Tips and Tricks for Spaced Learning)
• Testing during and after a learning ‘session’ are both effective
• Start with testing facts and concepts before moving on to application
• Feedback is critical (duh!) and good feedback even more so (we’ve previously blogged about effective (peer) feedback here).
• It’s unclear if it’s better to use self-generated tests or teacher generated ones
• Testing shortly after learning (that evening, the next day) is better than delayed (the next week)

In case you’d like to read more about this (of course you do!), you can go to:

Agarwal, P. K., Roediger, H. L., McDaniel, M., & McDermott, K. B. (2013). How to use retrieval practice to improve learning. Institute of Education Science. http://www.retrievalpractice.org

Roediger, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten benefits of testing and their applications to educational practice. In J. Mestre & B. Ross (Eds.), Psychology of learning and motivation: Cognition in education, (pp. 1-36). Oxford, UK: Elsevier.

For tips and tricks on how to use this strategy:

https://www.retrievalpractice.org/strategies/

http://www.learningscientists.org/blog/2016/6/23-1

http://www.learningscientists.org/s/Dutch-Six-Strategies-for-Effective-Learning-posters-2t76.pdf

References

Adesope, O. O., Trevisan, D. A. & Sundararajan, N. (2017). Rethinking the use of tests: A meta-analysis of practice testing. Review of Educational Research. 87, 659–701. doi:10.3102/0034654316689306

Agarwal, P. K., Roediger, H. L., McDaniel, M., & McDermott, K. B. (2013). How to use retrieval practice to improve learning. Institute of Education Science. http://www.retrievalpractice.org

Bae, C. L., Therriault, D. J., & Redifer, J. L. (2018, online first). Investigating the testing effect: Retrieval practice as a characteristic of effective study strategies. Learning and Instruction. doi:10.1016/jlearninstruc.2017.12.008

Karpicke, J. D., Butler, A. C., & Roediger, H. L. (2009). Metacognitive strategies in student learning: Do students practise retrieval when they study on their own? Memory, 17, 471–479. doi:10.1080/09658210802647009

Roediger, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten benefits of testing and their applications to educational practice. In J. Mestre & B. Ross (Eds.), Psychology of learning and motivation: Cognition in education, (pp. 1-36). Oxford, UK: Elsevier.

 

 

[1] In statistics, an effect size is a quantitative measure of the strength of an effect of a phenomenon or intervention on a population (test group) when compared to another population that weren’t part of the phenomenon or intervention (control group).