Does Virtual Reality Lead to Better Learning?

Mirjam Neelen & Paul A. Kirschner – Original work by Wilfred Rubens

VR and gamification will transform training and education in 2021, according to Mike Todd in a recent article in FE News. Todd is CEO at Near-Life, offering learning technology to create ‘immersive learning’. Those who know us wouldn’t be surprised that we’re immediately suspicious when we read an article like this.

After all, it’s not likely that a CEO selling VR for learning would say that VR for learning will likely fail because X, Y, or Z.

Luckily and coincidentally, Wilfred Rubens (@wrubens on Twitter, you can find his Dutch blog here) came to the rescue, shining some light on the actual state of the art in VR for learning. On the exact same day as the publication of the FE News article, he posted a blog on ‘virtual reality to develop employee soft skills’. Mirjam asked Wilfred if there are any VR studies that look at actual learning results, instead of just ‘affective’ measures, such as ‘learners enjoy VR training more than face-to-face or traditional eLearning’ or ‘Learners say VR leads to better learning results than face-to-face or traditional eLearning’.

The next day, in response to her question, Wilfred published another blog in which he discussed current research around the question as to whether VR leads to better learning. With thanks to Hans Luyckx (@luyckxh), who pointed him to the research.

In this blog, we summarise Rubens’ initial blog on VR for soft skills and then provide the full second blog on ‘Does VR lead to better learning?’ (and, just to be clear, Wilfred is OK with this J).

Virtual Reality to Develop Employee Soft Skills – A Summary

An article by Jeanne C. Meister for Harvard Business Review, titled ‘How Companies Are Using VR to Develop Employees’ Soft Skills’ triggered Rubens to write his blog. In summary [with our comments between square brackets]:

• Organisations invest a lot of money in so-called ‘soft skills’, such as collaboration and solving conflicts. VR could be a possible solution, for example when face-to-face trainings aren’t possible, such as right now in COVID times. Potentially, VR training is more effective and efficient as well.
• There are plenty ‘training’ offerings online to support the development of soft skills. Rubens adds the quotes for ‘training’ because the offerings mostly ‘explain’ how you can improve your collaboration, but usually they don’t offer any opportunity to practice and receive feedback [Mirjam/Paul: Thus, it’s actually ‘online information’ and not training].
• VR offers an opportunity to practice actual interactions with an avatar through role-play. With VR, learners can practice soft skills in a safe space, without the workplace running any potential risks [Mirjam/Paul: For example, employees violating certain behavioral policies]. Learners can also practice regularly and spread that practice out over time.
• VR still isn’t cheap. Meister discusses a ‘study’ by PwC [We added the quotes because the study wasn’t very well designed, which was analysed in a blog here] that shows that VR could be cheaper than traditional face-to-face soft skills training (e.g., PwC argues that learners completed a training 4 times faster than an in person and 1.5 times faster than an eLearning programme). Rubens concludes, rightly so, that we need more research to make ‘hard claims’ around training efficiency and cost savings.
• Meister also interviewed 300 L&D leaders. She discusses three cases to demonstrate how organisations use VR to offer trainings in customer service, presentation skills, and employees’ performance evaluations. Only one case shows actual impact (less unsatisfied customers and quicker handling of customer questions), instead of just the learners saying how much they enjoyed the training.

Example of leadership training in VR by Mursion

Rubens concludes that VR to develop soft skills could indeed be considered if we can make a reasonable case that VR training leads to soft skill improvement and that the high development costs of VR outweigh cost reductions, such as, for example, shorter trainings or faster skills development.

This brings us to the question as to what research we have to demonstrate that VR leads to better learning and/or performance outcomes? We’re moving on to Rubens’ second blog, answering this exact question.

Does VR lead to better learning outcomes?

A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda (Radianti et al, 2020) is a very comprehensive study in which the researchers ultimately studied 83 articles on VR in higher education. Among some other topics, they focused on the question of which learning theories have been applied to the design and development of VR applications for higher education. They also looked at how learning outcomes have been evaluated. Some of the findings:

• Few papers have examined learning outcomes after the application of VR in a specific domain. Most evaluations were actually ‘usability’ tests. Almost half of the articles didn’t specify an evaluation method for the learning outcomes. Some used questionnaires (22%) or user activities while logged into the VR application (12%). Other ways of evaluating the application, such as observations or exams, were even less common. The authors refer to two articles[1] that did evaluate how much knowledge or skills learners gained after using VR. These articles are not freely accessible. The review doesn’t make any statements about the articles’ conclusions.
• The authors advise that we need better evaluation methods to determine the impact of VR.
• Around 70% of the reviewed articles don’t use explicit learning theories as a theoretical foundation. Other articles discussed learning theories, but didn’t link them to the use of VR. The authors only evaluated the app’s features or usability, but not the learning outcomes. A third group of articles highlighted the underlying theories of educational VR design, but didn’t report in detail on the technical development.
• VR appears to be a promising field[2]. This systematic review identifies 18 application possibilities, which, according to the researchers, indicates that many disciplines see the potential of this technology.

The Applications of Virtual Reality Technology in Medical Groups Teaching[3] is from 2018, and is a review of 21 studies on the use of VR in healthcare. The researchers looked at the advantages and disadvantages of using VR. The benefits mainly consist of better learning outcomes. The researchers conclude that:

• the use of VR has led to better learning outcomes in 17 (74%) studies. Twenty studies (87%) reported that medical professionals trained via VR exhibited higher accuracy in their practice.
• the application of virtual reality plays an important role in improving the performance of different medical groups.

Note: About a year ago I [Wilfred Rubens] wrote a blog about a Danish study by Maransky showing that the effectiveness of VR for learning depends on the virtual pedagogical agent (the ‘virtual teacher’).

[Added by Mirjam & Paul]: Eurekalert summarises Makransky’s research results as follows:

In a study with 66 7^th and 8^th-grade students (half boys, half girls) at a Danish science talent school, Makransky and colleagues found that the girls learned most in the VR-simulations, when the VR-teacher there was a young, female researcher named Marie, whereas the boys learned more, while being instructed by a flying robot in the form of a drone.

Virtual reality is developing rapidly. This technology hasn’t been used much for learning, training and education yet.

Questions such as within which fields, for which learning questions, and for which knowledge and skills does VR work effectively are critical, as well as under which conditions and conditions VR contributes to better learning outcomes?

At this point, these questions can’t be answered unequivocally. To do this, we’ll have to apply VR more often, conducting research in which we look beyond satisfaction or ‘usability’.

---

[1] Farra S. L. , Smith S. J., & Ulrich D. L. (2018). The student experience with varying immersion levels of virtual reality simulation. Nursing Education Perspectives, 39(2), 99-101. DOI:10.1097/01.NEP.0000000000000258

Zhang K., Suo J., Chen J., Liu X., & Gao L. (2017). Design and implementation of fire safety education system on campus based on virtual reality technology. Federated conference on computer science and information systems, IEEE, p 1297-1300. DOI:10.15439/2017F376

[2] On 16 January, Ben Rooney (@benjrooney) tweets: “I edited the Telegraph’s tech section in the late 90s. AR was going to be massive back then. Yet here we are…” – just so that we can put things in perspective a bit when it comes promise.

[3] Samadbeik, M., Yaaghobi, D., Bastani, P., Abhari, S., Rezaee, R., & Garavand, A. (2018). The applications of virtual reality technology in medical groups teaching. Journal of Advances in Medical Education & Professionalism, 6(3), p 123-129.

---