Mirjam Neelen & Paul A. Kirschner – Original work by Wilfred Rubens
VR and gamification will transform training and education in 2021, according to Mike Todd in a recent article in FE News. Todd is CEO at Near-Life, offering learning technology to create ‘immersive learning’. Those who know us wouldn’t be surprised that we’re immediately suspicious when we read an article like this.
After all, it’s not likely that a CEO selling VR for learning would say that VR for learning will likely fail because X, Y, or Z.
Luckily and coincidentally, Wilfred Rubens (@wrubens on Twitter, you can find his Dutch blog here) came to the rescue, shining some light on the actual state of the art in VR for learning. On the exact same day as the publication of the FE News article, he posted a blog on ‘virtual reality to develop employee soft skills’. Mirjam asked Wilfred if there are any VR studies that look at actual learning results, instead of just ‘affective’ measures, such as ‘learners enjoy VR training more than face-to-face or traditional eLearning’ or ‘Learners say VR leads to better learning results than face-to-face or traditional eLearning’.
The next day, in response to her question, Wilfred published another blog in which he discussed current research around the question as to whether VR leads to better learning. With thanks to Hans Luyckx (@luyckxh), who pointed him to the research.
In this blog, we summarise Rubens’ initial blog on VR for soft skills and then provide the full second blog on ‘Does VR lead to better learning?’ (and, just to be clear, Wilfred is OK with this J).
Virtual Reality to Develop Employee Soft Skills – A Summary
An article by Jeanne C. Meister for Harvard Business Review, titled ‘How Companies Are Using VR to Develop Employees’ Soft Skills’ triggered Rubens to write his blog. In summary [with our comments between square brackets]:
- Organisations invest a lot of money in so-called ‘soft skills’, such as collaboration and solving conflicts. VR could be a possible solution, for example when face-to-face trainings aren’t possible, such as right now in COVID times. Potentially, VR training is more effective and efficient as well.
- There are plenty ‘training’ offerings online to support the development of soft skills. Rubens adds the quotes for ‘training’ because the offerings mostly ‘explain’ how you can improve your collaboration, but usually they don’t offer any opportunity to practice and receive feedback [Mirjam/Paul: Thus, it’s actually ‘online information’ and not training].
- VR offers an opportunity to practice actual interactions with an avatar through role-play. With VR, learners can practice soft skills in a safe space, without the workplace running any potential risks [Mirjam/Paul: For example, employees violating certain behavioral policies]. Learners can also practice regularly and spread that practice out over time.
- VR still isn’t cheap. Meister discusses a ‘study’ by PwC [We added the quotes because the study wasn’t very well designed, which was analysed in a blog here] that shows that VR could be cheaper than traditional face-to-face soft skills training (e.g., PwC argues that learners completed a training 4 times faster than an in person and 1.5 times faster than an eLearning programme). Rubens concludes, rightly so, that we need more research to make ‘hard claims’ around training efficiency and cost savings.
- Meister also interviewed 300 L&D leaders. She discusses three cases to demonstrate how organisations use VR to offer trainings in customer service, presentation skills, and employees’ performance evaluations. Only one case shows actual impact (less unsatisfied customers and quicker handling of customer questions), instead of just the learners saying how much they enjoyed the training.
Example of leadership training in VR by Mursion
Rubens concludes that VR to develop soft skills could indeed be considered if we can make a reasonable case that VR training leads to soft skill improvement and that the high development costs of VR outweigh cost reductions, such as, for example, shorter trainings or faster skills development.
This brings us to the question as to what research we have to demonstrate that VR leads to better learning and/or performance outcomes? We’re moving on to Rubens’ second blog, answering this exact question.
Does VR lead to better learning outcomes?
A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda (Radianti et al, 2020) is a very comprehensive study in which the researchers ultimately studied 83 articles on VR in higher education. Among some other topics, they focused on the question of which learning theories have been applied to the design and development of VR applications for higher education. They also looked at how learning outcomes have been evaluated. Some of the findings:
- Few papers have examined learning outcomes after the application of VR in a specific domain. Most evaluations were actually ‘usability’ tests. Almost half of the articles didn’t specify an evaluation method for the learning outcomes. Some used questionnaires (22%) or user activities while logged into the VR application (12%). Other ways of evaluating the application, such as observations or exams, were even less common. The authors refer to two articles that did evaluate how much knowledge or skills learners gained after using VR. These articles are not freely accessible. The review doesn’t make any statements about the articles’ conclusions.
- The authors advise that we need better evaluation methods to determine the impact of VR.
- Around 70% of the reviewed articles don’t use explicit learning theories as a theoretical foundation. Other articles discussed learning theories, but didn’t link them to the use of VR. The authors only evaluated the app’s features or usability, but not the learning outcomes. A third group of articles highlighted the underlying theories of educational VR design, but didn’t report in detail on the technical development.
- VR appears to be a promising field. This systematic review identifies 18 application possibilities, which, according to the researchers, indicates that many disciplines see the potential of this technology.
The Applications of Virtual Reality Technology in Medical Groups Teaching is from 2018, and is a review of 21 studies on the use of VR in healthcare. The researchers looked at the advantages and disadvantages of using VR. The benefits mainly consist of better learning outcomes. The researchers conclude that:
- the use of VR has led to better learning outcomes in 17 (74%) studies. Twenty studies (87%) reported that medical professionals trained via VR exhibited higher accuracy in their practice.
- the application of virtual reality plays an important role in improving the performance of different medical groups.
Note: About a year ago I [Wilfred Rubens] wrote a blog about a Danish study by Maransky showing that the effectiveness of VR for learning depends on the virtual pedagogical agent (the ‘virtual teacher’).
[Added by Mirjam & Paul]: Eurekalert summarises Makransky’s research results as follows:
In a study with 66 7th and 8th-grade students (half boys, half girls) at a Danish science talent school, Makransky and colleagues found that the girls learned most in the VR-simulations, when the VR-teacher there was a young, female researcher named Marie, whereas the boys learned more, while being instructed by a flying robot in the form of a drone.
Virtual reality is developing rapidly. This technology hasn’t been used much for learning, training and education yet.
Questions such as within which fields, for which learning questions, and for which knowledge and skills does VR work effectively are critical, as well as under which conditions and conditions VR contributes to better learning outcomes?
At this point, these questions can’t be answered unequivocally. To do this, we’ll have to apply VR more often, conducting research in which we look beyond satisfaction or ‘usability’.
 Farra S. L. , Smith S. J., & Ulrich D. L. (2018). The student experience with varying immersion levels of virtual reality simulation. Nursing Education Perspectives, 39(2), 99-101. DOI:10.1097/01.NEP.0000000000000258
Zhang K., Suo J., Chen J., Liu X., & Gao L. (2017). Design and implementation of fire safety education system on campus based on virtual reality technology. Federated conference on computer science and information systems, IEEE, p 1297-1300. DOI:10.15439/2017F376
 On 16 January, Ben Rooney (@benjrooney) tweets: “I edited the Telegraph’s tech section in the late 90s. AR was going to be massive back then. Yet here we are…” – just so that we can put things in perspective a bit when it comes promise.
 Samadbeik, M., Yaaghobi, D., Bastani, P., Abhari, S., Rezaee, R., & Garavand, A. (2018). The applications of virtual reality technology in medical groups teaching. Journal of Advances in Medical Education & Professionalism, 6(3), p 123-129.
9 thoughts on “Does Virtual Reality Lead to Better Learning?”
Thanks for this article. We apply VR for participating in traffic and also for drug effect studies. Several PhD researchers have shown that behavior in VR corresponds to behavior in traffic. Also when a distinction is made between mistakes and violations. The male female differences also appear to occur in the traffic simulation and errors and violations occur in a weakened form even 3.5 years after the VR training. This makes VR interesting for assessment. You don’t learn anything from VR in itself. The learning effect (transfer and retention) is determined by the intelligence you add to it. This is separate from VR. We use an adaptive system for instruction in which automation, the reduction of errors, is central. By comparing error decreases, we show what the relative learning speed is on procedures. By following students after obtaining the driving license, we have been able to demonstrate a series of correlations including the replacement value of normal training, which is almost 2: 1 and the associated higher success rate. We have also proven that due to the learning method, the accident involvement within the group of simulator students has decreased significantly in the first 12 months after obtaining the driving license. We have now also released the driving simulator lessons from Corona online and are investigating the replacement value of this form of VR.
The question if VR will enhance learning sounds quite general to me. We could also ask ourselves if classrooms, blackboards, pencils, and paper benefit learning. Of course, it can but only when used in the right way. Just like any other tool that is available. VR is a form of simulation and simulations can be a great way to learn new things. Without simulations, Schiphol for instance would be a very dangerous place. Thru simulations, we can learn in situations that otherwise wouldn’t be possible or very dangerous for the learner and or the environment.
The VR/AR/XR-technology can make simulations look like real-world experiences. When we start to use VR because it is VR then we are organizing ‘gadget-education’ as I like to call it. By using gadgets we can make it look like we understand the future and develop future-proof education. As you know organizing future-proof and valuable education is another level.
Ask airline pilots, soldiers, firefighters, and formula 1 drivers if VR-simulations are tributing to their learning process and there will be no doubt. If you narrow it down to learning soft-skills then it comes down to the purpose and quality of the VR-product.
Don’t shoot us, we’re just the piano players :-). Maybe you can contact Wilfred on this.
Thanks for the article and bringing the blogs to a wider audience.
I suppose one thing you can say in favour of VR-based learning/training is that real evidence is pretty thin and therefore it is hard to categorically rule anything out!
Once the marketing puff pieces – of which there are many! – are filtered out there do seem to be emerging some interesting studies of the actual learning effect, but too often comparing to traditional classroom or video-based methods rather than directly comparing to desktop computing simulations or other interaction schemes. An opportunity missed in my opinion!
I’m excited about the potential of VR to let people explore concepts in an immersive way. So for example, an immersive experience to explore and discover a cell structure. To establish if the VR aspect of such a learning experience is what makes it effective, we’d need to compare it to a non-VR learning experience that was of demonstrable equivalent quality. And then have a valid assessment of the material immediately after the learning session, as well as at intervals after. Only then would we be able to answer questions about the effectiveness of VR as a learning tool. This would be enormously expensive to do.
Plus, VR is obviously not a silver bullet. It will only form part of a learning solution. How does it best add value? With what other instrucitonal/learning approaches can it effectively be combined? These are questions that also need to be explored.
In my bachelor’s thesis for vocational teachers, I and my fellow students used VR glasses, to let the students change brake pads on a car. Afterwards, I taught the students several topics based on the work they did in VR. . With the simple study we did, we assessed the students to level 4-6 on Thalheimer’s evaluation model (LTEM). We also got students who welcomed us back with teaching in VR. As several comments point out, pedagogy is important in teaching planning.
Thank you for the share!