Paul A. Kirschner & Mirjam Neelen
Motivation, engagement, commitment, drive, grit … Some people seem to be obsessed with these concepts. For them, they’re like magic wands that can solve almost all of the problems in education or learning in general and/or are primary objectives for education and learning (we’ve blogged about the topic before here). It’s a mystery to us why people are so fascinated, often to the point of obsession, with motivation and engagement because…
Reason 1: Measurability
First, there’s a real problem with measurability. Motivation, engagement, and any similar affective state for which people experience positive or negative emotions or feelings, are almost never measured in a direct and objective manner. In order to give it a shot anyway, researchers use various self-reporting methods, such as surveys, Likert scales, journals / diaries, log books, and/or semantic differential scales. Unfortunately, we’ve known for a long time based on many studies that such self-reported measurements are the least reliable measurements that exist. We’d even go so far as to question whether one, with good conscience, can label these methods as ‘measurements’ at all. From many different contexts, for example nutritional science, we know that what people claim they do or how they behave rarely has anything to do with reality. For example when people report on their food intake in a journal / diary they seem to gain weight through the air that they breathe.
Here’s a quote from an article (in Dutch but with an abstract in English) by Hilberink and Jacobs on how (un)reliable self-reporting is in the context of smoking:
The literature states that risk groups in general lie about their smoking behaviour more often than smokers without a chronic condition. Participating in a ‘quit smoking intervention’ seems to increase the tendency of smokers with COPD to self-report inaccurately. In a survey before the start of the intervention, 19% claimed to be a non-smoker. After the intervention this was 52%. Lying (to just say it bluntly) about the degree of substance abuse is symptomatic of any addiction. Therefore, it seems a good idea to not only approach smokers with COPD as patients with COPD (p. 349).
To put it ‘scientifically’: There is little to no correlation between what people say they do and what they actually do (see Busato c.s., 1995; Hamaker, 1999).
Note: The external physical characteristics of engagement can be measured somewhat objectively because we can observe if someone is ‘engaged’ (as in, ‘doing something’). One problem here is that an observation by a teacher or researcher is in itself subjective and subject to a plethora of biases (e.g., we often see what we want to see; it’s only human and teachers and researchers are human) so even such direct observation can be suspicious. A second problem is that, after having determined that a learner is engaged, we still need to interpret the meaning of that engaged behaviour. For example, we don’t know, even when we observe that someone’s ‘engaged’, whether (s)he’s actually doing something useful, whether (s)he’s learning or whether (s)he’s just wasting time. In other words, even when we’re trying to measure engagement in an objective manner, it still often says next to nothing about learning.
As Arvind Verma astutely noted in his blog: “Engagement is not productivity or an output— using an analogy, engagement may be smoke but it is not fire.”
Reason 2: The (non)relation between motivation / engagement and learning
A second problem with motivation or engagement or… is in our opinion that, as educators or learning professionals, motivation or engagement is not really helpful if we don’t know if and how it contributes to learning and achievement. We definitely want learners to ‘be enjoyably busy’, but in an effective way. Here the emphasis is on effective and not on ‘being enjoyably busy’. What we don’t want is that our learners are really ‘busy’ having fun and being super engaged either without actually learning something or even to the detriment of learning. In other words, we don’t want their learning to be ineffective because if this is the case, no one benefits.
Note: In a recent New York Times piece about ‘Silicon Valley Billionaires Remaking America’s Schools’ the following note about the engaging math programme Dreambox Learning (supported by the chief exec of Netflix!):
It really can suck a kid in,” said Brenda Peiffer, a former school counselor, whose son, a third grader, was assigned DreamBox for homework. After noticing that he seemed more interested in spending points to customize his avatar than in actually doing math, she put the kibosh on DreamBox.
Also, it’s not really very useful to choose a learning approach in which learners are very engaged, where they possibly learn something along the way, but are spending a lot of – often wasted time – to get there. Studying / learning that way is not only inefficient, but it can also be very demotivating in the end. Even if learners are engaged at first and are enjoying the learning experience but not really getting anywhere, after a while they’ll probably think “Why am I doing this?” “What the heck am I doing?”, “What’s in it for me?” In other words, in the end, the experience often won’t be very satisfying (another hard to measure affective variable) and as a consequence, the initial interest to invest time and energy in what at first was such a motivating activity, will fade over time and finally disappear. The initial motivation will only be maintained if it goes hand in hand with positive results (positive reinforcement; remember that?). Without positive result, demotivation or amotivation follows, or, worst case scenario, the learner will feel helpless and stop.
As a side note here: Of course it’s nice to be able to ‘chill’ a bit as an instructor while learners are ‘engaged’ but that’s a perk and not a goal. As a teacher, you teach because you love to teach, because you’re passionate, and because your main goal is that learners actually learn in an effective, efficient, and enjoyable way (the so-called 3-star learning experience :)).
Super! The teacher chills while the learners are engaged!
Don’t get us wrong. We love having learners being motivated and engaged and strive for them to be just that. Motivation and engagement are, in themselves, excellent paths on the road to learning. However, motivation and engagement neither can replace learning nor be a proxy for learning. Carl Hendrick writes
I’ve long thought that one of the weakest proxy indicators of effective learning is engagement, and yet it’s a term persistently used by school leaders (and some researchers) as one of the most important measures of quality. In fact many of the things we’ve traditionally associated with effective teachers may not be indicative of students actually learning anything at all.
Professor Rob Coe, professor in the School of Education and Director of the Centre for Evaluation and Monitoring (CEM) at Durham University created a nice list of these meaningless proxies for learning:
Prof. Rob Coe, From Evidence to Great Teaching, Presentation at ASCL 20, March 2015
Ok, we’re slightly side tracking. Why this blog? What triggered it? It was the graph below:
This graph (the Dutch text says ‘Through using Blackboard with assignments, tests, and instructional videos I am motivated to learn more than through just a book’ with, as is traditionally the case, the answer options in a Likert scale from completely disagree to completely agree) was published in a blog about how ICT motivates students
Hocus, pocus (the rabbit’s coming out of the hat), the vast majority of students agree or completely agree that the BB, offering assignments, assessments, and videos is more motivating than just a book.
Fantastic, but now, the elephant in the room: Did they learn more or even the same? Dead silence around this question, except for the next figure in the blog (see below), namely that the students felt or thought that they were more both engaged and learned more from the tests in BB.
It’s beside the point that the question was a poor one as it asked two different things at the same time (!), the fact is that learning just wasn’t measured. What was measured is the students’ perception of whether they had learned (or their engagement or both) and we know that such self-report is not reliable and also that student judgements of learning are notoriously wrong. Great that you’re asking the learners’ opinion but what’s the point? And what’s the point if they’re indeed more motivated?
If we’d ask those same students on a 5-point Likert scale to respond to the statement “I enjoy eating potato chips/crisps, cola and cake more than fruits and vegetables”, almost all of them would probably completely agree (with an exception for the ‘foodies’). However, an answer like that doesn’t tell us if they would be healthier after eating these appealing fatty, salty, and sugary foods. Of course in this case we can guess that they would actually be less healthy over time (i.e., obestity, diabetes, high blood pressure, heart problems…).
Dear fellow researchers, teachers, trainers, instructional designers, and all other learning professionals: Can we please agree that motivation, engagement, fun, and many other positive emotions during learning are great to strive for but let’s first go for learning. Without learning, what’s the point of learners being motivated and engaged‽
Busato, V. V., Prins, F. J., Hamaker, C., & Visser, K. (1995). Leerstijlenonderzoek gerepliceerd: De samenhang tussen leerstijlen en intelligentie [Learning style research replicated: The consistency between learning styles and intelligence]. Tijdschrift voor Onderwijsresearch, 20, 332-340.
Hamaker, C. (1999). Leerstijlen: categorieën of dimensies? [Learning styles: Categories or dimensions] In R. Hamel, M. Elshout-Mohr, & M. Milikowski (Eds.), Meesterschap, Zestien stukken over intelligentie, leren, denken en probleemoplossen. (pp.49-60) Amsterdam: Vossiuspers AUP.
Hendrick, C., (2015, March 22). Engagement: Just because they’re busy, doesn’t mean they’re learning anything [Blog post]. Retrieved from https://chronotopeblog.com/2015/03/22/engagement-just-because-theyre-busy-doesnt-mean-theyre-learning-anything/
Hilberink, S. R & Jacobs, J. E. (2014). How reliable is self-reported smoking in patients with COPD? Huisarts & Wetenschap, 57(7), 348-350.
Singer, N., (2017). The Silicon Valley billionaires remaking America’s schools. The New York Times. Retrieved from https://www.nytimes.com/2017/06/06/technology/tech-billionaires-education-zuckerberg-facebook-hastings.html?_r=2
Van Hove, E., (2017, April). Onderwijs met ICT motiveert [Education with ICT motivates] [Blog post]. Retrieved from https://storify.com/eusvanhove/blended-learning
Verma, A., (2015, May 6). The top 20 potential problems with employee engagement [Blog post]. Retrieved from https://www.linkedin.com/pulse/top-20-potential-problems-employee-engagement-arvind-verma
 Currently, there’s some ongoing research on certain physiological responses (e.g., heart rate, electrodermal activity (EDA), micro facial expressions or micro gestures, speech characteristics, etc) to be able to measure affective states in an objective manner. Paul is part of a project (SLAM) on this topic in Finland.