Do or Do Not: The Doer Effect

Learning By Doing

Let's pretend you work for a software company, and your manager wants you to create a clickable prototype for an app you are about to launch. 

Part of your prototype includes a dropdown menu that appears when users hover over it; however, you've never mocked up a dropdown menu before. 

You have two learning paths available to you. Would you rather:

1. Watch a video of someone making a dropdown menu.
2. Find a written worked-out example and follow along with your favorite prototyping software. 

Based on your choice: 

• Which path do you think will be easier to follow in the short term? 
• Which learning path will result in longer-term learning? 
• Which learning path might generalize to other related tasks?

"Do…or do not. There is no try." —Master Yoda, The Empire Strikes Back (1980)

Since the early days of this blog, we've opened with a Learning By Doing activity. What is the point of that? Is there evidence that this is a useful thing to do? 

Before jumping to the data, the idea of "learning by doing" is not at all new. Origins of the idea can be found in quotes by Aristotle [1] and Confucius [2]. John Dewey popularized the idea in American education in his book, Democracy and education [3].

There are theoretical reasons to believe that learning by doing will result in more durable and lasting changes. For example, I remember my memories better than I remember yours. Why? Because our brains are selfish. It is advantageous to our survival to remember the things we've done, both in terms of our successes and mistakes. We can also teach ourselves new strategies for solving problems that we've encountered several times in the past [4].

There are also empirical reasons why we learn better by active engagement. For example, we know from memory research that if we have an active hand in generating items to remember, we have a better shot of remembering them later. This effect goes by the name the generation effect.

If you're in a MOOC, be a Doer!

It seems that "learning by doing" is an effective learning mechanism when we test people in the lab. What does it look like out in the wild? What does the evidence look like in the classroom? 

Fortunately, with the advent of online education, we now have a ready-made venue that offers the opportunity for naturalistic experiments. Many "massive open online courses" (or "MOOCs") offer the learner several different types of learning activities. Most MOOCs have video-based lectures, online textbook passages; some even offer active-learning resources such as computer tutors or simulations. For MOOCs that offer both, which learning activities offer the best learning outcomes? 

Dr. Ken Koedinger and his collaborators conducted a pair of studies to answer precisely this question [5, 6]. They categorized students into several groups, based on their in-class behavior. Students who primarily watched videos were categorized as "Watchers", those who read the text were called "Readers", and those who completed the interactive learning activities were categorized as "Doers". Then the researchers looked at their performance on both quizzes and the final exam. The learning outcomes were extremely consistent. No matter which outcome variable they used, students who were categorized as Doers outperformed the Readers and Watchers. 

To estimate the impact of engaging in more interactive learning resources, they computed a statistical model that looked at the impact of pretest, doing activities, watching videos, or reading text on the final exam grade. The magnitude of the impact of doing the activities was huge. Completing the learning activities was six times more impactful than just watching videos or reading the text. 

This is strong evidence that learning by doing is an effective learning mechanism in online classrooms.

The Classroom Connection

The implication for education is fairly straightforward since the evidence was taken straight from an online course. In fact, their results are completely consistent with the ICAP framework from a previous post. As you move from a passive learning experience (e.g., reading text or watching a video) to more interactive learning environment (e.g., solving a problem or drawing a diagram), then learning tends to improve.

The educational goal, obviously, is to make the "lesson come alive" by actively engaging students in their learning. Active learning can assume an unlimited number of forms. But the point is that you don't want to rely just on asking your students to watch a video. Instead, follow up with a series of questions. Or, better yet, ask them to engage in the same activities as the video (kind of like the standard "I do, we do, you do" sequence). The danger is, if videos (or text) aren't followed up with an activity, then students risk tricking themselves into thinking "they get it."

Thanks for reading. Now go do something! 

Share and Enjoy!

Dr. Bob

Going Beyond the Information Given

[1] “For the things we have to learn before we can do them, we learn by doing them.” 

― Aristotle, The Nicomachean Ethics

[2] I hear and I forget

     I see and I remember

     I do and I understand 

—Confucius

[3] Of course, merely acting does not guarantee learning. There has to a meaningful connection of action to its consequences for there to be any useful learning. Dewey, J. (1923). Democracy and education: An introduction to the philosophy of education. macmillan.

[4] Anzai, Y., & Simon, H. A. (1979). The theory of learning by doing. Psychological review, 86(2), 124.

[5] Koedinger, K. R., Kim, J., Jia, J. Z., McLaughlin, E. A., & Bier, N. L. (2015, March). Learning is not a spectator sport: Doing is better than watching for learning from a MOOC. In Proceedings of the second (2015) ACM conference on learning @ scale (pp. 111-120).

[6] Koedinger, K. R., McLaughlin, E. A., Jia, J. Z., & Bier, N. L. (2016, April). Is the doer effect a causal relationship? How can we tell and why it's important. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 388-397).