Clueless: The Illusion of Explanatory Depth

Learning By Doing

You've seen a bicycle before, right? Of course you have! You probably learned how to ride when you were a kid, although maybe it's been a while since you've last ridden. I'm guessing you're probably not a bike mechanic, but you are familiar with the general shape of a bike and roughly how it works.

To kick off this post, I would like you to do two things. First, I would like you to rate your knowledge and familiarity of bicycles on a scale from 1 ("I know nothing about bikes or how they work.") to 7 ("I have a complete understanding of how a bike works."). The second task requires a pen and some paper. Below is a partial sketch of a bike; however, you will notice that it's missing a couple of parts (Fig. 1). I would like you to finish my drawing. Specifically, I would like you to add the pedals, chain, and the missing pieces of the frame [1]. 

Ready? Let's get started.

Figure 1. Complete the drawing of a bike by adding the missing pieces of the 
frame, the pedals, and show where the chain goes (used with permission).

The answer to this task is probably parked in a bike rack not far from where you're sitting. But if you need to see an image of a basic bike, with no gears or brakes, here is a great example. Now that you've seen the answer, how did you do? Did you make any mistakes?

"I thought I knew more than I did."

Most people, myself included, walk around thinking that we have a pretty good understanding of the way the world works. But every once in a while, we are confronted with the uncomfortable realization that we don't know as much as we think we do. If I asked you to re-rate your knowledge about how a bike works on the same 7-point scale, would it go up, down, or stay the same? If I had to guess, I would say it probably went down. This task is likely harder than you thought it would be [2].

The reason it was so hard is because of the illusion of explanatory depth, which is the belief that you understand something better than you actually do [3]. The illusion doesn't usually happen with facts or procedures. In other words, we're pretty good at estimating when we don't know a piece of trivia (e.g., "When did Amelia Earhart become the first woman to fly across the Atlantic ocean?" ) or how to do something (e.g., "Take the first derivative of f(x) = 3x² + 4x - 5"). But with semi-complex mechanical objects (e.g., a lock or a crossbow), people are often overly confident when it comes to explaining how things work. 

How Does This happen? 

You might be asking yourself: How does the illusion of explanatory depth happen? 

There are several potential sources of the illusion of explanatory depth, but here are two. First, the illusion might arise from a confusion between familiarity and understanding. Since we have all seen many bikes in our daily lives, we come to think that we understand them. When we learn how to ride, we might also think that we understand how a bike works because we have experience interacting with them. 

Second, the illusion might be caused by the ease by which we can mentally simulate the mechanical device under question. For example, if you say, "Imagine a bike." I can do so easily. The detail of my mental image, however, is fairly sparse. The demands of the task don't require me to do anything more than envision something with wheels and a frame. Thus, my performance on the task seems adequate for the current purposes. Only when we raise the stakes do I stumble and discover my lack of understanding.

The S.T.E.M. Connection

The illusion of explanatory depth is a problem for education, partly because it seems inevitable. When you are learning something new, a necessary first step is to become familiar with the terminology and the concepts. You can't learn about the anatomical structure of a frog without first becoming familiar with the names of the organs. So what can be done about the illusion?

The most basic antidote for moving beyond a superficial understanding is to try and answer the question: Why? or How? Once you are tasked with explaining how something works, only then will you discover the gaps in your knowledge. It is both illuminating and humbling. Here are several examples that I've recently run into:

• When are you able to see a new moon? 
• Since the tension on a crossbow is strong, how do you draw back the cord?
• How do tumblers in a lock prevent the cylinder from turning? 
• How does a zipper work?
• Why does the toilet keep running?

The illusion of explanatory depth is strong, and it probably serves a purpose. We can't carry around knowledge of every mechanical device around in our heads. Otherwise, we would never have enough time to do anything else! But we also don't want to dupe ourselves into thinking we understand something when we clearly don't [4]. So the best treatment is to keep asking yourself: Can I explain what the parts are, where they go, and how they interact? In other words, keep asking yourself if you understand the way things work.


Share and Enjoy!

Dr. Bob

Going Beyond the Information Given

[1] The drawing task was taken from Lawson, R. (2006). The science of cycology: Failures to understand how everyday objects work. Memory & Cognition, 34(8), 1667-1675. I am grateful to Rebecca Lawson for allowing me to recreate Figure 1.

[2] Another good example of a deceptively difficult task – and one that we talked about in a previous post – is trying to remember what a penny looks like. We've seen hundreds, maybe thousands, of pennies. But it is surprisingly difficult to identify the real penny. Try it for yourself. 

[3] Rozenblit, L., & Keil, F. (2002). The misunderstood limits of folk science: An illusion of explanatory depth. Cognitive science, 26(5), 521-562.

[4] Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121.

Rolling in the Deep: Explanatory Depth

Think about the following questions:

1. On average, how many pounds of coffee does Columbia export to the United States in a year? 
2. What is the formal procedure for filing a patent with the U.S. Patent Office?
3. How does a sewing machine work?

Instead of producing an answer for each question, rate each question on the following dimensions:

• How likely am I to know the answer?
• How confident am I that I know the answer? 
• How well could I explain the answer to a friend?

Getting Meta-cognitive About Question Types

In a previous post, we talked about two types of meta-cognition: the what (what we know) and the how (how we know what we know). The above questions all target the what, but can be further categorized into 3 different types of questions.

Declarative Questions: The first question is a simple, declarative fact that can be answered by going to a search engine and entering a query that will return a sensible answer. For declarative questions, figuring out if you know the answer or not is a relatively straightforward process. You look into long-term memory and search for an answer (or a close approximation). If you don't get any hits, then you know you don't know the answer.

Procedural Questions: The second question is procedural, in that you must know the steps of the process necessary to achieve an end goal. To learn how to file a patent, you could also interrogate your favorite search engine, ask an expert (e.g., a patent lawyer), or learn the process yourself by reading through the steps on the U.S. Patent website. Again, knowing if you know how to do this might be as simple as asking yourself, "Have I ever filed a patent?" If the answer is "no," then you know you don't know how to do it. 

Mental-model Questions: The third class of question has a definite declarative component. The facts, however, are not stated in isolation. Instead, they are linked together in some coherent way. For simplicity, I will call this type of question a "mental-model question" because it requires knowledge about causal connections between related facts. When asked if you know how a sewing machine works [1], what probably happens is that you think about a sewing machine and what it does. Then you think about the various components that are in a sewing machine and how they interconnect. Finally, you run a mental simulation and see how the various components interact. 

The interesting feature about mental-model questions is that you typically don't get a binary "yes/no" when you ask yourself if you know the answer. For example, when you ask yourself if you know how a sewing machine works, the answer is probably not "yes" or "no," but somewhere in between. The degree to which someone understands something is what I am calling explanatory depth. If a person has deep knowledge of a topic, then they are able to provide lots and lots of details. If they only have a cursory understanding, then they will give you a much shorter and more incomplete explanation.

The Deep End of the Spool

How might we cultivate the skill of assessing our own depth of understanding of a topic? One way to evaluate the depth of our knowledge is to try to provide an explanation and see how far you can get. For instance, if someone asks me, "Do you know how a sewing machine works?" I might be tempted to say "yes." To test whether my meta-cognitive awareness about my knowledge of the inner workings of a sewing machine is well-calibrated, however, I could attempt to provide an explanation. My explanation would go something like this: 

There are two primary parts of the upper portion of the sewing machine: the needle and a spool of thread. The thread is fed through a hole in the bottom of the needle, which moves up and down, piercing the fabric. Underneath the fabric is a hole that the needle descends into, where it meets another length of thread coming from something called a bobbin. Through some bit of magic, the thread from the needle combines with the thread from the bobbin to form a stitch. 

After actually trying to explain how a sewing machine works, I realize now that my meta-cognitive awareness of sewing machine mechanics was not well-calibrated. As you can plainly see, I do a fair bit of hand waving when it comes to the process by which the thread from the bobbin intertwines with the thread on the needle. Now that I am better calibrated, meta-cognitively speaking, I would have to say that I do not understand very well how a sewing machine works because the depth of my explanation is lacking precise details. [2]

The STEM Connection

Why is this important for learning? The concept of explanatory depth is important because it is often the case that we only have a partial understanding of many different topics. That partial understanding, however, makes us overconfident about what we know. As stated in a previous post, we want our students to be highly calibrated in judging how well they know something.

There are a couple of obvious ways to test and improve explanatory depth, and thereby increase meta-cognitive awareness. As is the case for any desired skill, practice always helps. To build meta-cognitive awareness about mental-model questions, we need to give our students as many opportunities to explain their reasoning as possible. This might be in the form of small groups so that students can explain to each other how things work. Working in small groups also exposes students to the ideas of their peers, which could be an added bonus in that students may better understand an explanation from a peer who has recently gone through the process of acquiring his or her own understanding. This is obviously in direct contrast to hearing an explanation from a teacher who has understood the topic for several years, and has thus forgotten why the topic is potentially confusing. It might also be beneficial to have whole-class discussions so students can model and test their reasoning in front of others.

Another, perhaps more controversial, recommendation would be to start using oral exams. It's likely that students will mutiny if they hear their next exam is going to be a face-to-face conversation with their teacher. But hearing a student talk about what they know is probably the best way to diagnose how deeply a student knows a topic. To prepare, the teacher would need to develop a robust scoring rubric so that it is obvious how deep a student's explanation is. 

The depth of an explanation is a pretty good proxy for how well you know something. I strongly recommend trying to explain things as much as possible because when you falter (as I did above), you uncover new opportunities to learn and refine your knowledge!


Share and Enjoy! 

Dr. Bob

For More Information

[1] Miyake, N. (1986). Constructive interaction and the iterative process of understanding. Cognitive science, 10(2), 151-177.

[2] For a great animated gif that demonstrates the sort of magic I couldn't explain, see the wikipedia entry on the sewing machine (hat tip: Mike Hasko).