Thursday, March 22, 2018

More Human Than Human: Human Tutoring

Editorial Note: As of April 1, 2018, our paper, Learning from human tutoring, has been cited over 1,000 times. That qualifies it as a "citation classic." Congratulations to my co-authors, especially Micki, for making this happen!


Learning By Doing

Let's start with a quiz. Rank order the following learning strategies from most effective to least effective.
  1. Listening to a lecture
  2. Participating in a large-group discussion
  3. One-on-one human tutoring
  4. On-the-job training or an apprenticeship 
  5. Hands-on laboratory activity
  6. Collaborative problem solving
  7. Watching an online video
  8. Studying worked-out examples
  9. Solving problems with a computer tutor

"Sort It Out" –Vinny, Snatch (2000)

This is probably an impossible sorting task because it isn't always clear when one particular learning strategy should be employed. For example, it might depend on the learner's prior knowledge. If the material is completely new, then one-on-one human tutoring might be the best approach. Alternatively, if the student is familiar with the material and just needs a refresher, then watching a video might be the most efficient way to learn. In other words, there isn't a grand unified learning theory that can confidently recommend the exact learning activity for a student who is learning a specific topic.

That is not to say, however, we can't make an educated guess [1, 2]. We know that requiring a student to be active during the learning process (i.e., asking deep questions) is better than letting him or her be passive (i.e., watching a video). This general framework was covered in a previous post.

Thus, being active during learning is important. We also know that being interactive is also important for learning. What kind of instruction encourages interactivity? Both parents and students probably agree that one-on-one human tutoring is highly interactive and extremely effective. The open questions is: Why is human tutoring effective? 

Three Possible Explanations

Here are three possible explanations for why human tutoring is effective. 

The Tutor-centered Hypothesis

The first hypothesis suggests that human tutoring is effective because the tutor has a deep understanding of both the topic and pedagogical moves that are useful for eliciting learning from a student. For example, suppose a tutor is teaching how to solve projectile-motion problems. She knows that students frequently hold the misconception that motion in the x-direction depends on motion in the y-direction. Therefore, the tutor knows the exact question to ask when the student displays this confusion. Thus, tutoring is effective because the tutor knows exactly what to say and when to say it.


The Student-centered Hypothesis

In contrast, there is an equally valid way of looking at tutoring that focuses on the student. Instead of giving the tutor all the credit, the student is the reason why tutoring is effective. When the student is engaged with constructing his or her own understanding, then tutoring will be effective. We've seen in past research the importance of the generation effect and self-explaining. Tutoring will be effective in so far as the student is able to engage in these constructive learning activities.


The Interactive Hypothesis

The third explanation for the effectiveness of human tutoring takes a more holistic view of tutoring. Instead of crediting the student or tutor, the effectiveness of tutoring arises from their mutual interaction. In other words, the tutor appropriately sets the current learning task and the student then has the opportunity to engage in this task. The tutor is available to answer questions and give feedback as needed. The tutor is also available to back up and propose an easier task if the student gets stuck.


Let's Put It To a Test

To test these three equally plausible explanations, we conducted the following study [3]. The first part of the experiment asked tutors to teach eighth grade students about the human circulatory system. The tutors, who were nursing students, weren't restricted in any way. Their goal, of course, was to help prepare students to answer really tough questions about the circulatory system. In the second part of the experiment, we took the same tutors, gave them a fresh batch of students, and told them not to give away any information. Instead, we wanted them to get the students to do all the work.

Before we get to the results, we had to make sure that the tutors understood and followed our instructions. Sure enough, in the second round of tutoring, they explained a lot less and  asked a bunch more questions. Therefore, we were satisfied that our manipulation was a success. 

In the first study, we found a correlation between the number of tutorial explanations and shallow learning. I don't find this result very surprising. The student read the text and the tutor explained it, which was effectively like hearing the same lesson twice. The more interesting finding was that when students made reflective comments about their understanding or learning. Those types of comments were correlated with deep learning. So far, we have evidence for the first two hypotheses. What the tutor does is important for shallow learning (i.e., give tons of explanations) and what the students does is important for deep learning (i.e., be reflective or metacognitive).

In the second study, we found something else that was interesting. Recall that the tutors weren't allowed to explain or give feedback. Instead, they asked a lot of "scaffolding" questions (e.g., giving hints, asking fill-in-the-blank questions, or asking for an example). There was about three times as much scaffolding in the second study, and the number of explanations dropped precipitously (see Fig. 1). What makes this finding interesting is that the learning outcomes were the same in both studies. The students excelled on their post-tests. The reason why, we argue, is because the tutors created situations where the student could be generative, and when they engaged in the generative activities, they learned a great deal from the tutoring session.


Figure 1. The number of scaffolding and explanation
episodes for the first and second study.



The S.T.E.M. Connection

The results from the first study clearly indicate that tutors, when left to their own devices, will try and explain the material to the student. This isn't necessarily a bad thing. Repetition generally leads to learning. Unfortunately, that isn't the best way to learn the really hard material. Instead, it is a good idea to create a tutoring situation where the tutor is asking the student questions and pushing them to further develop a line of reasoning. Tutors are effective because when they ask a tough question, and when they hear the student struggle, then they back up and ask an easier question. That is where the metaphor of "scaffolding" comes from. On the flip side, it is important for the student to monitor his or her understanding. We've talked about the importance of metacognition and in this case, metacognitive monitoring was correlated with learning. To the extent possible, we should teach our students to monitor their evolving comprehension.

Human tutoring is effective for so many reasons, and it doesn't just boil down to one thing. But as we continue to investigate and probe, eventually we will be able to figure out tutoring's secret sauce. Once we do, then we can provide tutors with concrete suggestions to become even more effective. And maybe, just maybe, we can build a computer tutoring system that is as good as humans. Maybe even better [4]. 


Share and Enjoy!

Dr. Bob

Going Beyond the Information Given

[1] Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012). The Knowledge‐Learning‐Instruction framework: Bridging the science‐practice chasm to enhance robust student learning. Cognitive science, 36(5), 757-798.

[2] Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49, 219-243.

[3] Chi, M. T., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25(4), 471-533.

[4] VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems and other tutoring systems. Educational Psychologist, 46, 4, 197-221.

Thursday, March 1, 2018

Reading Room Material: Electric Dreams & Déjà vu

Phil: Do you ever have déjà vu, Mrs. Lancaster?
Mrs. Lancaster: I don't think so, but I could check with the kitchen [1].

I hate to admit it, but I don't know anything about déjà vu. However, I believe that it is: 1) something that has happened to most of us; and 2) a completely fascinating subject in its own right. Like me, you might be wondering: What causes déjà vu?

In the movie The Matrix (1999), déjà vu was explained as, "a glitch in the matrix." In other words, "they" (i.e., the system administrators) changed something. That's all well and good for people who inhabit a computer simulation, but what about people in real life? Is there another explanation?


Philip K. Dick's Electric DreamsSeason 1, Episode 1 "Real Life"

There's an alternative explanation that that recently came up in a sci-fi television show, but sounded like it was based in sci-nonfi(ction). According to the first episode of Electric Dreams ("Real Life")déjà vu is result of two normal cognitive functions: perception and memory [2]. In this episode, George, the main character, owns a company that produces virtual reality (VR) headsets. Unfortunately, he hit his head extremely hard. To deal with his current situation, he does two things. He uses a VR prototype to help him process what happened. In an unusually gripping VR experience, he is left wondering what is real and what is virtual. In addition to treating himself with technology, he contacts his personal doctor, Paula. Here is their dialog:

PaulaThere's a theory that what we call déjà vu is actually a delay in real-time information being transmitted from one hemisphere of the brain to the other. That lag is only microseconds, but when it's received in the second hemisphere, it creates a small, momentary dissonance in perception. Now, that dissonance is misinterpreted by the cerebral cortex as memory, so the mind believes that normal events have been experienced twice.

George: Yeah, but this isn't déjà vu. It's like the memories have been there all along.

PaulaWell, they are memories, but they're memories of simulated events, events from your VR program. I think your brain's having trouble processing these simulated events, and it's creating a dissonance in your cerebral cortex, just like déjà vu.

George: I tell you it feels much more real than that. I keep having these flashes of that life, and I swear, it feels right. Feels true.

According to Paula, déjà vu is a result of delayed signals traversing the corpus callosum, which is a thick band of neurons that connect the left and right hemispheres [3]. I know the show is fiction, but how good is its explanation for déjà vu? 

To evaluate Paula's theory, I consulted two resources. The first, Psychology Today, is a publication geared toward non-specialists, while the second, Neuron, publishes cutting-edge neuro-scientific researchPsychology Today returned 275 hits when I searched for "déjà vu" (mostly in the form of blog entries), and Neuron had exactly three articles with those search terms. These results tell me that this is an extremely popular topic without many studies devoted to uncovering the neuroscientific underpinnings of the phenomenon. Despite that fact, I found a small amount of evidence for the idea that there are brain systems that are out of sync [4]. For example, one study claimed that individuals who suffer from "complex partial seizures" experience hallucinations and déjà vu. They hypothesize that seizures disrupt memory and perceptual neural networks by causing them to be temporarily unsynchronized [5].

In addition to the asynchrony explanation, there is another piece that seems to fit with current findings in cognitive psychology. George reports that his experiences are authentic and feel "real" and "true." I find that interesting given the research on source monitoring. In a previous post, we discussed how people can convince themselves that they were the source of an idea, only to find out later that they picked up the idea from someone else.

Maybe there's something to the déjà vu theory put forth in Electric Dreams. But anyway you slice it, be sure you don't jeopardize recovering from a concussion with too much VR. Otherwise, you'll be staying at Mrs. Lancaster's Bed & Breakfast for a very...long...time.


Share and Enjoy!

Dr. Bob


More Material

[1] Dialog from the movie Groundhog Day (1993). The main character wakes up and experiences the exact same morning routine as the day before. 

[2] Special thanks to Springfield! Springfield! for the transcript of "Real Life". Watch Electric Dreams on Amazon Prime. 

[3] If you want to read about some really freaky stuff, checkout what happens when patients have their corpus callosum severed, effectively creating individuals with two separate brains.

[4] Uhlhaas, P. J., & Singer, W. (2006). Neural synchrony in brain disorders: relevance for cognitive dysfunctions and pathophysiology. Neuron, 52(1), 155-168.

[5] Medvedev, A. V. (2002). Epileptiform spikes desynchronize and diminish fast (gamma) activity of the brain: an “anti-binding” mechanism? Brain Research Bulletin, 58(1), 115-128.

Thursday, January 18, 2018

The Stay Puft Marshmallow Man Paradox: Ironic Processing

Learning By Doing

Let's play a simple game. There's only one rule: Don't think about white bears. I will give you a minute.

Okay, so how did you do? Did you think about white bears? If you didn't think about white bears, what did you think about instead? What was your strategy? Maybe you should teach your strategy to Dr. Ray Stantz from the movie Ghostbusters (1984).

"I couldn't help it. It just popped in there." --Dr. Ray Stantz

Why, oh why, did Dr. Raymond Stantz conjure up the Stay Puft Marshmallow Man? All he had to do was clear his mind! WHY!? It's easy. He fell victim to a rather pernicious feature of the human mind. Sometimes, when you actively try to suppress thinking about something, your mind goes ahead and thinks about it. If you ever had the experience of not being able to stop laughing in church, then you've experienced ironic processing. Ironic Processing is the cognitive phenomenon of your mind betraying you and doing exactly the opposite of what you tell it. 

The game company Hasbro cleverly figured out a way to monetize ironic processing. They designed a board game aptly called Taboo. If you haven't had the frustrating experience of playing this game, the rules are as follows. You are given a word, and your goal is to get your partner to say that word. But here's the catch. You aren't allowed to use certain words as clues. For example, suppose I want you to say the word "Sweet." I am not allowed to use the words: Sugary, Tea, Nice, Sour, Sixteen. How evil is that? I'm terrible at this game because as soon as I read the list of verboten words, I immediately want to say them. Why? Ironic processing.


"Isn't it ironic...dontcha think?" --Alanis Morissette 

So what is going on? Why doesn't your brain do what it's told? According to one theory, the mind draws upon two separate processes to direct our behavior [1]. The first is an action-oriented process. It has a goal, and it motivates us to take steps toward that goal. Let's call this the "Operate" process. The second process needs to evaluate whether the goal has been achieved. Let's call this the "Test" process. The Operate and Test processes work in tandem to achieve a goal.

The problem arises when the Test process is checking Operate's progress before the Operate process has completely finished. In other words, Test is the annoying kid in the back seat asking, "Are we there yet?" Thus, if you are actively trying to suppress a thought, and the Test process kicks in to evaluate, then it ends up causing a violation of the thought suppression. By testing if you aren't thinking about white bears, you are now in violation of the rule. The Test process puts the "irony" in ironic processing.


The S.T.E.M. Connection

What is the connection to education? Put yourself in the shoes of a student who has test anxiety. It might be tempting to advise that student to not think about his or her anxiety. You could tell him or her to avoid negative thoughts about failure or the implications of failing. I'm sure you can see the problem with that advice. It would be analogous to telling the student not to think about "white bears." The problem is, if the student is thinking about failing, then they clearly aren't thinking about the material on the test. As we have seen in previous posts, working memory and attention are severely limited resources. If they are focused on the wrong information, then there will be fewer resources available to do well on the test.

What advice should we give instead? We might take a cue from high-pressure sports where the athlete faces negative thoughts (e.g., "Don't screw up. Don't screw up"). The advice for them is to focus on something (e.g., a word or concept) that is related to the task a hand [2]. In other words, if the student is worried about failing, then give that student something to think about instead.

It's not easy to do, obviously. But knowing how ironic processing works might help students understand how their mind betrays them. More importantly, knowing about the Operate and Test processes might also help students formulate their own strategies for handling situations when processing turns ironic. Once they have those strategies in hand, perhaps they can teach them to Dr. Stantz so he doesn't "accidentally" destroy downtown New York.


Share and Enjoy! 

Dr. Bob


Going Beyond the Information Given

[1] Wegner, D. M. (1994). Ironic processes of mental control. Psychological Review, 101(1), 34.

[2] Dugdale, J. R., & Eklund, R. C. (2002). Do not pay any attention to the umpires: Thought suppression and task-relevant focusing strategies. Journal of Sport and Exercise Psychology, 24(3), 306-319.