Thursday, October 22, 2015

Achievement Unlocked!: Intrinsic Motivation

Editorial Note: This post represents something of a milestone for Dr. Bob's Cog Blog. Although technically not a year old, this is the blog's 52nd post. Given this milestone, I would like to take a moment to reflect on the process of blogging for a year (and enjoy some cake!). I will attempt to write my reflection in the context of intrinsic motivation.

Why write?

You are probably familiar with the distinction between intrinsic and extrinsic motivation. People are said to be intrinsically motivated when they engage in the activity for its own sake. Rock climbing is a great example because the activity itself is rewarding. While climbing, you are completely focused on the task at hand. Getting to the top provides a strong rush of accomplishment. Alternatively, people are said to be extrinsically motivated when there is an external reason for doing something. Getting paid to paint a fence is a classic example of extrinsic motivation. 

I find the distinction tenuous because motivation is almost always is a mixture of the two. For example, why do I go to work? I go to work because I get paid (external motivation); in addition, I also have the opportunity to solve interesting problems with highly intelligent people (internal motivation). Why do I play video games? I enjoy it for it's own sake (internal motivation), but I also do it because I am able to socialize with my friends by playing online (external motivation) [1]. Is there an example of purely internal or external motivation? Perhaps, but I would argue those tend to be the exception, rather than the rule.

Why, then, do I write? The act of putting words on paper isn't the first thing people think of when asked, "Are you doing anything fun this weekend?" Writing can be an excruciating process. I sometimes sit for long stretches of time, staring out the window, trying to come up with just the right example. Other times, I erase huge chunks of prose because they don't sound quite right. I decided to start a blog (and to stick with it) because I felt it was a worthwhile endeavor.

I am also extrinsically rewarded by writing a blog. Blogger – the platform that this blog is published on – generates reports on how often each post is read. That is extremely rewarding (especially since I love to analyze data). Since I'm not in academia anymore, I don't really have the opportunity to submit papers for publication. Blogging is a great alternative because it provides an outlet for some of my ideas.

Writing is about evolving.

One of my coworkers asked me, "Will you ever run out of topics?" To be honest, I had to answer in the affirmative. The reason why is because there are only a finite number of concepts that have been developed in the field of Cognitive Science. If you look in any introductory textbook, you will certainly find a limited supply. So it is very likely I will run aground and have nothing left to say. However, new studies are being published every day; so it's unlikely that I will run out anytime soon. 

So where do we go from here? First, I plan to revise my publication schedule and post a new blog according to a variable reinforcement schedule (i.e., unpredictably). This will help provide me with the time to learn about more contemporary research. As you may have noticed, most of the topics covered so far have concentrated on concepts that have been in the literature for some time. Second, I really enjoyed the guest post by Dr. Jason Chein. I plan to invite more guest writers to connect the topics that they are passionate about to education. Admittedly, this change is a bit selfish because it helps expose me to new ideas.

The STEM Connection

How does writing a blog relate to teaching? Teachers, you often find yourselves in a fairly tricky situation. On the one hand, intrinsic motivation is all around you. When you finally settled on teaching as your chosen profession, you had to completely buy into the notion that what you do makes a real difference in real students' lives. Hopefully, there is evidence all around you. You get to see it in your students as their faces light up when they finally "get it." You can also find examples of intrinsic motivation in conversations with other dedicated teachers. They help reinforce the idea that the bustling, chaotic classroom environment is full of learning activities that are enduring and meaningful. 

On the other hand, external motivation can be in scarce supply: voters turn down pay raises for educators; documentaries unveil "rubber rooms" where ineffectual teachers are warehoused [2]; and administrators are pressured to heap ever more demands on teachers' time. Thus, the well of internal motivation has to run deep enough to offset the waning external factors. 

My recommendation, then, is to focus on the intrinsic reasons for teaching, and reinforce those with the company that you keep. Additionally, if it is at all possible, collect data on your students' progress so you can see how far they've come. Finally, it doesn't hurt to have a hobby that helps refill the gas tank (like writing!) [3].

Share and Enjoy!

Dr. Bob

For More Information

[1] I think it's interesting that the video game industry has fully embraced awarding achievements. This is the ultimate in external motivation. I think this helps prove my point that motivation is multi-faceted.

[2] Chilcott, L. & Birtel, M. (Producers), & Guggenheim, D. (Director). (2001). Waiting for 'Superman' [Motion picture]. United States: Electric Kinney Films.

[3] How does becoming more busy help increase motivation? To find out, take a look at Chapter 6: The Art of Motivation Maintenance in Grant, A. M. (2013). Give and Take: Why helping others drives our success. New York: Penguin.

Thursday, October 15, 2015

The Pain Teaches Me: Desirable Difficulties

No Pains, No gains. 
–Robert Herrick (1650)

It's a simple matter of logic. 
I'm not like other people. 
I can't stand pain, it hurts me. 
Daffy Duck (1961)

Pop Quiz: We all have implicit theories about the conditions under which learning is maximized. Let's make your theory explicit. For each item below, which scenario will lead to more robust learning?

Option A
Option B
Studying in the same room where you are going to take the test

Studying in various environments
Solving a bunch of problems that are similar

Switching between different types of problems
Cramming for an exam the night before

Studying a little bit over a longer period of time
Reading and rereading material until it becomes familiar

Reading material once and quizzing yourself
Reading a text that contains gaps in the reasoning

Reading a highly coherent text that fills in all the gap

Recalibrating Our Intuitions

Sometimes our intuitions about what is best for learning are mis-calibrated. The reason is likely due to a distinction between storage strength and retrieval strength [1]. Storage strength is how well a memory is embedded within a larger network of related concepts. Retrieval strength refers to how quick or easily you can recall an item from long-term memory.  

When learning scenarios boost retrieval strength, we might be tempted to believe that we will remember the same material days, weeks, or maybe even months later. However, storage strength is more likely to be related to our successful recall. 

How, then, can we design learning events so that storage strength is maximized? One way is to introduce what is called a desirable difficulty into the learning situation. The difficulty is said to be "desirable" when it leads to more robust learning. Here are some desirable difficulties that we can introduce into the learning process.

Consistent vs. Inconsistent Settings

There is a semi-famous memory study where experimenters asked volunteers to memorize a list of words in one of two settings [2]. The first group studied their list on dry land. The second group donned scuba gear and jumped in a pool to study their list. Then the experimenters altered where the volunteers had to recall their list. The consistent group studied and recalled their list in the same setting (either both on land or both underwater) and the inconsistent group switched the context of study and recall (e.g., if they studied underwater, then they recalled on land). Guess what the experimenters found? The consistent group had better recall than those who switched settings.  

But what if we alter the amount of study that occurs? In other words, what if we asked our brave group of volunteers to study their list twice, either in the same setting or two different settings? Who would have better recall then? It turns out that studying the same material in two different settings provides a slight edge in performance. Why is this the case? It might be because the students are forced to generalize their knowledge so that idiosyncratic environmental cues are not used to help recall the information. 

Blocked vs. Intermixed Problem Solving

The above studies manipulated the context of study and retrieval. What if we manipulate the material instead? It might seem on the surface that you would want to study the same thing over and over until it becomes proceduralized. A more difficult study scenario would be to take concepts taught in different lessons and intermix them. For example, suppose you are teaching separate lessons on perimeter, area, and volume. Should you ask your students to solve a block of perimeter problems, then a block of area problems, and then a block of volume problems? Or should you teach all the lessons and then have students solve a mixture of problems from all three lessons? Suggestive evidence from a study that contrasted blocked versus intermixed problems found a consistent advantage for the intermixed problems [3].

Mass vs. Distributed Practice

As we saw in an earlier post, our memory seems to be designed such that information that isn't required tends to drop off in terms of its retrieval strength. Names of people who we haven't seen in five years don't come to mind as readily as the name of someone you saw last week. We also know that an attempt to retrieve a memory can help to reinforce that item in long-term memory. So, how do we optimize our study so that we counterbalance memory decay with making sure that we aren't wasting time on items that are already known? The best method is to increase the delay between study sessions, which will lead to a longer retention interval [4]. 

Presentation vs. Generation

When trying to learn something new, should I read and reread the material while taking notes? Or should I spend less time reading and taking notes and try testing myself instead? Perhaps a counterintuitive finding from the memory literature is that attempting to remember something has the effect of increasing that memory's strength. Therefore, it might make sense to spend less time reading and more time quizzing yourself.  

Why might this be the case? We read earlier about the generation effect, which states that memories are more likely to be recalled when we create them ourselves. When we quiz ourself on the material that we just read, it pushes our cognitive processing of the text toward the generation end of the spectrum (with the other end being repeated exposure). 

Incomplete vs. Complete Text Material

Do you learn better from a text passage that is complete or incomplete? It turns out that this is a trick question because it depends on your level of prior knowledge [5]. If you are a relative expert in a domain, you tend to learn more from an incomplete text because you must actively try to make sense of the material. If it is a more complete text, then the presentation becomes tedious and you tend to passively process the information. A low-knowledge reader, on the other hand, does not have the background to piece together an incomplete text. Instead, they need the extra information to help connect sentences together, as well as link together higher-order concepts.

The STEM Connection

Learning is hard. Why would we want to make it even more difficult? The main reason is to ensure that we don't fall into the trap of mistaking retrieval strength for storage strength. When we recognize that the goal of a learning event is to create a durable memory trace, then we can intentionally make our learning more difficult. Like exercise, our hard work will be rewarded when our learning withstands the test of time. 

Share and Enjoy!

Dr. Bob

For More Information

[1] Bjork, E. L., & Bjork, R. A. (2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. Psychology and the real world: Essays illustrating fundamental contributions to society, 56-64.

[2] Godden, D; Baddeley, A. (1975). Context dependent memory in two natural environments. British Journal of Psychology 66 (3): 325–331.

[3] Rohrer, D., & Taylor, K. (2007). The shuffling of mathematics practice problems im- proves learning. Instructional Science, 35, 481–498.

[4] Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354-380.

[5] McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14(1), 1-43.

Thursday, October 8, 2015

Better Call Saul: Memory Scanning

Let's start with a little warm-up exercise. Memorize the following list of numbers: 

   5, 9, 3, 7, 2

Once you can recite those numbers without error, answer the following question: Does your list contain the following number?


How long did it take you to answer? What if you were given the same list, but without the last digit? Would it have taken you the same amount of time? What if you were given a much longer list? Would it have taken the same amount of time?  

Reaction Times & Mental Events

Cognitive Science is a relatively new discipline. According to whom you ask, it got its start in the mid-1960s. Behavioralism was all the rage thanks to psychologists like B.F. Skinner, Edward C. Tolman, and Clark L. Hull. By the 1960s though, the theory was wearing a little thin. Psychologists began to realize they needed more than drives, stimulus-response pairs, and reinforcement schedules to explain the complexity of human behavior. Thus, experimental psychologist began to investigate how the mind encoded, stored, and manipulated symbols. In other words, scientists began to talk about the mind as a computer. It's not much of a coincidence that the rise of cognitive science coincided with the computer revolution [1].

One of the early studies of cognition that deviated from the behaviorist tradition was conducted by a scientist named Saul Sternberg. He was interested in understanding how immediate memory (a distant relative of short-term memory) worked. Specifically, he was interested in measuring how long it takes to scan immediate memory for a specific item.

Here's how he set up an experiment to measure how long it takes to scan immediate memory. First, he defined a set of items that he wanted his volunteers to memorize. Saul chose the domain of whole numbers between 1 and 9. Then, within the set of all possible items, he selected a subset of numbers for his participants to memorize. He also varied how many items were in the set (that is, the set size). For example, suppose Saul selected a set size of five, and the items in our set are: 3, 8, 9, 2, 6. Once his participant had memorized those five items, he would present a probe (e.g., 9) and ask the participant if the probe was part of the original set. In this case, the proper response is "yes" (i.e., a positive item). Saul was interested in measuring the duration between the probe and the response, or the reaction time.

When is memory scanning finished?

Saul repeated the above procedure for set sizes between one and six. Each time he measured the reaction time and plotted it against the set size (see Fig. 1). In addition, he also included probes that were not in the original set of items (e.g., for the set: 5, 9, 3, 7, 2 the probe is 6 is not in the list). We will call these negative items.

Figure 1. Mean reaction time (RT) as a function of set size
for positive and negative items.

What can we conclude from the evidence collected so far? The simple conclusion is a linear relationship between set size and the mean reaction time. For each item that you add to the set, you need an extra ~40 milliseconds to verify that the probe is in the list (or not).

A follow-up question is: Why is there a linear relationship between set size and reaction time? First, it helps to discriminate between two types of memory scanning. The first type, which we will call self-terminating serial search, compares the probe to the first item in the set. If it matches, then the search immediately stops or terminates. If it doesn't match, then the probe is compared to the second item, and so on for all the items in the set. If memory scanning is a self-terminating serial process, then there should be a linear relationship between set size and reaction time. 

The second type of memory scanning is called exhaustive serial search. In this case, the probe is simultaneously compared to all the items in the set. Only after testing all of the items can the participant give a response. 

On average the exhaustive serial search will require more comparisons than the self-terminating serial search. To discriminate between the two types of search, we need to introduce another concept called serial position, which is equivalent to the ordinal number (e.g., 1st, 2nd, 3rd, etc.) of each item. In the example above, the probe, 9, matched the 3rd item. If we use a self-terminating search, then it would be faster because it would stop after comparing only three items. In addition, negative items will take longer, on average, because you need to compare all of the items in a set to know that the probe is not among them. 

An exhaustive search would take longer than the self-terminating search because it would have to test all five items. Negative items would take the same amount of time because all items need to be tested. If we use an exhaustive search, then there should be no discernible difference between positive and negative items.

If you introspect into your own process of completing this task, what would conclude? Do you use a self-terminating or an exhaustive search? According to Sternberg's paper, we use what he calls high-speed exhaustive search. In other words, he didn't find an effect of serial position on reaction time, nor did he find a difference between positive and negative items (the orange and yellow data points in Fig. 1 are very similar). This finding is surprising because even his participants who were tested using sets that they memorized extremely well reported that they thought they used a self-terminating serial search.

The STEM Connection

How does this connect to education? First, it is an interesting, real-world demonstration of a linear function. According to the original study [2], the mean reaction time is given by the following linear equation: RT37.9s + 397.2; where s is equal to the set size. You can talk to your class about what it means to have a y-intercept of approximately 400 milliseconds, and does that value have meaning with a set size of zero. You can also talk about the limits of extrapolating past the given information. For instance, what would it mean to have a set size of 79, and is that even realistic? 

Another discussion point around this topic is its generality. Saul Sternberg kept referring to immediate memory in his original study. Since then, cognitive science has introduced refinements to this concept in the form of short-term memory and working memory. In addition, there is also a distinction between short-term and long-term memory. Do the findings from the original memory scanning experiment generalize to different types of memory? For example, do we use a serial exhaustive search when scanning long-term memory? Do the results generalize to other types of items, such as images, sounds, and words? Science advances when we start to question the boundary conditions of the original findings. 

Memory scanning studies, which made extensive use of reaction times as a dependent measure, helped to usher in a new branch of experimental psychology. They helped move us away from training animals to investigating unseen mental processes. And for that, we are very lucky for Saul Sternberg.

Share and Enjoy!

Dr. Bob

For More Information

[1] Isaacson, W. (2014) The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution. New York: Simon & Schuster.

[2] Sternberg, S. (1969). Memory-scanning: Mental processes revealed by reaction-time experiments. American Scientist, 421-457.

Thursday, October 1, 2015

If the Shoe Fits: The Instructional Fit Hypothesis

Take a look at the map below. Suppose you live on Jackson Avenue, and you need to go to the store to buy a new pair of shoes. What route will you take? 

Once you've planned your route, how will you remember it? Will you:
  • Form a mental map, position yourself on that map, and update your position as you travel
  • Form a mental list of verbal directions (e.g., head toward Main Street; after Main, turn right on 1st; then, take a left on Madison; the shoe store will be on the right). 

In the unlikely event that you take a wrong turn, which strategy will be more helpful in getting you to your destination? Which representation is easier to store in working memory? Which one is easier to use while driving? 

Connecting Instruction to Learning

In this blog, we've discussed many different types of representations that the mind uses to store and organize information. We also have talked about various learning strategies that help us acquire new information. One aspect of the discussion that's been missing is how these pieces all fit together. Are there some learning strategies that are more likely to give rise to one type of representation over another?

To answer this question, my collaborators and I set out to test the hypothesis that certain forms of instruction inspire learning events that translate into specific types of representations. We called it the instructional fit hypothesis, in that instruction should match the type of learning that we want to elicit. Before we talk about how we tested the instructional fit hypothesis, let's review the assumptions on which it is based.

Our Assumptions

First, we assume there are different types of mental representations that we use to reason about the world and to solve problems. This assumption is supported by plenty of evidence that people construct and use many different types of representations. For the purposes of the present discussion, let's focus on two representations: mental models and problem-solving schemas. As we saw, a mental model is an image or dynamic simulation that allows the individual to make inferences based on that model. For example, we might not know how many windows there are in our house, but we can mentally walk from room to room and count them. A mental model can also be incomplete or incorrect, in which case I can add details to or correct my model as I encounter new relevant information. problem-solving schema, in contrast, can be thought of as a recipe for solving a problem. In a previous post, we introduced the idea of a production rule, which is an if/then statement that says what to do when certain conditions are met. A problem-solving schema links multiple if/then statements so that a problem can be solved.

Second, we assume that certain types of representations are better suited to solve specific types of problems. It is more expedient to use a mental model of the circulatory system to diagnose a heart problem than it would be to use a problem-solving schema in which several production rules have to be tested to find one that matches the symptoms to the root cause. Likewise, it is easier to solve a multi-step math problem using a problem-solving schema than by constructing a mental model of that particular problem. To be effective and efficient, the representation and the problem-solving demands should match. 

Our third and final assumption is that certain types of instruction lead students to engage in specific types of cognitive processing. For example, suppose I instruct one of my classes to write a summary of a passage about the circulatory system. For my other class, I ask them to answer difficult questions like: Why would the distribution of oxygen be less efficient if there is a hole in the septum? The first class would concentrate on a surface-level understanding of the text because the task requires them to remember the sentences of the text rather than the underlying meaning. The second class would need to understand the interplay of multiple structures as well as their functions within the system as a whole.

Now that we've laid out all of our assumptions underlying the instructional fit hypothesis, let's put it all together. First, we start with the question: What do we want our students to be able to do or know? These are the task demands. Once we know what they are, then we ask, which representation is best suited for our learning goal? Then we figure out which cognitive processes are most likely going to lead to the generation of that representation? Finally, we ask, which instructional activity will most efficiently give rise to those cognitive processes? If we sketch it out, the chain of events might look like something like this:

Figure 1. The hypothesized chain of events.

The Study and the Evidence

To test the instructional fit hypothesis, we asked high-school students to learn about an advanced topic in physics (electrodynamics) under a couple of different experimental conditions. The first condition nicely mapped all of the steps from Figure 1. The instructional activity inspired the cognitive processes that we believed would lead to a useful representation to solve electrodynamics problems (i.e., a problem-solving schema). For the other experimental condition, the fit wasn't as nice. The instructional activity prompted the students to build and modify a mental model. While useful to visualize the problem situation, a mental model does not specify how to arrive at a numeric solution.

We asked our participants to solve their electrodynamics problems with a computer tutor called the Andes Physics Tutor [2]. Students can ask Andes for a hint to help them when they get stuck. As one measure of how difficult it was for students to solve the problems under the two different conditions, we counted the number of hint requests. It turned out that students who were prompted to form a problem-solving schema asked for fewer hints than the students who received the mental-model instructional activity. This provided preliminary evidence in favor of the instructional fit hypothesis.

The STEM Connection

The implication  of the instructional fit hypothesis for STEM education is fairly straight forward. The bottom line is: try to align instructional activities to cultivate the mental representation(s) that will be most useful to your students as they work to achieve specific learning objectives. Fitting instructional activities to the task demands, however, can sometimes be a challenge. One way to accomplish this would be to begin designing a new lesson by conducting a rigorous task analysis. If you're fortunate enough to know someone who is already an expert in the target domain, consider asking her to talk through her process while she solves a problem similar to one you would like your students to be able to master. After she is done, go back and ask, "How did you know to take this step?" or "What knowledge did you rely on to figure this out?" The goal is to figure out which representations an expert in the area relies on to produce an efficient solution. 

Once you have a handle on the task demands and the representations an expert uses, the hard part is to figure out what instructional activities can most effectively inspire those types of representations. In our study, we relied on 20 years of research on self-explaining to come up with our activities. That literature was robust enough that we could theorize about a potential match or mismatch between the instructional activities and the representations that are needed.

Obviously, this is a time-consuming process. But if we can understand the chain of events a little better, then we will certainly be able to improve our instruction! 

Share and Enjoy!

Dr. Bob

For More Information

[1] Nokes, T. J., Hausmann, R. G., VanLehn, K., & Gershman, S. (2011). Testing the instructional fit hypothesis: the case of self-explanation prompts. Instructional Science, 39(5), 645-666.

[2] VanLehn, K., Lynch, C., Schultz, K., Shapiro, J. A., Shelby, R., Taylor, L., et al. (2005). The Andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence and Education, 15(3), 147–204.