A High-Pitched Cavitation: Feedback

Learning By Doing

Let's play a game called, Concept Identification. No wait. That sounds really boring. How about Counter Spies, Like Us? That sounds more like a game people would actually play! Here's the backstory: 

You are a counter-intelligence officer, and you just intercepted a code from an enemy spy. Your goal is to classify their coding patterns into two types. The first is called "DAX" codes and the other is "MED" codes [1]. Based on your previous training, you were given the following examples. 

DAX Codes: 

 

MED Codes:

 

Now it's your turn to classify two new codes as either DAX or MED. There is one of each. The answer can be found at the end of this post [2].

(A)

(B)

How did you do? If you got them both right, then you lead your team to victory and earn a promotion! If you got one right and one wrong, then you get out of the war zone in time, but leave your mission incomplete. If you got them both wrong, then you might want to rethink your career in counter-intelligence.

"Thanks for the Feedback." –Nobody, ever.

In some situations, you want feedback. In fact, you can't survive without it. For example, it's difficult to improve your job-related skills if your manager doesn't give you explicit feedback. In other circumstances, however, feedback—especially negative feedback—is neither wanted nor appreciated. Tell a coworker they look "tired" and don't be surprised if they throw you some shade.

So what's the deal with feedback? When should we give it? When should we withhold it? When is it effective, and when does it backfire spectacularly? The obvious answer is, "It depends." Let's dig in and talk about what it depends on. 

"Right up to your face and diss'd you." –The Sounds of Science, Beastie Boys

Kluger and DeNisi conducted a meta-analysis of empirical studies on feedback [3], and they concluded there is a moderate effect size of feedback on performance (d = 0.41). However, the authors go on to explain that in a third of the studies, feedback actually decreased performance. The obvious question is why?

According to their theory, the effects of feedback can be categorized into three hierarchical categories: task learning, task motivation, and meta-tasks (which includes feedback about your self). Their theory states that feedback is effective when it is directly targeted to the task. But if you move up the chain, then feedback begins to lose its effectiveness. It can even backfire (i.e., make people worse) when the person feels it is targeted at them personally.

Figure 1: A hierarchical arrangement of control processes.

Let's look at some examples. Suppose we played an interactive version of Counter-Spies, Like Us, and I give you a 3-by-3 grid. I then ask you to give me a MED code. My job as the trainer is to tell you if you're right or wrong. That feedback is targeted toward the task learning and should move your attention to applying more effort to finding which features of the grid correspond to its label. Negative feedback often results in the person increasing their effort, so long as the feedback is clear, non-arbitrary, and the learner feels like there is hope in detecting the pattern. 

Suppose the feedback wasn't about the task. Instead, the feedback caused you to move your attention up the hierarchy where you focus attention on your self. You might have doubts about your ability to detect patterns, and that you lack the intelligence to do anything difficult. 

What might that feedback look like? If the feedback was something like, "No, that's not right. Most people get this eventually." Which causes you to think, "Oh great. So if I'm not getting this, then what does that say about me??" The goal, of course, is to keep the learner's attention on the task and on the specifics of what can be done to improve.

The S.T.E.M. Connection

There are many other factors that contribute to one's ability to learn from feedback, including personality factors, learning goals, prior knowledge, and task complexity. Each of these factors can interact in complex ways. 

Fortunately, Dr. Valerie Shute has compiled an extremely clear set recommendations for maximizing the positive effects of feedback and minimizing the negative effects. Starting on page 177 her paper, Focus on formative feedback, Dr. Shute outlines 31 perscriptive guidelines for offering formative feedback. The guidelines include what you should do when giving feedback, what to avoid, and when to give feedback. I highly recommend taking a look at this valuable resource. Here's just one example:

#	Prescription	Description and references
2	Provide elaborated feedback to enhance learning.	Feedback should describe the what, how, and/or why of a given problem. This type of cognitive feedback is typically more effective than verification of results (e.g., Bangert-Drowns et al., 1991; Gilman, 1969; Mason & Bruning, 2001; Narciss & Huth, 2004; Shute, 2006).

Both giving and receiving feedback is a difficult process. But, as we have seen, there are ways that we can maximize our benefit from formative feedback. Just keep it task-focused...and stop dissing people! 😉

Share and Enjoy!

Dr. Bob

Going Beyond the Information Given

[1] Tweney, R. D., Doherty, M. E., Worner, W. J., Pliske, D. B., Mynatt, C. R., Gross, K. A., & Arkkelin, D. L. (1980). Strategies of rule discovery in an inference task. Quarterly Journal of Experimental Psychology, 32(1), 109-123. 

[2] Code (A) is MED, and Code (B) is DAX. The rule that generates MED codes is there must be a yellow square in the bottom row. DAX codes are the opposite. They must not have any yellow squares in the bottom row.

[3] Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on perfor-
mance: A historical review, a meta-analysis, and a preliminary feedback interven-
tion theory. Psychological Bulletin, 119(2), 254–284.

[4] Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-189.

Achievement Unlocked!: Intrinsic Motivation

Editorial Note: This post represents something of a milestone for Dr. Bob's Cog Blog. Although technically not a year old, this is the blog's 52nd post. Given this milestone, I would like to take a moment to reflect on the process of blogging for a year (and enjoy some cake!). I will attempt to write my reflection in the context of intrinsic motivation.

Why write?

You are probably familiar with the distinction between intrinsic and extrinsic motivation. People are said to be intrinsically motivated when they engage in the activity for its own sake. Rock climbing is a great example because the activity itself is rewarding. While climbing, you are completely focused on the task at hand. Getting to the top provides a strong rush of accomplishment. Alternatively, people are said to be extrinsically motivated when there is an external reason for doing something. Getting paid to paint a fence is a classic example of extrinsic motivation. 

I find the distinction tenuous because motivation is almost always is a mixture of the two. For example, why do I go to work? I go to work because I get paid (external motivation); in addition, I also have the opportunity to solve interesting problems with highly intelligent people (internal motivation). Why do I play video games? I enjoy it for it's own sake (internal motivation), but I also do it because I am able to socialize with my friends by playing online (external motivation) [1]. Is there an example of purely internal or external motivation? Perhaps, but I would argue those tend to be the exception, rather than the rule.

Why, then, do I write? The act of putting words on paper isn't the first thing people think of when asked, "Are you doing anything fun this weekend?" Writing can be an excruciating process. I sometimes sit for long stretches of time, staring out the window, trying to come up with just the right example. Other times, I erase huge chunks of prose because they don't sound quite right. I decided to start a blog (and to stick with it) because I felt it was a worthwhile endeavor.

I am also extrinsically rewarded by writing a blog. Blogger – the platform that this blog is published on – generates reports on how often each post is read. That is extremely rewarding (especially since I love to analyze data). Since I'm not in academia anymore, I don't really have the opportunity to submit papers for publication. Blogging is a great alternative because it provides an outlet for some of my ideas.

Writing is about evolving.

One of my coworkers asked me, "Will you ever run out of topics?" To be honest, I had to answer in the affirmative. The reason why is because there are only a finite number of concepts that have been developed in the field of Cognitive Science. If you look in any introductory textbook, you will certainly find a limited supply. So it is very likely I will run aground and have nothing left to say. However, new studies are being published every day; so it's unlikely that I will run out anytime soon. 

So where do we go from here? First, I plan to revise my publication schedule and post a new blog according to a variable reinforcement schedule (i.e., unpredictably). This will help provide me with the time to learn about more contemporary research. As you may have noticed, most of the topics covered so far have concentrated on concepts that have been in the literature for some time. Second, I really enjoyed the guest post by Dr. Jason Chein. I plan to invite more guest writers to connect the topics that they are passionate about to education. Admittedly, this change is a bit selfish because it helps expose me to new ideas.

The STEM Connection

How does writing a blog relate to teaching? Teachers, you often find yourselves in a fairly tricky situation. On the one hand, intrinsic motivation is all around you. When you finally settled on teaching as your chosen profession, you had to completely buy into the notion that what you do makes a real difference in real students' lives. Hopefully, there is evidence all around you. You get to see it in your students as their faces light up when they finally "get it." You can also find examples of intrinsic motivation in conversations with other dedicated teachers. They help reinforce the idea that the bustling, chaotic classroom environment is full of learning activities that are enduring and meaningful. 

On the other hand, external motivation can be in scarce supply: voters turn down pay raises for educators; documentaries unveil "rubber rooms" where ineffectual teachers are warehoused [2]; and administrators are pressured to heap ever more demands on teachers' time. Thus, the well of internal motivation has to run deep enough to offset the waning external factors. 

My recommendation, then, is to focus on the intrinsic reasons for teaching, and reinforce those with the company that you keep. Additionally, if it is at all possible, collect data on your students' progress so you can see how far they've come. Finally, it doesn't hurt to have a hobby that helps refill the gas tank (like writing!) [3].


Share and Enjoy!

Dr. Bob

For More Information

[1] I think it's interesting that the video game industry has fully embraced awarding achievements. This is the ultimate in external motivation. I think this helps prove my point that motivation is multi-faceted.

[2] Chilcott, L. & Birtel, M. (Producers), & Guggenheim, D. (Director). (2001). Waiting for 'Superman' [Motion picture]. United States: Electric Kinney Films.

[3] How does becoming more busy help increase motivation? To find out, take a look at Chapter 6: The Art of Motivation Maintenance in Grant, A. M. (2013). Give and Take: Why helping others drives our success. New York: Penguin.

Getting Off the Couch: Motivation

In the movie Office Space, the main character is struggling to figure out what he wants to be when he grows up. He recalls a procedure from high school for determining what his profession should be:

Our high school guidance counselor used to ask us what you'd do if you had a million dollars and you didn't have to work. And invariably what you'd say was supposed to be your career. So, if you wanted to fix old cars then you're supposed to be an auto mechanic.

Suppose you didn't have to get up and go to work tomorrow. How would you spend your time? What would motivate you to get out of bed?

"It's a problem of motivation, all right?" -Peter Gibbons

Motivation is a slippery subject. To help clarify our discussion, let's start with a definition. I am using the word motivation to describe your own private rationale for engaging in some activity. In other words, motivation is your internal mechanism for figuring out how to spend your time. You might be motivated to engage in an activity because it will result in some concrete output (e.g., earning a paycheck, painting a picture, or writing a poem), builds a new memory or skill (e.g., going sight-seeing, attending a top-rope course, or skateboarding), or just passes the time (e.g., watching television or playing a game).

What are the sources of motivation? Off the top of my head, I can think of a few:

• Power
• Money
• Prestige
• Fame
• Impressing a potential mate
• Demonstrating mastery
• Contributing to something bigger than yourself (meaning)
• The promise of a better future
• Someone in power tells you what you must do (authority)
• Your or someone’s life depends on you completing a task (survival)

Some of these sources are better motivators that others [1]. For example, seeing a grizzly bear is a powerful motivator to leave the situation (i.e., survival). On the other hand, some sources are more nuanced, and they might ebb and flow. On some days you might feel like practicing the piano, while other days you just can't bring yourself to sit down in front of the keys and practice your scales (i.e., demonstrate mastery).

Thanks for all your hard work...bzzz!

Now that we have a working definition, let's talk about what the data say about motivation. One of my favorite motivation studies was led by the behavioral economist Dan Ariely [2]. The study included two experiments. The first of which asked undergraduate participants (who we will call "laborers") to find duplicate letters (e.g., "ss") on a sheet of paper filled with hundreds of letters. There were three different experimental conditions. In the Acknowledged condition, the laborers turned in their work to the experimenter. The experimenter then checked the work to see if the laborer found all the duplicate letters. In the Ignored condition, the experimenter took the sheet and, without checking the accuracy of the completed work, placed it on a very tall ream of paper. In the Shredded condition, the experimenter took the laborer's sheet and promptly ran it through a paper shredder. Bzzt!

After completing the first sheet, the laborer had to make a choice. Does she want to fill out another sheet or stop? Of course, there was a catch. The first sheet paid a "salary" of $0.55; but for each sheet thereafter, the salary decreased by $0.05. There was a diminishing return on the laborer's time. The experimenters were interested to see if manipulating the meaning of the work had any impact on how many sheets the laborers completed in the three experimental conditions. What would you predict? 

If you mentally placed yourself in the participant's shoes, you may have predicted that the Acknowledged condition completed far more sheets. And you would be correct. On average, they completed about 9 sheets, which was many more than the participants in the Ignored (~7 sheets) and Shredded (~6 sheets) conditions. By the way, there was no statistical difference between the Ignored and Shredded conditions.

In the second experiment, participants were asked to assemble Bionicle Lego robots. What could be more fun than getting paid to play with Legos!? In the Meaningful condition, the robots were placed on the experimenter's table, so the laborer could see the fruits of her labor. In the Sisyphus condition, when the laborer turned in a robot, the experimenter promptly disassembled it. Like the previous study, the laborer could decide to stop at any time. Laborers in the Meaningful condition assembled an average of 10.6 robots, while the Sisyphus condition only assembled an average of 7.2 robots.

What is the implication for motivation? It demonstrates that when you hold money constant, people are willing to work much longer on tasks that they find even the tiniest bit meaningful. The meaning in this experiment, of course, was derived from the acknowledgement from another person. The person in charge had to acknowledge that the work had been completed. Looked at another way, the worst thing a manager can possibly do is fail to acknowledge that an employee has done a task. Instead, a manager should help employees see how their work is in some way connected to a greater purpose or project. In other words, don't run your employee's work through the proverbial paper shredder once they are done.

The STEM Connection

The danger in education is that in-class assignments and homework can feel like busy work. Unfortunately, students can't fall back on rationalizing the time they spend by thinking, "Well, at least I'm getting paid." Instead, students need to find motivation elsewhere. Teachers and guidance counselors might have to periodically remind students that they are investing their time in the promise of a better future. 

Some students, however, are more concrete and live for the moment. How do we help this type of student find motivation to study? Offering rewards won't work because studying is, by its very nature, a deferred investment. Offering an external reward can also easily undermine students' intrinsic motivation [3]. Instead, the Lego study suggests that students might be motivated by keeping track of their progress. Each completed assignment is an incremental step along a much greater path. If each assignment and exam can be quantified in some way, it is highly motivating to look back and see how far you've come.

Another source of inspiration for how to motivate people is the video-game industry. We all know how addictive video games can be. What is it about their design that draws us in and keeps us coming back? Keeping track of progress is almost universally used, and so is the idea of "leveling up." As you play the game, the user becomes more proficient. Like the idea of the flow channel, a game needs to be simultaneously accessible to beginners and challenging for expert players. Thus, games need to evolve to be commensurate with the user's improving skill. Video games allow kids to demonstrate their proficiency. Is there a way we can engineer the classroom experience so that demonstrating mastery is looked upon favorably (e.g., spelling bees, math competitions, debates)? 

In summary, motivation is fickle. Sometimes we have it; sometimes it is nowhere to be found. Probably the most reliable source of motivation is spending time on an activity that we chose for ourself, and that we find meaningful. If we can help our students find a connection to something beyond themselves, then we can tap into the same motivation that has built things like Wikipedia. 


Share and Enjoy!

Dr. Bob

For More Information

[1] Pink, D. H. (2011). Drive: The surprising truth about what motivates us. New York: Penguin.

[2] Ariely, D., Kamenica, E., & Prelec, D. (2008). Man's search for meaning: The case of Legos. Journal of Economic Behavior & Organization, 67(3), 671-677.

[3] Kohn, A. (1999). Punished by rewards: The trouble with gold stars, incentive plans, A's, praise, and other bribes. New York: Houghton Mifflin Harcourt.