The Downside of Expertise: Part 2

Editorial Note: This is the second installment of The Downside of Expertise. In Part 1, we introduced the idea that experts can get stuck in a rut. They have a hard time ignoring their expertise when it comes to things riding a bike, swinging at a pitch, or generating creative solutions to a problem. In Part 2, we explore how this line of research got started, and how it might connect to education. Go find your chessboard, and we'll get started! 

Check and Mate!

As it turns out, much of what we know about expertise began with studying those who play chess — a lot of chess. To simulate the methodology that the early studies used, let's start with two chessboards. For one of the boards, we've caught the players in the middle of a game. For the other board, the pieces are scattered randomly about the board. Can you tell which board is an actual game and which one is random [1]?

Chessboard A

Chessboard B

Now that we've determined which board is which, what if I asked you to memorize the configuration of the pieces on the board? Do you think you could do it? How long would you have to study the board until you memorized all of the pieces? How many pieces do you think you could get right after studying the board for one minute?

Let's up the ante one more time. Suppose that we pitted your board memorization skills against a chess master. Amazingly, a master can reproduce an entire chessboard in about 5 seconds [2]. In a controlled laboratory experiment, scientists compared asked a chess master, a class A player, and a chess novice to memorize a mid-game or randomly scattered board [3]. For a mid-game board, the chess master was able to remember more pieces than either the class A player or the novice. He was able to memorize the spatial configuration of approximately 25 pieces because he was able to chunk them into higher-level patterns of positions one might expect during a chess game. Given what we know about well-developed retrieval structures, the master's performance comes as no surprise. 

But how did the chess expert fair with the random board? As it turns out, his performance was reduced to a novice, despite the fact that he had an extremely elaborate schema for chess positions. Because the positions didn't make any sense in the random configuration, he was forced to fall back to brute-force memorization, which carries with it all of the standard limitations of working memory. In fact, the novice slightly outperformed him!

The same principle of expertise holds for chess masters and Major League Baseball batters: Expertise is highly narrow. Once you depart from the patterns that experts expect, they perform at novice levels.

The STEM Connection

There are definite downsides to expertise when the situation completely violates an expert's deeply engrained schema or mental model. But what, if any, are the downsides for education? A master teacher is an expert in a particular content area (e.g., how to solve algebraic problems) and owns a vast repertoire of pedagogical content knowledge (e.g., how to teach algebra). 

Like the creativity study, can content knowledge get in the way of teaching? One way that it might be harmful is when teachers forget what it is like not to know something. Or, stated differently, experts might not remember the developmental sequence that a student must undergo when learning something new. When this forgetting takes place, educational researchers call it the expert blindspot.

In a series of studies [4-6], Mitch Nathan and his collaborators demonstrated the conditions under which expert teachers are blinded by their expertise. They asked teachers with different levels of content knowledge to rank-order six math problems that varied along two dimensions. For the first dimension, the unknown (x) was placed either at the beginning (e.g., x • a + b = c) or the end (e.g., [c - b]/a = x) of the problem. The start-unknown problems were essentially algebraic problems because the student had to apply the inverse of the operators to compute the unknown value. The end-unknown problems were essentially arithmetic problems because they could be solved by applying each operator in the prescribed order. 

For the second dimension, problems were stated either symbolically (e.g., x • 6 + 66 = 81.90), as a word equation (e.g., "Starting with some number, if I multiply it by 6 and then add 66, I get 81.90. What did I start with?"), or as a story problem (e.g., "When Ted got home from his waiter job, he multiplied his hourly wage by the 6 hours he worked that day. Then he added the $66 he made in tips and found he earned $81.90. How much per hour did Ted make?"). Take a moment to figure out which problems you think are the most difficult:

1. Start-unknown; story problem
2. Start-unknown; word equation
3. Start-unknown; symbolic equation
4. End-unknown; story problem
5. End-unknown; word equation
6. End-unknown; symbolic equation

All teachers agreed that the end-unknown problems were easy because they basically tell the student what to do. The surprising result was that students found the verbal problems (i.e., the story problem and word equation) to be easier than the symbolic equation for the start-unknown problems. The reason why is that the format of these problems prompted students to fall back onto various problem-solving patterns that were logic-based. In other words, students have been reasoning verbally for much longer than they have symbolically, and this became evident when they were faced with challenging, algebraic problems. 

Expertise is typically a great thing to aspire to, and I highly recommend it. So I don't want to give the impression that you should avoid becoming an expert. But I do want to make explicit some of the misconceptions that people might have about what experts can and can't do. As the authors of Freakonomics are fond of saying, there's a "hidden side to everything" [7]. For expertise, the hidden side is a difficulty in suppressing one's own expertise.


Share and Enjoy!

Dr. Bob

For More Information

[1] Chessboard A is an actual mid-game; whereas Chessboard B is a random assortment of pieces. There are multiple ways to figure it out, but one piece of evidence is that there are two black bishops on the same colored squares, which can never happen in an actual game.

[2] de Groot, A. D. Thought and choice in chess. The Hague: Mouton, 1965. 

[3] Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive psychology, 4(1), 55-81.

[4] Nathan, M. J., & Koedinger, K. R. (2000). An investigation of teachers' beliefs of students' algebra development. Cognition and Instruction, 18(2), 209-237.

[5] Nathan, M. J., & Koedinger, K. R. (2000). Teachers' and researchers' beliefs about the development of algebraic reasoning. Journal for Research in Mathematics Education, 168-190.

[6] Nathan, M. J., & Petrosino, A. (2003). Expert blind spot among preservice teachers. American Educational Research Journal, 40(4), 905-928.

[7] Dubner, S. J., & Levitt, S. D. (2010). Freakonomics: A Rogue Economist Explores the Hidden Side of Everything. HarperCollins.

The Downside of Expertise: Part 1

Editorial Note: I am so excited about the next topic that I had to split it across two posts. For Part 1, I introduce the idea that, while expertise is great, there is a cost associated with it. In Part 2, I will talk about the origin of research on expertise and its educational implications. Let's jump in with a real mind scrambler!

The Backwards Brain Bicycle 

Grab a junky bike and pull the handlebars and stem out of the steerer tube. Next, weld one gear onto the steerer tube and another gear onto the stem (the stem is the piece connected to the handlebars). Once you're done, it should look like this:

Due to your modification, when you turn the handlebars to the right, the front wheel pivots to the left (and vice versa). Before you embark on your maiden voyage, what do you suppose will happen? If you're not sure, take a look at this fascinating video.

What happened to this poor engineer? Why did it take him eight months to learn how to ride his "backwards brain bicycle"? The answer to that question is related to why a softball pitcher can reliably strike out the best batters from Major League Baseball (MLB).

"Swing and a miss" --Harry Doyle

In MLB, there is a distance of exactly 60.5 feet between the pitcher's mound and home plate. When a pitcher throws a 95 mph fastball, the ball arrives in a little less than half a second (.43 seconds, to be precise). That means the batter needs to decide whether he should swing (or not) in about a quarter of a second. Otherwise, the ball will blow past him as he stands there and ponders whether he should swing. 

Because it truly is a split-second decision, the batter must look for an edge. One place to find an edge is to move upwards in the stream of events and find a reliable cue for swinging. One of the cues that batters use is the pitcher's grip on the ball. If they hold it with two fingers over the top, then it is a fastball. If they put more distance between their forefinger and the thumb, then it will be a curve ball. The other cue that batters look for is the spin on the ball, which transmits itself as a certain "color" of pitch. 

MLB players log thousands of hours behind the plate trying to hone their batting ability. In effect, they become experts in watching, categorizing, and reacting to a variety of different pitches. So if they truly are expert batters, then why can softball pitcher Jennie Finch strike out batting legends Barry Bonds and Albert Pujols? [1]

The explanation is fairly simple. Expertise is highly narrow. When faced with a typical softball pitch, MLB batters are watching for something that will never come. They've essentially wired their brain to perceive and react (without much conscious intervention, mind you) to a highly narrow band of stimuli. Because overhand pitches used in MLB tend to fall, the batter watches for a ball that starts high and gets pulled to the ground. A softball pitcher throws underhand, so the ball starts low and has the possibility of rising. They also are watching for the aforementioned cues of the pitcher's grip and the spin on the ball, but the grip that a softball pitcher uses is completely different. By changing the narrow band of stimuli that a batter is trained to read, you can essentially reduce and expert batter to a novice, or possibly even worse.

But let's stop talking about muscular expertise. What about conceptual expertise? Are there any hidden costs there?

Looking in All the Wrong Places

Another way in which expertise can steer a person wrong is by biasing her to look for solutions within the prescribed content area of her specialty. Consider the following experiment [2] where baseball experts were given a creativity test called the Remote Associates Test (RAT). Their job was to look at a list of three words and figure out what single word binds them together. There were multiple experiments and conditions in the original study, but the one relevant to the current discussion was between lists of words where the domain knowledge was relevant and applicable and a different list of words where the domain knowledge was irrelevant and misleading. 

To make this concrete, suppose you are a baseball expert, and I give you the following three words: 

Baseball-Relevant:     WILD     DARK       FORK

What word connects these three? A baseball expert might answer PITCH (e.g., wild pitch, pitch dark, and pitch fork). The word "pitch" comes straight out of baseball, and it is therefore relevant to the solution to this RAT problem. When solving baseball-relevant problems, baseball experts had an accuracy rate of about 38%. This was roughly the same accuracy rate among baseball novices, who identified the connecting word 40% of the time.

But then the experimenters switched things up and gave baseball experts and novices a list of words that seemed like they might be connected to baseball, but ultimately they were not connected. Here is an example: 

Baseball-Irrelevant:     PLATE     BROKEN       SHOT 

What single word connects these three [3]? The first two words seem to hint at HOME (e.g., home plate and broken home), but then HOME doesn't really go with the last word (what is a home shot or a shot home?). How do you think the experts did? Their performance plummeted. Their accuracy rate dropped by over half, to 15%. Baseball novices didn't show the same drop in their performance; in fact, they showed the same accuracy rate of 40% on the baseball-relevant and irrelevant tasks. 

What's going on? Why can't baseball experts suppress their knowledge? Even when the experimenters warned them that baseball knowledge was irrelevant, they still couldn't turn it off. They seemed to be biased towards looking for solutions that are aligned with topics that they know a lot about, which actually interfered with performance when the solution was not aligned with their area of expertise. What may also be surprising is that the experts, at least for this task, did not out-perform their novice counterparts. In my next post I will explore situations in which being an expert can help, or hinder, performance, depending on the task at hand.

That concludes Part 1 of The Downside of Expertise. Check back next week for the conclusion and the connection to education!

Share and Enjoy!

Dr. Bob

For More Information

[1] Why MLB hitters can't hit Jennie Finch and science behind reaction time. Sports Illustrated, Volume 119, Issue 4. (July 29, 2014) [link] [video]

[2] Wiley, J. (1998). Expertise as mental set: The effects of domain knowledge in creative problem solving. Memory & Cognition, 26 (4) 716-730.

[3] The word that binds PLATE, BROKEN, SHOT together is GLASS.

The Thin Red Line: Precise Elaboration

Below are two lists of sentences. The first list contains short sentences with the following structure: <definite article> <adjective> <subject> <verb> <article> <direct_object>. The second list contains long sentences with the same structure, plus an extra prepositional phrase (in italics) at the end that modifies the base sentence. 

After reading the sentences, suppose I were to give you a memory test. Which sentences do you think would be easier for you to remember? Why is that the case?

Short Sentences

1. The short man bought the broom. 
2. The old man used the paint.

3. The fat man looked at the warning.

Long Sentences

1. The short man bought the broom to operate the light switch.

2. The old man used the paint to change the color of his cane.

3. The fat man looked at the warning that said keep off the thin ice.

Many of the previous posts about memory talked about memorizing individual items, like numbers, letters, or words. While it is essential to be able to remember small bits of information, it is also important that we have the ability to remember longer, more complicated pieces of information. Reading often requires the ability to encode large blocks of complicated information. What kind of strategies exist to help boost our memory for sentences? 

Elaborative Processing

As stated previously, one strategy to boost your memory is to add supplemental information. It seems counter-intuitive that adding information would increase the likelihood of remembering because now you have more to remember! It works because the mind craves two things: order and redundancy.

To demonstrate, a pair of scientists constructed some cleverly-worded sentences and gave them to participants to read [1]. The first type of sentence, or base sentences, included two nouns connected by a verb. Here is an example: 

Base Sentence: The woman hit the butcher. 

For the second type of sentence, the scientists embellished the base sentences with some color commentary, like this:

Embellished Sentence: The woman hit the butcher with a sausage.

Half of the participants read the base sentences and the other half read the embellished sentences. After they studied the sentences, the experimenters then checked the accuracy of the participants' memory for the underlined words. You can probably guess the outcome of the experiment. If you guessed that the people who read the base sentences remembered fewer items (57%) than those who read the embellished sentences (72%), then you would be correct. The explanation was that the additional information helped create a rich context for remembering the other words in the sentence. The additional commentary helped glue everything together. 

"Doctor, isn’t that incision a bit high for an appendix?"

This is an interesting finding because it shows that our memories are sensitive to the information that surrounds the target information. However, the results left a couple of open questions. First, does the type of elaboration matter? By type I am referring to the relevance of the elaboration to the main sentence. An imprecise elaboration does not add relevant information. On the other hand, a precise elaboration adds information that is semantically connected to the base sentence. A precise elaboration, for example, might explain why being "thin" is relevant to the developing story. 

The second open question deals with the generation effect, which distinguishes between elaborations that are provided by someone else or that we generate ourselves. In the experiment described in the previous section on elaborative processing, the elaborations were provided by the experimenters. What if people were able to supply their own color commentary? Would that prove to enhance memory even more?

To test these questions, some other scientists wrote another set of cleverly-worded sentences that the they gave participants to study [2]. The sentences included the following four types:

1. Base sentence: The thin man picked up the scissors.
2. Imprecise-elaboration: The thin man picked up the scissors to cut the tag off his hat. 
3. Precise-elaboration: The thin man picked up the scissors to cut the belt in half.
4. Self-elaboration: The thin man picked up the scissors ______________.

There were two phases to the experiment. For the first phase, participants either read the sentences that they were given (base, imprecise-elaboration, or precise-elaboration sentences), or they read the base sentence and generated their own elaboration (self-elaboration). During the second phase of the experiment, participants were given the base sentence with the adjective removed (e.g., thin), and their job was to recall the missing word.

Before I describe the results, can you guess the order of the conditions in terms of their performance on the cued recall test?

Fig. 1: Results from the elaboration study.

As you can see from the graph, the individuals who read sentences that were imprecisely elaborated performed the worst. They recalled fewer adjectives than the participants in the other conditions of the experiment. The best performance was found in the group that read the precisely elaborated sentences, with the self-elaboration not too far behind. 

In summary, I guess it would be a mistake to say that merely adding additional information helps boost our memory. Instead, the additional information needs to be semantically relevant.

The STEM Connection

The research on elaboration has useful applications for education. First, textbooks should be written with the second study in mind. That is, textbook authors should strive to ensure that their elaborations are precisely worded and relevant to the material being presented. The goal is to create a coherent mental model for the reader. For example, it would be insufficient to say, "Hot air rises." because an explanatory mechanism is not mentioned. Instead, the reader is forced either to memorize this bit of information or supply their own explanation for why hot air rises. Self-generated elaborations aren't necessarily a bad thing (as we will see in a future post), but the reader might be lazy and neglect the intellectual work needed to understand the passage.

The second application is on the student side. Students should train themselves to supply their own explanatory or causal mechanism when it is missing. If a piece of text doesn't make sense or seems incomplete, the reader should ask, "Why does hot air rise?" This type of self-prompting is beneficial because the reader is effectively training herself to become more meta-cognitively aware.

The research behind elaborative processing and precisely worded elaborations is interesting because we've started to move past merely memorizing lists of words; however, we are still at the sentence level. The next goal is to figure out how we process larger chunks of texts – like entire paragraphs! 


Share and Enjoy!

Dr. Bob

For More Information

[1] Anderson, J. R., & Bower, G. H. (2014). Human associative memory. Psychology Press.

[2] Stein, B. S., & Bransford, J. D. (1979). Constraints on effective elaboration: Effects of precision and subject generation. Journal of Verbal Learning and Verbal Behavior, 18(6), 769-777.

Doin' the Bull Dance. Feelin' the Flow: Flow

Grab your clubs and let's hit the links. You are lining up a putt. If you sink it, you will be two under par. All of the noises around you fade away. You don't even notice the people waiting for you in the golf cart. Time slips by, unnoticed. You draw in a breath, hold it, and swing. The ball meanders its way to the hole and drops in effortlessly.

If golf isn't your thing, then get roped in. I will be your belay. Get ready to scale the side of that boulder. You've been practicing rock climbing indoors, and now you are ready to put your skills to the test outdoors. You chalk up and move methodically up the face of the rock. Like the golfer, your attention is completely focused on the task at hand. You don't notice anybody or anything. All you care about is reaching the top. 

It is difficult to simulate exactly what it feels like to be "in the zone" because what works for each person is different. Some may experience it while programming a computer, whereas others might find it while painting. Even though the activities that get people in the zone are highly varied (e.g., sports vs. coding vs. art), there are many similarities in the experience itself. That experience is what many call Flow.

In the Zone

Why do we do what we do? Many of our daily activities are born out of duty or obligation. For example, we have to finish a presentation for our boss, or we have to buy groceries for the family. But some activities we do merely for the sake of doing them. In other words, the activity is so pleasurable that we do it for its own sake. These types of activities are the most conducive for getting into the zone, or what is also known as a flow state.

A flow state is facilitated and characterized by doing a task where:

1. There is a high likelihood of success
2. The environment is conducive to focus and concentration
3. There is a clear set of goals
4. There is immediate feedback
5. The task is all encompassing (i.e., daily concerns fall to the side)
6. The experience is enjoyable and done for its own sake
7. There is a lack of self-awareness
8. Time passes unnoticed

Some of the characteristics have to do with the sense of "self" while engaged in the activity. The concept of self is a little slippery, but we will define it as "thinking about yourself" — a meta-reflection of sorts. When engaged in flow, you stop thinking about yourself and any of the other thoughts and concerns that you have. Flow is probably enjoyable because you are focused on something other than your problems. 

It's also the case that you don't notice (or miss) the passage of time when you are in a state of flow. You start the activity, get in the zone, and then are shocked when you look at the clock and notice that a few hours felt like a couple of minutes. In addition to the passage of time, the surrounding environment, including things and people, fade into the background and aren't noticed. If external stimuli (e.g., time, self, others) aren't part of the task, then they fall outside the realm of attention and awareness. 

Finally, the other flow characteristics have to do with the task and the person. The goals are easily defined, as is the feedback. While rock climbing, you know if you've made a mistake (or are about to). The environment itself is set up so that you can completely focus.

How do I get into flow?

Because flow is a pleasurable experience, many of us spend at least some of our free time trying to get into the zone. But how can we maximize our chances of achieving flow? The answer to that question is best expressed in Csikszentmihalyi's (pronounced: CHEEK- sent-mə-HY-ee) book, Flow [1]. In it, he includes this helpful illustration: 

The highest probability of finding flow is when there is a tight coupling between a person's skill (represented on the x-axis) and the difficulty or challenge of the task (represented on the y-axis). If the person has a high amount of skill, but engaged in a task that is not at all challenging, then she will be bored. Conversely, if the person has a low amount of skill, but takes on a task that is too challenging, then that person will experience anxiety and frustration. The balance between a person's skill level and the difficulty of the task creates a flow channel. The flow channel is dynamic in the sense that it changes as a person becomes more proficient. 

The STEM Connection

Although rare, some of the best educational experiences happen when a student is able to experience flow while learning. How, then, can we apply the eight characteristics of flow to the classroom? First, we need to engineer a setting where the student is not distracted by irrelevant stimuli (that's hard to do, I know). We should also design a set of educational activities where there is a high likelihood of success based on the student's level of skill. Perhaps we could allow the student to select the level of difficulty for herself. In so doing, the student is given a chance to test the calibration of her metacognitive skills. 

In addition, the educational activity should have a clear goal with immediate feedback. Computer tutors are a natural fit because many are designed with the principle of providing immediate feedback [2]. Finally, the flow channel indicates that a student's proficiency at a particular task is going to: a.) be initially different, and b.) change over time. An added challenge to educators is to create learning experiences that accommodate a diverse and dynamically changing student population. Again, a computerized tutoring system is an ideal candidate if it can estimate a given student's level of expertise and tailor the training accordingly [3].

It's my hope that every sphere of life —work, home, or school — presents opportunities for us to get in the zone (and stay there as long as possible).

Share and Enjoy!

Dr. Bob

For More Information

[1] Csikszentmihalyi, M. (1990). Flow: The Psychology of Optimal Experience. New York: HarperCollins Publishers.

[2] Corbett, A. T., & Anderson, J. R. (2001, March). Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement and attitudes. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 245-252). ACM.

[3] Vanlehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227-265.