Better Call Saul: Memory Scanning

Let's start with a little warm-up exercise. Memorize the following list of numbers: 

   5, 9, 3, 7, 2

Once you can recite those numbers without error, answer the following question: Does your list contain the following number?

   7 

How long did it take you to answer? What if you were given the same list, but without the last digit? Would it have taken you the same amount of time? What if you were given a much longer list? Would it have taken the same amount of time?  

Reaction Times & Mental Events

Cognitive Science is a relatively new discipline. According to whom you ask, it got its start in the mid-1960s. Behavioralism was all the rage thanks to psychologists like B.F. Skinner, Edward C. Tolman, and Clark L. Hull. By the 1960s though, the theory was wearing a little thin. Psychologists began to realize they needed more than drives, stimulus-response pairs, and reinforcement schedules to explain the complexity of human behavior. Thus, experimental psychologist began to investigate how the mind encoded, stored, and manipulated symbols. In other words, scientists began to talk about the mind as a computer. It's not much of a coincidence that the rise of cognitive science coincided with the computer revolution [1].

One of the early studies of cognition that deviated from the behaviorist tradition was conducted by a scientist named Saul Sternberg. He was interested in understanding how immediate memory (a distant relative of short-term memory) worked. Specifically, he was interested in measuring how long it takes to scan immediate memory for a specific item.

Here's how he set up an experiment to measure how long it takes to scan immediate memory. First, he defined a set of items that he wanted his volunteers to memorize. Saul chose the domain of whole numbers between 1 and 9. Then, within the set of all possible items, he selected a subset of numbers for his participants to memorize. He also varied how many items were in the set (that is, the set size). For example, suppose Saul selected a set size of five, and the items in our set are: 3, 8, 9, 2, 6. Once his participant had memorized those five items, he would present a probe (e.g., 9) and ask the participant if the probe was part of the original set. In this case, the proper response is "yes" (i.e., a positive item). Saul was interested in measuring the duration between the probe and the response, or the reaction time.

When is memory scanning finished?

Saul repeated the above procedure for set sizes between one and six. Each time he measured the reaction time and plotted it against the set size (see Fig. 1). In addition, he also included probes that were not in the original set of items (e.g., for the set: 5, 9, 3, 7, 2 the probe is 6 is not in the list). We will call these negative items.

Figure 1. Mean reaction time (RT) as a function of set size
for positive and negative items.

What can we conclude from the evidence collected so far? The simple conclusion is a linear relationship between set size and the mean reaction time. For each item that you add to the set, you need an extra ~40 milliseconds to verify that the probe is in the list (or not).

A follow-up question is: Why is there a linear relationship between set size and reaction time? First, it helps to discriminate between two types of memory scanning. The first type, which we will call self-terminating serial search, compares the probe to the first item in the set. If it matches, then the search immediately stops or terminates. If it doesn't match, then the probe is compared to the second item, and so on for all the items in the set. If memory scanning is a self-terminating serial process, then there should be a linear relationship between set size and reaction time. 

The second type of memory scanning is called exhaustive serial search. In this case, the probe is simultaneously compared to all the items in the set. Only after testing all of the items can the participant give a response. 

On average the exhaustive serial search will require more comparisons than the self-terminating serial search. To discriminate between the two types of search, we need to introduce another concept called serial position, which is equivalent to the ordinal number (e.g., 1st, 2nd, 3rd, etc.) of each item. In the example above, the probe, 9, matched the 3rd item. If we use a self-terminating search, then it would be faster because it would stop after comparing only three items. In addition, negative items will take longer, on average, because you need to compare all of the items in a set to know that the probe is not among them. 

An exhaustive search would take longer than the self-terminating search because it would have to test all five items. Negative items would take the same amount of time because all items need to be tested. If we use an exhaustive search, then there should be no discernible difference between positive and negative items.

If you introspect into your own process of completing this task, what would conclude? Do you use a self-terminating or an exhaustive search? According to Sternberg's paper, we use what he calls high-speed exhaustive search. In other words, he didn't find an effect of serial position on reaction time, nor did he find a difference between positive and negative items (the orange and yellow data points in Fig. 1 are very similar). This finding is surprising because even his participants who were tested using sets that they memorized extremely well reported that they thought they used a self-terminating serial search.

The STEM Connection

How does this connect to education? First, it is an interesting, real-world demonstration of a linear function. According to the original study [2], the mean reaction time is given by the following linear equation: RT = 37.9s + 397.2; where s is equal to the set size. You can talk to your class about what it means to have a y-intercept of approximately 400 milliseconds, and does that value have meaning with a set size of zero. You can also talk about the limits of extrapolating past the given information. For instance, what would it mean to have a set size of 79, and is that even realistic? 

Another discussion point around this topic is its generality. Saul Sternberg kept referring to immediate memory in his original study. Since then, cognitive science has introduced refinements to this concept in the form of short-term memory and working memory. In addition, there is also a distinction between short-term and long-term memory. Do the findings from the original memory scanning experiment generalize to different types of memory? For example, do we use a serial exhaustive search when scanning long-term memory? Do the results generalize to other types of items, such as images, sounds, and words? Science advances when we start to question the boundary conditions of the original findings. 

Memory scanning studies, which made extensive use of reaction times as a dependent measure, helped to usher in a new branch of experimental psychology. They helped move us away from training animals to investigating unseen mental processes. And for that, we are very lucky for Saul Sternberg.


Share and Enjoy!

Dr. Bob

For More Information

[1] Isaacson, W. (2014) The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution. New York: Simon & Schuster.

[2] Sternberg, S. (1969). Memory-scanning: Mental processes revealed by reaction-time experiments. American Scientist, 421-457.

They Call Me the Working Man: Working Memory (Part 1)

Editorial Note: For the next two weeks, I want to discuss the distinction between short-term memory and working memory. Once we've sorted out the differences, then we will dive into the connection between working memory and intelligence. First, let's talk about how to model what's going through your mind...right now.

Short-term vs. Long-term Memory

In a previous post, we talked about the distinction between short-term and long-term memory. The evidence for proposing that there are two distinct systems came from a study that demonstrated enhanced memory for items that were early in a list of words, as well as superior recall for items later in the list. To make sense of this type of U-shaped curve, the authors theorized that the items early in the list made it into a permanent memory buffer, whereas the items that occurred later in the list were still hanging around in short-term memory.

In addition to behavioral evidence, there is also neuro-scientific evidence for the two memory systems. Using a methodology called a double-dissociation, neuroscientists demonstrated that some patients have damage to their long-term memory, but their short-term memory works just fine. The double-dissociation was established when they also found patients with the opposite problem. Namely, patients' long-term memory was intact; however, they had difficulty remembering information for a short period of time.

Working Memory & The Three Sub-components

Although short- versus long-term memory was successful in explaining some of the empirical findings, it became clear that it couldn't explain all of the behavioral results. Here is an example. Consider the following list of words: pit, day, cow, pen, rig. According to the research on the limitations of short-term memory, these five items should fit comfortably in short-term memory. But consider a different list of words: man, cap, can, map, mad. Does it seem harder to remember these words? According to the model of short-term memory, this list should be neither easier nor harder than the previous list of words because, again, there are only five items. How do we reconcile these observations?

Because the concept of "short-term memory" was unable to explain these findings, the concept of a temporary memory buffer had to be extended. To do so, a cognitive scientist named Alan Baddeley proposed a revision to short-term memory that he called working memory [1, 2]. It is similar to short-term memory in the sense that it is a temporary storage facility, but it had to be elaborated to help explain why phonetically similar words, such as cap/map and man/mad were easily to confuse when trying to remember them. The new model of memory included three distinct sub-components: the central executive, the phonological loop, and the visuo-spatial sketch-pad. To see how these components interact, Baddeley provided the following diagram (see Fig. 1).

Figure 1. A schematic representation of the working memory components.

Central Executive

The first component is called the central executive. It is responsible for focusing your attention on relevant information and to switch attentional focus when needed. In other words, it is the central executive's job to coordinate the flow of information to and from the subsystems to accomplish a task. An example of coordinating information occurs when you are attempting to navigate with a map. You have to hold spatial information from the map in mind while looking up at the real world. The central executive has to synthesize the spatial information from the map with the verbal information located on the street signs.

Phonological Loop

The next component is the articulatory or phonological loop. The best way to visualize the phonological loop is to imagine an extremely short cassette tape. When I say "extremely short," I mean it only can hold about two seconds of audio or phonological information. It's also called an "articulatory" mechanism is because it replays the audio over and over. This makes intuitive sense because when people have a list of numbers or words they have to remember for a short period of time, they repeat it to themselves over and over. The purpose of rehearsing the list is to hold that information until it can be recalled. After which time, it can be dumped from the phonological loop.

Visuo-Spatial Sketchpad

Finally, the visuo-spatial sketch-pad is meant to track and momentarily retain spatial information. For example, when driving on the highway, it is necessary to keep track of the arrangement of cars behind you so that you don't unintentionally cut someone off when changing lanes. A quick glance in your rearview mirror quickly updates the spatial information found in the visuo-spatial sketch-pad.

"Are you sure we're not getting some interference?"

Occam's razor posits that the simplest explanation is best. Do we really need three different sub-components? In the case of a momentary memory storage, I think it is completely warranted [3]. The concept of working memory, which includes a central executive aided by two sub-systems, can explain behavioral findings that a unitary concept of short-term memory could not. Probably the best example of a finding that working memory can explain, but short-term memory cannot, is the concept of interference. 

Suppose we play a game similar to the old electronic game Simon. We will play two rounds. In the first round, just play as usual. For the second round, however, you have to repeat the word the. How did you do? If you're like most people, repeating the doesn't really interfere with your ability to play the game because the information is held in a spatial buffer.

However, suppose I ask you to memorize the following list of words, but after you read through the list, you have to repeat the.

• Butterfly
• Airport
• Kitchen
• Church
• School
• Knife
• Solid

Now how did you do? If you're like me, it is impossibly hard. Why? Because the articulatory loop can't do its job refreshing the contents of the list that you want to remember.

That concludes Part 1 of our discussion of working memory. Check back next week for the link between working memory and intelligence, plus the connection to education!

Share and Enjoy!

Dr. Bob

For More Information

[1] Baddeley, A. D., & Hitch, G. J. (1974). Working memory. The psychology of learning and motivation, 8, 47-89.

[2] Baddeley, A. (2000). The episodic buffer: a new component of working memory? Trends in cognitive sciences, 4(11), 417-423.

[3] There have been further refinements to the model of working memory. For example, Baddeley proposed that an additional set of components are needed to bind episodic information held in long-term memory to the contents of working memory. Here is a schematic of those components (see Fig. 2). 

Figure 2. A further elaboration of the working memory model.

Baddeley, A. (2003). Working memory: looking back and looking forward. Nature reviews Neuroscience, 4(10), 829-839.

Midnight in the Garden of Encoding and Retrieval: Memory Models

A Simple Model of Memory

In a recent post, I made the following claim: Learning cannot occur when there is no attention. In other words, information must pass through all of the attentional filters before learning can take place. We also talked about working memory as an important buffer, and it is a location where information temporarily resides. But what happens after that? How do we store information for later use? Once we have a model for learning, then we can theorize where things might go wrong. If we better understand where things go wrong, then we can help debug those processes and help our students do a better job learning new information.

To motivate this a little bit, remember the first time you learned the Macarena? You first watched people flail on the dance floor in perfect synchrony. Then, you practiced the dance steps in the privacy of your bedroom. Finally, at the next party, you expertly threw down all of the moves. How does that map onto a simple model of memory?

Let us assume that information has gone from your sensory register (a buffer that holds a ton of information but for only a very brief duration) and through the selection mechanism employed by the attentional system. Now the information is in working memory. What next? Here is a simple process-model of memory that we can use to track what happens next.

Through the process of encoding, information is passed from working memory into long-term memory. There, the information undergoes a storage process. And now for the moment of truth. The final step in this model is retrieval, where information is pulled out of long-term memory. Retrieval is when you need to recall a fact (or procedure) and put it to use. 

Debugging the Process

Each step outlined above has some probability that it will fail. It doesn't necessarily mean something is wrong (only that we are human). Let's consider each process independently. 

Encoding: As a learner, you may fail to encode the information in a precise fashion that helps you recall it later. Let me give you a perfect example of a problem with encoding. You've seen a penny before, right? So which way is Lincoln facing? What words, if any, appear above his head? Is there any information to the left or right of Lincoln? If you're like most people, this is a very difficult task. It's even hard when you are asked to pick the real penny from a lineup of fakes, as opposed to recalling all of the various features. If you struggled with this, it's because you never bothered to encode the features of a penny. And why should you? They are readily available, and it is likely that nothing really depends on you encoding all of the features.

Storage: There is some decay function associated with memories held in long-term memory. Memories typically decay when they aren't actively used. There are exceptions, of course (i.e., permastore [1] or flash-bulb memories). But for the most part, memories that aren't used fade from long-term memory. Think back to your first history class. If you're like me, it's been a while since you've thought about the names of the U.S. presidents, in order, and the dates they were in office. If so, then it's likely those memories have faded away. 

Retrieval: Finally, there may be a problem when you go to remember something. You know it is there, but you can't get access to it. A great example of this is called the "tip of the tongue" phenomenon. It's largely a problem with retrieval because you know you know something. But at the time of retrieval, something is blocking your path to that information. Often, the memory is blocked because of interference from some other, related (but irrelevant) memory. A good strategy for getting around problems with retrieval is to leave it alone, and try recalling it at some later time. The reason this works is because the interfering memory has started to fade away. 

A STEM Example

Suppose you are tasked with learning a new procedure, such as programing a computer to add two integers. Your background research suggests Python is a great programming language for beginners because the syntax is simple. You also are delighted to discover that your computer already has Python installed. 

To start the interpreter, you locate and launch the application called Terminal. Then you type python to start a session. Next you learn that a program is a collection of functions, which are small blocks of code that do something useful. To define a function, you use the keyword def followed by the name of the function, any arguments you wish to include in parentheses, followed by the colon character ":". All functions require a return value, which you specify. Your little program ends up looking like this: 

def add_two_numbers(addend1, addend2):
    sum = addend1 + addend2
    return sum

By my count, putting this together requires learning at least 8 different pieces of new information. Some of it is pretty arbitrary (e.g., using a colon to close the definition portion of the function), and some is highly conceptual in nature (e.g., a program is a collection of functions).

Now, suppose you want to teach the above lesson to someone, but you soon discover that his or her program does not work. To help diagnose why, you might ask a series of questions to figure out which memory process is at fault. Is there a problem where the information was never encoded in the first place? Has there been a long lag between the initial encoding and the first attempt to retrieve the information? If so, then there might be a problem with decay. Those memories might just have faded because of the lack of use. Finally, it could be a problem with retrieval because Java is kind of like C++, which may share some syntax with Python.

I admit, when I wrote this up, I had to go find an old Python program that I wrote a couple years ago because I couldn't remember which keyword to use. Namely, I was getting interference from Lisp, which uses defun instead. It is likely that I never actually encoded def because I always have some example code laying around (like pennies).

The model presented above is an overly simplistic version of memory. But like most things in life, problems seem simpler to solve when you have a nice working model for how they should operate in an ideal setting. 

Share and Enjoy! 

Dr. Bob

For More Information

[1] The idea of a permastore is extremely interesting, and probably warrants a separate blog in and of itself. Basically, it's the hypothesis that there are some memories that you create that will never fade away, no matter how old you get. The hard part, of course, is verifying the veracity of those memories, especially if they are auto-biographical.