Criss Cross: Aptitude by Treatment Interaction

Learning By Doing

 Let's play a fun game called Guess Which One. The answers are provided in the next section. No cheating! 

1. Guess which list of word-pairs has more accurate recall:
     A. A list provided by an experimenter.
     B. A list that you personally generated.

2. Guess which study method leads to deeper learning:
     A. Re-reading the material
     B. Testing yourself on the material you just read.

3. Guess which instructional method is better: 
     A. One-on-one human tutoring
     B. An intelligent tutoring system (i.e., a computer tutor)

4. Guess which study strategy is more effective: 
     A. Paraphrasing an expository text
     B. Self-explaining an expository text

5. Guess which type of text leads to a better understanding of the subject matter: 
     A. A minimally coherent text
     B. A globally coherent text

"Criss cross" –Owen, Throw Momma From the Train

If you've been reading this blog for a while now, you may have noticed that some answers have been discussed in previous posts.

1. The generation effect would predict that personally generated items are more memorable than those provided by someone else; therefore, the answer is A. 

2. The research on desirable difficulties predicts that students are better off quizzing themselves than re-reading the material. The best answer is A. 

3. This is a tough one. If you believe the early research on Intelligent Tutoring Systems, humans were the gold standard. But then Kurt VanLehn called that conclusion into question. The answer is A (but I'll accept B if you cite VanLehn, 2011). 

4. The research on self-explaining pretty clearly indicates that students learn more when they self-explain because they are using their background knowledge and reasoning to repair their flawed mental models. The answer is unequivocally B.

5. The answer is A or B. Wait, what? That's right! The answer to #5 is "it depends." This post is about the conditions upon which learning outcomes depend. Read on.

The Aptitude x Treatment Interaction

To better understand what's going on with the fifth question, let's take a step back and talk a little bit about research methodology. One of the most common experimental studies is to contrast the outcome of an experimental group with a control group. But instead of just comparing the outcomes of an experimental condition with a control condition, you have two levels of each independent variable. 

To make this more concrete, suppose you hypothesize that listening to music hurts learning performance. However, you don't think that all music hurts. Instead, you hypothesize that lyrically complex music hurts learning lists of words; whereas, instrumental music doesn't have any impact at all. 

To test your hypothesis, you design a study where there are two types of music and two types of lists to memorize. For the musical manipulation, you play a lyrically complex song versus techno music without any words. For the item manipulation, the first is a list of only words, and the second list only contains numbers. When you run this experiment, you plot the results with a line graph (see Figure 1). 

Figure 1: A cross-over interaction between music type and item type. 

Notice that the impact of music depends on the interaction between the type of music and the item type. If you listen to techno music, then there isn't any improvement or cost to recall. If you listen to lyrically complex music, then you get a little boost when memorizing lists of numbers. But if you listen to a song with lyrics, then it completely wipes out a participant's ability to memorize words. 

Said another way, there is a music-by-item interaction. When we talk about learning manipulations, we need to be sensitive to potential interactions between a student's aptitude and the learning situation they are in. Why? Because their learning outcomes might depend on it! 

Going back to our rousing game of Guess Which One, the answer to question #5 is "it depends" because students who have lots of background knowledge learn better from a minimally coherent text while students who do not have the same background knowledge learn better from a globally coherent text [1]. In other words, there is an aptitude (i.e., high vs. low background knowledge) by treatment (i.e., high vs. low textual coherence) interaction.

Students with a large amount of background knowledge are better served by minimally coherent texts because they must supply the missing information. They need to do more generative work while reading the text. As we have seen in other contexts, being generative during learning is beneficial for deep learning. The low-prior knowledge students, however, require a maximally coherent text because they lack the background knowledge to generate the connections. Therefore, they need more support and scaffolding when learning a new topic. 

The S.T.E.M. Connection

The above finding underscores how important both formative assessments and personalized learning environments are. In theory, if a teacher had enough data to diagnose how well each student understood a topic, then they could assign each student a different text. A knowledgeable student would get a minimally coherent text, while a low-knowledge student would get a maximally coherent text. 

Unfortunately, in practice, things are much more tricky. It would be a lot to ask a teacher to come up with two (or more) versions of a textbook. However, some labs are applying latent semantic analysis (LSA) to help match a given student to a particular version of a text [2]. The goal is to select a text that maximizes the reading comprehension for a particular student. This is an exciting area of research and one to keep an eye on as more textbooks are distributed digitally.

Someday, perhaps we can synthesize all of the (right) answers to Guess Which One and develop a learning platform that can handle the multitude of interactions between all of the variables that influence learning. That would be extremely powerful (and wouldn't require anyone to be thrown from a train!).


Share and Enjoy!

Dr. Bob

Going Beyond the Information Given

[1]  McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14(1), 1-43.

[2] Wolfe, M. B., Schreiner, M. E., Rehder, B., Laham, D., Foltz, P. W., Kintsch, W., & Landauer, T. K. (1998). Learning from text: Matching readers and texts by latent semantic analysis. Discourse Processes, 25(2-3), 309-336.

Why Do I Need to Know This?: The Classic Student Lament

Learning By Doing

Read the following passage, and do your best to remember all the essential information. And before you ask...yes, there will be a quiz at the end. Good luck!

The procedure is really quite simple. First you arrange things into different groups depending on their makeup. Of course, one pile may be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities that is the next step, otherwise you are pretty well set. It is important not to overdo any particular endeavor. That is, it is better to do too few things at once than to do many. In the short run this may not seem important, but complications from doing too many can easily arise. A mistake can be expensive as well. The manipulation of the appropriate mechamism should be self-explanatory, and we need not dwell on it here. At first the whole procedure will seem complicated. Soon, however, it will become just another facet of life. It is difficult to foresee any end to the necessity for this task in the immediate future, but then one never can tell.

Pop Quiz:

1. What is the first step?
2. What is the second step?
3. What might happen if you do too many things at once?
4. How will you know when you are done? 

Once you have taken a shot at answering the questions, feel free to look back at the passage to check your answers. If you had a difficult time with the quiz, why did you struggle? What was it about the passage that was hard to comprehend? What would have made it easier? When you were reading, did you stop and wonder, "What is the point of this?"

If you are an educator, then you have probably been asked, "Why do I need to know this?" or some variation therein [1]. How did you respond? Is there a reasonable answer to this question, and one that makes sense to a novice? In the post that follows, we will look at two possible responses.

Response #1: Remember there are no stupid questions, just stupid people. – Mr. Garrison, South Park

An easy, knee-jerk reaction to declare it a stupid question. Or, you could respond by saying: 

• You need to know this because any bright, upstanding student in a modern world should know these things. 
• As a voting citizen, you need to be informed of certain facts and have the ability to critically evaluate politicians' claims. 
• You don't want to be tricked as a consumer, and you should want to be able to make informed decisions.
• Or, perhaps the worst response of all: Because I said so!

Honestly, it might be impossible for you to answer the question because, as an educator, you might not know what the future holds for your students. 

For example, there is currently a huge demand for Data Scientists. As of this writing, there were 31,913 results when I searched for "data scientist" in the Jobs section of LinkedIn [2]. Every major corporation is interested in hiring someone who can apply Machine Learning to help solve their business problems. In fact, the concept of Deep Learning wasn't possible until about 10 years ago, and the advances since then have continued to accelerated. Therefore, it would be especially difficult to explain to a student that they need to know something when the technology that draws upon that body of knowledge hasn't been invented yet. Who knows what the hot job title is going to be in 10 years from now?

One way to address the Student Lament is to appeal to the unknown future and explain that knowledge is cumulative, and that there exists a prerequisite structure wherein higher-order concepts build upon foundational knowledge. For example, to understand Machine Learning it helps to know linear regression. If you want to tackle linear regression, then you might want to learn how to solve linear equations. To solve linear equations, it might help to understand ratios...and so on until you get to the most basic principles of counting.

Response #2: Always the beautiful answer who asks a more beautiful question. –E. E. Cummings

A second response to the Student Lament is that it is exactly the right question to ask, and students should never cease asking why they need to know something. Let's look back at the passage that opened this post. When you read it, did you feel like something was missing? What got in your way of comprehending the text?

In the original study that used this passage, the authors had three experimental conditions [3]. In the baseline condition, the authors had participants listen to the passage, and then they answered some recall and comprehension questions. As you might imagine, they did terribly. The second condition gave them a title to the passage, which was "Doing Laundry," only after they heard the passage. This group didn't fare any better than the baseline condition. However, the third condition was given the title before they heard the passage. That group did the best. Why? Because they had some context about what the passage was about. Providing the context allowed the participants to better comprehend what was being said.


The same is true for our students. They will most certainly do better if they understand how the current lesson fits in with the overall content of the course. It also helps to provide context because then they can draw upon their prior knowledge to better comprehend the current lesson. The passage about laundry is purposefully written so that the listener has to fill in the gaps with their prior knowledge.

The S.T.E.M. Connection

The goal of this exercise is to simulate being a clueless student, which should (hopefully) increase our empathy towards their plight of learning something new. How might that change our instruction? One simple method is to use a warm-up task. You can ask your students to work on a task that is a prerequisite to the upcoming Lesson. You could also show a curriculum map, and indicate where they currently are, and show where they are going. With the right visualization, this could be extremely empowering to the student because they can chart their progress through the sequence. It might also help them see connections between (seemingly) disparate concepts and ideas. 

As the passage that opened this post stated, "It is difficult to foresee any end to the necessity for this task in the immediate future." While certainly true for laundry, it might also be true for lesson planning!


Share and Enjoy!

Dr. Bob

Going Beyond the Information Given

[1] Another personal favorite is, "Is this going to be on the test?" Apparently, knowledge is only worthwhile unless one is tested on it!

[2] https://www.linkedin.com/jobs/search/?keywords=data%20scientist 

[3] Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 11(6), 717-726.

Reading Room Material: Misbehaving

Your Tank is Empty

You glance down at your gas gauge. You're coasting on fumes. Time to fill 'er up! There are two gas stations across the street from each other. Which gas station would you choose? 

Gas Station A: The price for gas is $2.45 if you pay with cash, and a $0.05 surcharge for using a credit card.

Gas Station B: The price for gas is $2.50 for using a credit card, and a $0.05 discount for paying with cash. 

What is "Behavioral Economics?"

I'm not sure if this is apocryphal or not, but I believe Hitler famously asked, "Guns or butter?" [1] At first glance, his statement seems like a false dichotomy. But if you dig a little deeper, you find that the common denominator is nitrogen. You can either turn nitrates into gun powder or you can convert them into fertilizer [2].

What does "guns and butter" have to do with Economics? Simply put, Economics is the scientific examination of making decisions when resources are scarce. In a world of finite resources, you have to pick. Do you want guns and tanks, or do you want plows and tractors? You can't have both, so you have to choose.

Where, then, does the "behavioral" aspect of behavioral economics come into play? I'm glad you asked because Dr. Richard Thaler, author of the book Misbehaving, has a terrific answer [3].

What's the Big Idea? 

Before delving into Dr. Thaler's big idea, a little context is necessary. Every scientific theory is based on a few core assumptions. Economics is no different. Neoclassical Economics assumes that people are rational and are able to make optimal decisions to further their own agenda. In other words, I know what makes my life worth living. It might be different from yours, but we both are able to decide what's best for ourselves. In economic jargon, we are experts at maximizing our own utility (read: "happiness"). For simplicity, Thaler refers to these rational types of people as Econs.

Assuming people are rational, various factors are deemed irrelevant when making decisions. In the scenario that opened this post, both options should be equally appealing to an Econ. If we observed Econs filling up their cars, about half of them would choose Station A and the other half would choose Station B. 

But what do we observe when we watch Humans purchase gas? Which gas station would you purchase gas from? Gas stations that offered a surcharge on credit cards quickly learn that humans overwhelmingly prefer discounts.

Thaler's book is chocked full of big ideas. Probably the biggest among them is the idea that economic theory is based on the faulty assumption that we are Econs instead of Humans. Unfortunately for Neoclassical Economics, humans are swayed by supposedly irrelevant factors, or "SIFs" for short.

The Big SIF: The Sunk Cost Fallacy

What's an example of a SIF? Personally, one of the hardest lessons for me to learn was to ignore "sunk costs." I bet this has happened to you, so I'm sure you can sympathize. A couple of years ago, I bought a ticket to a brew festival. The lineup of craft brewers was incredible, and I was excited to try some new libations. But on the day of the beer fest, I came down with fever. I tried to coax myself to go. I couldn't let all that money go to waste! But then a lesson from college came back to haunt me. That money is gone. I can't jeopardize my health because that would be considered, "throwing good money after bad." In the end, I had to stay home (and ignore the sunk cost of my ticket!). 

The sunk cost fallacy is just one example of many SIFs that Dr. Thaler describes and had a hand in discovering. By the end of the book, you are left wondering how Economist can continue to believe in a rational individual. More importantly, How must economic theory change to accommodate these SIFs? I guess it's up to us misbehaving Humans to figure it out. 


Share and Enjoy!

Dr. Bob

More Material

[1] According to the all-knowing wikipedia, it looks like Hitler didn't coin the term, but the Nazis did use the concept of "guns or butter" in their propaganda. 

[2] Making decisions on a stingy planet also calls to mind our discussion on Opportunity Costs.

[3] Thaler, R. H. (2015). Misbehaving: The making of behavioral economics. WW Norton & Company.

How to Be Shifty: Creativity (Part 2)

Learning By Doing

You are the Creative Director of a boutique advertising firm, and you just landed a huge, new account. Your client is releasing a new carbonated beverage that features orange and vanilla. You have to come up with a new logo by the end of the week. Time is of the essence, so you crack your knuckles and get to work. Unfortunately, you hit a snag. None of your initial ideas are very creative. What do you do? How do you get "unstuck?"

Getting Philosophical: Ontological Trees

Before diving into some concrete advice, let's take a step back and talk about a fundamental way we make sense of the world, through ontologies. Ontologies are useful ways of organizing the world, psychologically speaking. For example, it is useful to distinguish between "things" and "events." Because things have a physical manifestation, they have dimensional properties such as height, length, depth, and weight. They can also be further sub-categorized as alive or inert; useful to humans or not; edible or poisonous; large or small; manmade or natural. How we describe physical objects is only bound by our creativity, and what we find psychologically useful.

Events differ from things because they have a different set of properties. Events take place in a specific location and have a finite duration (i.e., a starting and an end). Events usually have agents (e.g., ants, geese, or baseball players) and outcomes (e.g., building a colony, migrating south, or winning a game). 


So far, the ontological tree I've described only has two branches. We have a "things" branch and an "events" branch. How do we know if these two branches of our ontological tree are psychological distinct? To determine if they are separate, we can use a simple linguistic test. It is perfectly logical to say, "My car is blue;" however, you would rightfully worry about my psychological health if I said, "That baseball game is blue." We know that events and things are distinct because they have a non-overlapping set of properties that are specific to that branch of the ontological tree [1].


There are many ways to carve up the world, and Fig. 1 is a plausible, yet incomplete attempt [2]. A proper articulation of all the possible branches would take an incredibly long time. Instead, this tiny ontological tree is meant to clarify the case study in the next section.

Figure 1. A plausible, yet (woefully) incomplete ontological tree.

Shifting Takes Guts

Before the 1970s, most doctors assumed that the stomach was too acidic for bacteria to survive. If you went to your doctor complaining of a stomach problem, he or she would tell you that your stomach is producing too much acid, and that you need to figure out a way to reduce the acid production. Obviously, you would need to avoid eating certain foods, such as orange juice or overly spicy food. Your doctor would also tell you to avoid stressful situations because it was believed that stress increases the production of stomach acid.


In the late 1970s and early 1980s, this dogma was challenged by two physician researchers. Barry Marshall and J. Robin Warren were interested in the etiology of peptic ulcers. They noticed that patients with ulcers had an unusually high number of bacteria (Helicobacter pylori) in their stomach. This, of course, went against wisdom of the day. For them, the correlation was too strong to ignore. 


Because the explanation at the time was so ingrained, Marshall and Warren had a difficult time convincing the medical field of the causal connection between H. pylori and peptic ulcers. In very dramatic (and highly publicized) fashion, Marshall infected himself with the bacteria, developed an ulcer, and then cured himself by taking an antibiotic. After this dramatic demonstration, the medical community was finally convinced [3].


Going back to our ontological tree, one could argue that, before the 1980s, the medical community believed that the cause of peptic ulcers was: Mental state…Emotion…Stress. However, to make their scientific discovery, Marshall and Warren had to shift across the ontological branches to: Things…Objects…Natural Kind…Animals…Bacteria. That's a highly creative shift, and one that certainly needs to be celebrated. In October of 2005, Barry Marshall and Robin Warren were awarded the Nobel Prize in Physiology or Medicine for their discovery [4].

The S.T.E.M. Connection

What can peptic ulcers teach us about generating a creative logo for your client? My recommendation is to first identify an ontology that makes sense to you. Keep in mind that there may be domain-specific ontologies (e.g., a medical ontology). Then, try to situate your current position on that tree. Finally, look across the branches and ask yourself if there is a different way to frame the problem? Can you shift across the branches and search for a new solution there?


Here's an example that advice. Von Glitschka is the Creative Director/Illustrator of Glitschka Studios. Mr. Glitschka has produced several LinkedIn courses that teach how to design creative logos, and one method that he uses is called the "word association chart." At the center, you have your focal concept (e.g., orange vanilla soda). Then you apply different categories of words to your focus. For example, can you think of any puns or metaphors that are associated with vanilla beans and oranges? The word-association strategy is particularly insightful because analogies and metaphors are especially suited for helping people shift across ontological boundaries. For example, the metaphorical statement, Mary is going to blow her lid, is a mental state (i.e., anger) represented as a thing (i.e., steam) [5]. 

So, if you ever find yourself stuck, and you are looking for a creative solution, remember: Be shifty! 


Share and Enjoy!

Dr. Bob

Going Beyond the Information Given

[1] This description of an ontology might remind you of the representation we used of the concept whale in a previous post.

[2] Chi, M. T. H., & Hausmann, R.G.M. (2003). Do radical discoveries require ontological shifts? In L. V. Shavinina & R. J. Sternberg (Eds.), International Handbook on Innovation (pp. 430-444).

[3] Thagard, P. (1998). Explaining disease: Correlations, causes, and mechanisms. Minds and Machines, 8(1), 61-78.

[4] From the Nobel Website: The Nobel Assembly at Karolinska Institutet has today decided to award  The Nobel Prize in Physiology or Medicine for 2005 jointly to Barry J. Marshall and J. Robin Warren for their discovery of “the bacterium Helicobacter pylori and its role in gastritis and peptic ulcer disease."

[5] For other fun examples of metaphorical language check out: Lakoff, G. (2008). Women, fire, and dangerous things. University of Chicago press.

Straight Outta Left Field: Creativity

Learning By Doing

Before we get started, grab something to write with and let's play a game. Here's a brick. Set a timer for 5 minutes and think of as many uses for this brick as possible. This is a safe space, so no bad ideas. Ready? Go wild!

"She's a brick...house." –The Commodores

How many ideas did you generate? Are any of them particularly creative? If so, how did you make that determination? How do you know if something is "creative" or not?

The game you just played is called the unusual uses task in the creativity literature [1]. The experimenter gives the participant a common object (e.g., a brick, paper clip, or knife) and asks the participant to generate as many uses for that object in a constrained amount of time. The experimenter then scores the ideas for their "unusualness." Since this task has been around for a long time, the experimenters have a fairly robust understanding as to what counts as "unusual" [2]. In this context, unusual is synonymous with creative. 

A more formal definition of Creativity has several components. First, for something to be creative, it must be unusual or atypical; however, it can't be so "out there" that it is incoherent, useless, or irrelevant. Second, it is almost always the case that the judgement of a "creative" solution is done by a field of experts in a domain. For example, I might think that using a brick as a door-stop is insanely creative. However, an expert will tell me that my idea has been done so many times before that they have lost count. Thus, for something to be considered "creative" it cannot simply be an incremental change to the status quo. Instead, it must move the field forward.

The Componential Model of Creativity

Now that we have a handle on the definition of creativity, it would be nice to have a model for how people generate creative work. If we had such a model, then we can potentially train people in each of the components so they can become more creative. As it turns out, there are a couple of models of creativity. The next four sections describe the Componential Model of Creativity [3]. It was first introduced by Teresa Amabile, one of the pioneers in the field of creativity research.

Domain-relevant Skills

The first component of the model is the development and acquisition of domain-relevant skills. The general consensus is that it takes 10 years of deliberate practice in a particular domain to become an expert [4]. A precursor for producing truly creative work is a huge warehouse of domain-relevant knowledge, including both declarative and procedural knowledge. To make a creative, novel contribution in a given area, you must first have an expert-level understanding of it.

Creativity-relevant Processes

In addition to an abundance of skills honed over time, the componential model suggests that having a set of creativity-relevant personality characteristics increases one’s capacity to be creative. For example, it helps if the individual can see connections between disparate ideas, sees the value in risk-taking, and does not succumb to authority. The creative individual is also able to fluently generate lots and lots of ideas, while also using her domain knowledge to select ideas that have potential. Finally, creative individuals have a "tolerance for ambiguity" in the sense that they can hold competing thoughts in their mind at the same time [5].

Task Motivation

The creative individual also has task motivation. In other words, a creative person has passion for their work. More specifically, they are intrinsically motivated to work on a challenging project. There seems to be a large amount of variation between individuals in terms of their motivation. For example, I might think that machine learning is the next best thing, but I could care less about the latest fashion trends. In other words, task motivation is largely a fit between the domain and a person’s interests.

The Social Environment

The previous components focus on the internal cognitive and motivation processes that operate within a person. The social environment is an interesting component of the model because it recognizes forces outside the creative individual. As stated in the introduction, a creative work is done within a domain, such as art, science, or engineering, and the field is comprised of other individuals who evaluate the creative output (e.g., art critics, fellow scientists, and other engineers). The social environment can both positively and negatively affect creativity. 

A shining example of an organization that fosters creativity is the design firm IDEO or the animation studio Pixar. Both organizations understand the importance of experimentation. IDEO’s mantra is, “Fail often in order to succeed sooner.”

The perfect example of a soul-crushing organization is Initech. In this fictitious company, the managers care more about enforcing arbitrary rules than letting their employees work on problems that interest them.

The S.T.E.M. Connection

You might be asking yourself, why is it important to study a model of creatively? The short answer is: If you can understand what factors influence creativity, then maybe you can teach people how to become more creative. According to the model, there are several entry points for educational intervention. For example, we know for a fact that we can teach domain-relevant skills. Our educational system is completely built around that goal. 


Teaching creativity-relevant skills could go either way. According to the model, there are some personalities that are more creative (i.e., those who take risks and question authority). We generally think that an individual is born with his or her personality, and it is difficult to change; however, there are strategies that have been developed to help train people to become more fluent in idea generation. If it is possible to teach brainstorming, then at least part of the domain-relevant skills can be taught. 


Task motivation may be a little more difficult to teach. Or it may not be amenable to training at all. Instead, I believe it is more about finding the right fit between the individual and his or her passions. Those who know what they are passionate about are lucky. For those who do not know, they may need extra guidance and other interventions to help them identify what gets them out of bed in the morning. 


Finally, as educators, we can structure the social environment so that it is conducive to creative thinking [6]. For example, we know that rigid deadlines and extrinsic rewards are a motivation killers. We also know that being overly critical, especially early in the creative process, can backfire tremendously. Thus, a creative space is one in which exploration, play, and failure are encouraged, and evaluation is withheld until the later parts of the creative process.


With the componential model of creativity in hand, it is my hope that we can expand each person's creative capacity. Now, if I could just figure out to do with this brick. 


Share and Enjoy!

Dr. Bob

Going Beyond the Information Given

[1] Beaty, R. E., & Silvia, P. J. (2012). Why do ideas get more creative across time? An executive interpretation of the serial order effect in divergent thinking tasks. Psychology of Aesthetics, Creativity, and the Arts, 6(4), 309.

[2] Rumor has it that some employers use the unusual uses task to assess the creativity of job applicants. If you aspire to work for one of these companies, it might not be a bad idea to practice thinking of unusual uses for common objects because, as [1] suggests, later ideas are better than earlier ideas. This finding also suggests you can get better at generating novel uses for common objects with practice. 

[3] Amabile, T. (2013) Componential theory of creativity. In E. Kessler (Ed.), Encyclopedia of Management Theory (pp. 135-140). Thousand Oaks, CA: SAGE Publications, Inc.

[4] Because of the time and energy required, it is an extremely rare individual who can make a creative contribution to more than one domain. They do in fact exist, and they go by the fun name of polymaths. 

[5] Tolerance for ambiguity also goes by the name Janusian Thinking, which I was first introduced to in the article: Rothenberg, A. (1996). The Janusian process in scientific creativity. Creativity Research Journal, 9(2-3), 207-231. Janus is the Roman god who had two faces. One looked toward the past, while the other gazed forward, into the future. 

[6] If design thinking is your jam, then Standard d.school has several materials to get your started, as does IDEO.