Asking Good Questions

How do you know that you’ve asked the right question?

Photo by 🇻🇪 Jose G. Ortega Castro 🇲🇽 on Unsplash

Background

Season 4 of True Detective just wrapped. It stars the indomitable Jodie Foster and amazing newcomer Kali Reis (former world champion female boxer of Native American descent). Jodie’s character, police chief Liz Danver, has a habit of asking her fellow crime fighters to “ask the question”. She would then respond “That’s not the right question.” or “Yes! That’s the right question!” The story is set in Alaska and opens with the discovery of a group of scientists frozen naked in the outdoors at night. “What were the scientists doing out in the cold at night?” “Why were the scientists naked?” All valid questions. But Chief Danver thought that the question from her deputy was the right question: “What makes a man so frighten that he runs into the snow without his shoes?” Boom! The perspective shifts!

You would be familiar with this famous quote from Albert Einstein:

“If I had an hour to solve a problem and my life depended on the solution, I would spend the first 55 minutes determining the proper question to ask… for once I know the proper question, I could solve the problem in less than five minutes.”

So asking good questions is key to problem-solving. But how do we know that we’ve asked the right question? Is there a systematic way to ask good questions? To be honest, I have not been impressed with what’s written thus far on what constitutes a good or right question.

And so I dedicate my 27th weekly article to discussing the topic of asking the right question.

(I write a weekly article on bad thinking and bad practices in data analytics / data science which you can find here.)

Taxonomy of Questions

Problems are problems because the answers or solutions are NOT obvious. That is the default assumption that we are starting with when discussing any taxonomy of good questions.

I tend to think of questions, whether good or bad, as belonging to a drill-down hierarchy consisting of 3 levels. At the first level, questions are about clarifying the observed phenomenon to ascertain if it’s a real problem. At the second level, the questions are about uncovering and challenging the underlying assumptions on which we frame the problem. And at the final level, the questions are about defining and constraining the search boundaries for solutions. At each level, we can ask mediocre questions or good questions.

Let’s go back to the True Detective scene where the police discover a group of scientists frozen naked in the snow. Here are some examples of Level 1 questions:

1. What is the probability of finding a bunch of scientists frozen in the snow … naked?
2. Do the scientists look like they froze to death peacefully?
3. Did the scientists actually freeze to death?

Let’s now look at some Level 2 questions:

1. Were the deaths intentional or accidental?
2. What makes a man so frightened that he runs into the snow without putting on his shoes?

And finally, let’s look at Level 3 questions:

1. Was this a revenge killing or was it to send a message?
2. Who knew about the hidden crime that the scientists were involved in previously?

Getting it Good

But how do we know that we are asking good questions? Let’s return to the objectives of each of the 3 levels of questions.

1. Level 1 — reduce equivocality and ambiguity about a phenomenon; increase our confidence in interpreting the situation. → WHAT are we solving?
2. Level 2 — uncover hidden and foundational assumptions; change our perspective and perception of the problem → WHY are we solving it?
3. Level 3 — combines contextual knowledge to define the search boundaries for solutioning. → HOW can we solve it?

For Level 1, a good question must lead towards a convergence of interpretation of the phenomenon. If there are still multiple interpretations, then we haven’t asked the right questions. In the above True Detective Level 1 questions, Q2 and Q3 are good questions.

For Level 2, a good question must lead towards the construction of the problem statement. Having a converged interpretation of a phenomenon isn’t the same thing as landing on the right problem statement; we may all agree to what we are seeing but not agree to why it’s a problem worth solving. A good Level 2 question should shift our perspectives by revealing operating biases in the way we think about the problem statement. In the above True Detective Level 2 questions, Q2 is a good question.

For Level 3, a good question must significantly shrink the search boundaries of the solution space by giving rise to a workable hypothesis; a good Level 3 question must be data-testable. In short, answering a good Level 3 question should point us towards a possible solution. A good Level 3 question shouldn’t be open-ended. In the above True Detective Level 3 questions, Q2 is a good question.

Conclusion

The ability to ask good questions is going to be an extremely valuable skill in the digital / knowledge economy. We need to be able to distinguish questions that are soliciting information versus those that are shaping hypotheses. Sadly, we don’t talk enough about it; we don’t spend enough time unpacking it. There’s some literature out there, but they come across as abstract. Even this article feels somewhat abstract. Thinking about thinking isn’t easy, and can often lead to circular self-referencing outcomes. Hopefully, this article inspires you on your own inquisition journey.