Three Decision-Centric Questions To Clarify ChatGPT Hype And Guide Your AI Strategy

Earlier this year, ChatGPT yanked artificial intelligence (AI) out of computer science abstraction and science fiction fantasy and into the pragmatic conversations of regular business people. A simple chat interface that let everyone play combined with odd and humanizing new risks of technology that hallucinates and gets dumber at math in old age was a powerful alchemy that brought the potential of AI to life.

Renowned technology analyst firm Gartner has charted how technology hype cycles like this rise to a peak of inflated expectations before falling into a trough of disillusionment on their way to eventual long-term productive use. So, how can companies cut through AI hyperbole and disillusionment to find ways that AI can improve business decision-making productivity today?

Rather than thinking of AI as a magic wand that can improve every decision all at once, it helps to break the decision-making process down into three phases: framing, deciding, and learning. Then we can ask how AI helps in each phase.

#1 Framing - How can AI help business people to frame their decisions?

Clearly understanding the problem to be solved and the questions to be answered is essential to making better, faster decisions – it’s hard to make the right decisions by asking the wrong questions. Fortunately, the decision framing step is where new generative AI models like ChatGPT have the most potential to help.

This opportunity arises because a decision’s framing phase is almost entirely people-driven, and people are riddled with decision-making biases. We tend to frame our decisions by asking questions that support our preconceptions (selection bias), align with the last information we heard (recency bias), paint an optimistic picture (planning bias) or go along with the rest of the team (groupthink). These and many other biases cause one nearly universal result – people frame their decisions too narrowly.

Decision-makers need decision coaches or brainstorming partners to broaden their decision-making frameworks. An ideal brainstorming partner is highly informed by general business knowledge, uninfluenced by team politics, open-minded about our specific situation and a little bit whacky to prompt out-of-the-box thinking. We don’t need an expert, and we don’t need perfection. Instead, we need someone who can invoke expertise and spark creativity to ensure we aren’t missing the most critical few good questions that drive most decision-making outcomes.

Generative AI models like ChatGPT can be that decision coach and brainstorming partner. Consider this simplified real-world example from a large consumer goods company. They had a standard framework for making change-management decisions that looked something like this:

How should we address XYZ change management problem?

1. 1. What is the background of the problem?
   2. What is the urgency of addressing this issue?
   3. What is the impact if we do nothing?
   4. What are the immediate implications of the proposed solution?

Here’s a representative interaction of a decision-maker asking ChatGPT to suggest three additional questions that would improve the decision:

ChatGPT: What are the potential risks of not addressing this change management problem?

Decision-maker: Are you listening to me? That is exactly what question #3 is about! Oh, wait, maybe we should adjust that question, like “What are the risks and benefits if we do nothing?”

ChatGPT: How will addressing this problem impact our customers or stakeholders?

Decision-maker: I see what you are getting at, but it overlaps with some of these questions. Let me think about how to bring the customer lens to these decisions.

ChatGPT: What are the longterm [sic] implications if we do not take action on this issue?

Decision-maker: Hmmm…that’s not quite right, but you are definitely on to something. We do tend to focus too much on the short-term and ignore the long-term implications of our change-management decisions. I will update the question, “What are the immediate and long-term implications of the proposed solution?”

In one sense, none of ChatGPT’s coaching suggestions were “correct” as stated. However, all of them helped the decision-maker to think more clearly and broadly about the decision. This beneficial partnership between an imperfect but articulate AI and a biased but expert person is a game changer now and in the future.

#2 Deciding - How can AI give business people the insights and recommendations they need to make the right decision?

Creating a super-intelligence that can look at a bunch of data, understand what’s going on and give people the “right answer” has been the dream of AI technologists since the beginning.

Unfortunately, generative AI models like ChatGPT do not solve this problem for three reasons.

First, decision-makers don’t want insights and recommendations from broadly smart but also inexperienced and slightly whacky brainstorming partners like ChatGPT. Instead, decision-makers need insights and recommendations from reliable, highly informed domain experts with deep knowledge of their business and their specific situation.

In addition, AI models like ChatGPT are not made to analyze the large lakes of business data at every modern company's heart. They can take a swing at imitating Hemmingway but can’t replace your data analyst. Instead, when it comes to insights and recommendations, decision-makers need what venture capitalist Andreesen Horowitz has called synthesis AI. Unlike generative AI models that create large volumes of general information from straightforward prompts, synthesis AI models consume large volumes of business-specific information and boil it down to straightforward insights and recommendations.

Finally, decision-makers won’t trust AI-generated insights and recommendations if they are not transparent and traceable. The AI must be built around decision logic that human decision-makers can understand and be tied directly to specific enterprise data. That allows decision-makers to get accurate and understandable answers from the AI system when they ask:

• Transparency: What logic did you use to come up with that recommendation?
• Traceability: What specific data and information gave you those insights?

Consider another simplified real-world example from a large food company. They have standard frameworks for making commercial decisions like “How should we change our media spend?” or “Do we need to adjust our creative for brand XYZ?” Here is an example framework:

Do we need to adjust our creative for brand XYZ?

1. 1. Is the brand vibrant and relevant?
   2. Is the brand perceived as intended?
   3. Does the brand have strong mental presence?
   4. Are consumers emotionally connected?
   5. Is the brand distinct from competitors?

In their case, AI-driven insights and recommendations based on their specific brand health data are traceable and transparent according to that same decision framework. As a result, decision-makers can drill down into AI recommendations until they get to the underlying data that the AI used to generate a recommendation.

Here’s a simplified version of how that looks:

Decision-maker: Do we need to adjust our creative for brand XYZ?

AI recommendation: Yes, to match your new positioning.

Decision-maker: How does brand vibrancy and relevance impact your recommendation?

AI insight: The brand is not being associated with its selling message, whereas competitors are at much higher rates. This is limiting purchases and emotional connection.

Decision-maker: What is our competitive position for brand funnel metrics?

AI insight: Brand XYZ is among market leaders in all metrics. The brand is in 4th place for Awareness, Purchase and Usage, and in 3rd place for Familiarity. Here is a graph of the current brand funnel metrics.

This example differs significantly from the previous ChatGPT “incorrect but still helpful” brainstorming coach. Here, the AI is a trusted analyst. The logic of the AI’s recommendation is entirely transparent, and the decision-maker can use the AI-generated insights to trace back to the underlying business dynamics to gain a more robust understanding and inform their judgment when making the decision.

#3 Learning - How can AI help business people continuously improve their decision-making over time?

Brainstorming better decision frameworks and providing transparent and traceable insights and recommendations are game-changing AI capabilities. However, even more AI value is unlocked when companies have decision intelligence systems that create a system of record for business decisions. These detailed records of past decisions and their outcomes are powerful training data for a new type of AI, a “learn AI” that trained on past decisions to suggest changes that optimize decision frameworks, insights and recommendations.

The missing piece at most companies back is a lack of decision records to use as AI training data. Generative AI like ChatGPT uses huge volumes of information on the Internet. Synthesis AI uses huge volumes of business data. Learn AI needs a similar volume of information about the what, who and how of business decisions and their outcomes before it can find ways to guide better decision-making.

A final real-world illustration shows how this approach can improve business outcomes. After keeping detailed records of their change management decisions and outcomes for six months, the consumer goods company mentioned in the first example above learned that many of their decisions were not adequately considering the long-term impacts. They solved this by refining their decision framework to separately consider short-term implications for the first year and long-term implications over three years.

Simple changes like this can have huge impacts. In this case, decision-making improvements have saved the company over $1 million across 60 decisions, and the value continues to grow.

The Power of Human + Machine

Thinking of people as irreplaceable team members in AI-powered decision-making systems highlights how to use AI now to leverage human expertise and expand our brilliance.

Just like automated safety systems in new cars watchfully ride along with us, beeping warnings and ready to hit the brakes if we fail to see a hazard, AI technologies can ride along with business people to help us brainstorm frameworks, synthesize insights and recommendations, and learn to improve our decision outcomes.