What Is A “Good” Decision? How Is Quality Judged?

Introduction

What is a good decision? How are decisions judged?

Most people, when asked, define good decisions as the ones that turned out well. But, must decisions be judged in hindsight—after the outcomes are apparent? Are we forced to wait until outcomes are apparent before knowing whether or not we made a good decision?

The short answer is no. Decisions do not need to be judged entirely in hindsight.

It is both possible and necessary to judge the quality of decisions before knowing the final outcomes. Excellent decision makers develop decisions that have a high probability of resulting in the desired outcomes. They judge quality in foresight and therefore have more happy endings.

Good decisions (in foresight) result in fewer surprises (in hindsight).

The quality of a decision (judged in foresight) is determined by the rigor that is applied to the problem being solved.

This paper provides an overview of how to judge the rigor of one's decision making. It describes how anyone can make better (higher quality) decisions, in any situation.

Decision Quality

Why are some decisions better than others? It depends on the quality and quantity of the information and reasoning being employed to arrive at the decision. Incorrect and/or incomplete information and reasoning lead to erroneous predictions of future outcomes and therefore to ill-informed decisions because of ill-informed expectations of what will happen as a result of a decision.

A decision is of high quality to the extent that the decision maker knows what risks they are taking by making that decision. They know how good or bad their information is and the biases inherent in their reasoning.

A good decision is one in which you know what you do not know.

Good Decisions

If a decision is made between options without a full understanding of the implications of one choice over another, but the outcome is good, was it a good decision?

No, it was just a "fortunate outcome." It was luck. Even bad decisions can have fortunate outcomes. Rigorously developed decisions (good decisions) have a higher probability of resulting in fortunate outcomes.

A bad decision is a quick, poorly informed choice. The decision maker chooses between options without understanding everything they need to know about the pros and cons of each option, or even whether all options have been considered. They do not know how good or bad their information is.

A high quality (good) decision is based on a methodical analysis of the available information and on sound reasoning.

The goodness of a decision (its quality) is a measure of how well informed the decision maker was when they chose between the options available to them.

A bad decision is one in which the decision maker was poorly informed, because of bad information, incomplete information, or faulty reasoning.

Good decisions do not depend on luck. They are not just the result of "throwing the dice"; they are examples of well-informed risk-taking.

Example Decision Part 1

You hear that an elderly family member, who lives alone, has sold their house and has invested all of their life savings into a mining venture. They invested all their money in is a gold mine in New Guinea. They heard about the gold mine in a randomly received email message.

Was this a good decision? How would you find out?

Example Decision Part 2

You call family members to investigate further and learn:

  • The elderly family member was a geologist in their youth and was actually familiar with the company and the mine site.
  • They had lived in New Guinea and knew the company's managing director.
  • Before investing they visited the mine site and had independent analysis done of the core samples.
  • All of the mine's senior management were investigated for criminal records.

Do you believe now that the investment was a good decision? It could be, but it depends on whether the decision maker understood what risks they were taking when they made the decision to invest.

High-quality decisions require good information in combination with sound reasoning to interpret that information.

Good information is information with known bias (no information is bias-free). Good information is as correct as it can be. Its weaknesses are understood.

Sound reasoning is thinking that questions itself. It is reasoning that recognizes the effects of our own bias and that of others.

Good decisions therefore depend on learning how to seek out the right information, to access its veracity, and to control the quality of one's reasoning.

Steps of Decision Making

All decisions follow the same four-step pathway. The quality of a decision is determined by the rigor that is applied to each of the following steps.

  1. Identify the problem
  2. Identify alternative solutions
  3. Compare alternatives
  4. Choose

Sources of Errors

The reasons why every decision is not brilliant can be categorized in terms of three sources of error.

  1. Known unknowns
  2. Unknown unknowns
  3. Analytical bias

Rigor is applied to each of the steps of decision making by addressing the sources of error in each. Good decisions are ones in which the effects of the sources of error are understood and minimized.

Step 1: Identify the Problem

How a problem is defined determines what is being solved for. A good decision begins by carefully defining the problem for which a solution is sought. No amount of analysis will result in a good decision if the problem is incorrectly defined at the outset.

A common cause of poor-quality decisions is poorly defined problems. For example, you and your spouse agree that the family car is getting old and undependable. You agree that something needs to be done, but do you agree on the specifics of the problem for which a solution is sought?

Depending on one's perspective the problem that seeks a solution can be defined in the context of a variety of questions:

  • Who is going to fix it?
  • Should the vehicle be replaced?
  • Should we buy a new or used vehicle?
  • Can we afford to purchase a nicer vehicle that will make our family appear more successful?
  • Is a car necessary? Can we live without one?
  • Is a car the best form of transportation for our family?

How the problem is defined then determines what solutions are considered. Bad decision making begins with a vaguely defined problem. A lack of clarity results in disagreements about the relative value of the solution options that are identified.

Using our investment example, let's consider: What problem was the elderly family member solving when they decided to invest all their money in a single, foreign mining company?

Were they trying to support an old friend in the mining business? Do they consider the investment to be a highinterest loan to a trusted friend?

You cannot judge the quality of a decision without knowing what the problem was, for which the decision is considered to be the best solution.

Example Decision Part 3

Let's imagine that the elderly family member in our example was motivated by a desire to move into an expensive retirement home. In order to do so they were told they needed to double their wealth to be able to afford the move.

The elderly family member was looking for ways to quickly double their wealth when the e-mail arrived about the gold mine in a country they had travelled to as a youth. The elderly family member interpreted the timing of the e-mail message as a good omen and decided to investigate further.

The elderly family member thoroughly investigated the mining company, going so far as to visit the mine site at the invitation of an old friend working for the company. The mine appeared to be well run and growing fast.

The family member hired geologists to review the mineral assays and core samples.

The family member studied trends in gold prices. Commodity market experts predicted a slow but steady price rise over the next two years.

By the time the elderly family member was finished researching the company, they were so enthusiastic that they invested not only their savings, but also the proceeds from the sale of their home. They believed the stock price would quadruple over the two-year period and had visions of living on a yacht.

Was it a good decision to invest everything in the gold mine?

Which Problem Were They Trying to Solve?

Effective decision making begins by carefully considering the question for which a solution is being sought. What question was the elderly family member attempting to solve when they decided to invest everything they had in one company in New Guinea?

They started out by wanting to move into a high-end retirement home. When they ultimately decided to invest everything in the gold mine, they were actually solving a different question entirely: "How can I make the most money from this gold mine?"

The problem they should have been solving for was, "Is this gold stock the best way to invest my money? (... so as to be able to move into an expensive retirement home?)"

The context of the original decision was lost. All other possible investments options were overlooked in favor of this one convenient "solution" that appeared to them in the form of an e-mail.

Good decisions start with a firm understanding of what it is that the decision is attempting to achieve. They do not drift off of their objective.

Choosing the right target (the problem that needs solving) is critical to ultimate success; but staying on target is just as important.

Step 2: Creative Alternatives

Once the problem has been correctly defined it is then possible to identify alternative solutions.

If you are retired with fixed income and need more wealth, what are all the options available to you for increasing your wealth?

This is where creativity comes in. Time and effort are required if the same old tired (or convenient new) solutions are to be avoided.

Creativity ensures that the identification of options is not restricted by mental barriers or old habits. Creativity expands the field of opportunity. It focuses on identifying all the possible solutions.

Step 3: Compare Alternatives

After all possible options have been identified, it is necessary to compare them. This is where the "tools of decision making" come into play. Techniques such as spreadsheets, regression analysis, discounted cash flows, and decision trees are tools that allow the pros and cons of choices to be coherently analyzed and compared.

High-quality decisions depend on a thorough, balanced, reasoned, and structured analysis of the available options. This is the technical part of decision making. It is the math.

Decision tools help to ensure that the comparison of options is fair and thorough.

Step 4: Choose

After all the options have been compared, someone must actually make the decision.

The final choice is a subjective, personal balancing of all the available information. Rarely is there one obvious choice on which everyone would agree.

High-quality choices are ones in which the decision maker knows what they are getting themselves into. They know what could go wrong. The options have been fairly and thoroughly compared and the risks are understood.

Good decisions come from knowing what you do not know.

Did the elderly family member in our investment example make a good decision? Did they know what risks they were taking? The only way to find out is to ask them questions and to judge their level of understanding yourself.

Rigorous Analysis vs. the Final Judgment

Do not confuse disagreements over the final selection (the decision) with the quality of analysis leading up to the decision. The final decision is a judgment call. It is subjective. However, the analysis that leads up to it should be objective.

Using our investment example, does it seem wise for a senior citizen to invest all of their life savings in a single company, even if they are well informed about that company? Most people would say no, but the decision to do so is not necessarily an issue of decision quality. It is a matter of personal preference

Whether someone thinks it is a good idea to invest in a gold stock, given the available information, is a matter of risk aversion. If you are a risk seeker, you will make riskier decisions, given the same information, than someone who is risk averse. A high-quality decision to one person may be considered way too risky by another person with a different perspective.

If you understand the risk that you are taking when making a decision, then you are making a high-quality decision. This is true even if others consider your decision to be wrong because they disagree with your interpretation of the information.

Choices are opinions. Decisions are always subjective. They are a consequence of the decision maker's history, education, state of mind, and other influences. The analysis that leads up to a decision can be objective but the final decision is ultimately based on the decision maker's viewpoint. A decision is a judgment call based on the available information and the decision maker's interpretation of it.

For example, you can provide countless examples of how reckless driving causes accidents, but doing so does not necessarily convince young drivers to become cautious drivers. Knowing what can go wrong does not prevent people from choosing risky options.

The Imperfections of Predictions

The complication is that decisions are based on predictions of future outcomes. And all predictions are subject to error.

Decisions are choices. They are the selection of one option (possible solution) over others. If there were only one option to choose from, there would be no decision to make.

The options that are being selected between are compared on the basis of predicted future outcomes. Someone must forecast what will happen as a result of making one choice or another and the decision maker chooses the best of the predicted outcomes.

We never actually know everything about what might happen. Our information is always imperfect. That is why decisions involve risk.

Risk comes from having an imperfect prediction of future outcomes on which to compare choices.

Good decision makers work hard to understand what they do not know. Bad decision makers blindly accept what they are told or think they already know.

Known Unknowns

Known unknowns are the things that you know to worry about. They are the sources of error that are included in your thinking.

For example, when choosing a new car, the buyer knows that there are many alternative brands of vehicles. They know that prices vary between brands and models. They know that various dealers charge different prices for the same model. They are aware that prices vary according to the time of year. When making a decision, the buyer must then balance these known unknowns in order to arrive at a choice that offers the best value to them.

Unknown Unknowns

Unknown unknowns are the things you do not know to include in your thinking. They are "off the radar." This is the category of information that leads to big surprises.

In our car-buying example, let's imagine that the buyer selects a big SUV, which has mediocre fuel efficiency. They made this choice without knowing that Saudi Arabia is about to have a civil war and that fuel prices are going to rise quickly. Had they known about the impending war and its probable effect on fuel costs, they would have chosen a different vehicle.

Unknown unknowns are the reason we hire subject matter experts such as lawyers, doctors, and economists.

Mechanics are hired to inspect used vehicles before making a purchase so that there will be fewer surprises. Rented expertise helps us understand what we do not know to consider in our analysis of a given situation.

Experience and knowledge convert unknown unknowns to known unknowns. Important decisions justify more time, effort, and expertise being involved in the process.

Analytical Bias

Analytical bias refers to the way individuals perceive the world around them. Bias is a result of one's experience, education, and genetics. It is the expression of how one thinks and reasons about particular subjects.

Everyone is biased; all the time. No one is truly objective on any subject. No one thinks like a machine. There is a personal component to everything we think, and in all of our interpretations of the world around us.

Analytical bias has a very detrimental effect on decision making. In its various forms, personal biases discourage us from being thorough in our analyses and from being objective in making comparisons. It exaggerates our understanding of the factors that relate to a decision and encourages quick, poorly informed decisions.

Analytical biases are the forces at play in the back of our minds, which determine how we interpret a situation. Unfortunately we cannot eliminate bias from our decision making. All we can do is learn to manage and understand the implications of our analytical biases.

Decision Making Best Practices

Good decisions require rigor. Time is needed to overcome biased and unreflective thinking and to thoroughly identify and compare options.

Below is a list of decision making practices that will help anyone reduce the weaknesses inherent in spontaneous decisions:

  1. Slow down
  2. Think critically
  3. Question assumptions (yours and theirs)
  4. Restate the problem
  5. Focus on major factors
  6. Collaborate
  7. Impose creativity
  8. Focus in and out
  9. Structure the analysis
  10. Determine what information, if available, would change the decision
  11. End with a 100,000-foot review

Following these practices helps to ensure that decisions are rigorously managed. Each of the practices contributes to the effective management of the four steps and/or the sources of error.

These practices are described in more detail in a related paper called:

Conclusion

How can we consistently make high-quality decisions? The answer is simple: by methodically controlling the steps of decision making and the sources of error in those steps.

How do we do that? First and foremost, by being aware of how biased information and unreflective thinking lead to bad decisions.

As a consequence of this awareness, we must:

  • Ensure that the best available information is employed and viewed in a critical light.
  • Think reflectively about what we believe and why we believe it.
  • Solve the right problem.

Finally, we must accept that the final decision is a subjective choice of the decision maker. There is no "right" answer. "You can lead a horse to water, but you can't make them drink."