Considerations

Is Data Making You Dumb?

Decisions over data

In this era of big data, leaders often worship data and ignore rationality.^xxiii Data-first approaches unfortunately influence “data-inspired” decisions more often than “data-driven” ones. As Turing Award winner and prominent causal AI researcher Judea Pearl said: “Data are profoundly dumb. Data can tell you that the people who took a medicine recovered faster than those who did not take it, but they can't tell you why.”^xxiv

A “data-inspired” approach led officers to the wrong question about armor placement in World War II. Wald found a better answer because he modeled the environment and relationships that created the data set. Without knowing where data comes from, descriptive stats are just a snapshot. Executives that use data for decisions without the discipline of a framework risk making the same mistakes.^xxv

Organizations are awash in data and generate exponentially more every day. The harsh reality is that most of this data is irrelevant to decision making.

Unfortunately, most organizations don’t have the right data for making decisions. Starting with data leads to fitting decisions to data that is available. This imposes artificial constraints on the scope of the problem and reduces the efficacy of the decision process—if it doesn’t eliminate it entirely.^xxvi

Companies jump to data before they know what data a decision requires, says Pratt:

We see a lot of folks who start with a data science mindset. (They ask,) ‘Where's the data? We need to get the data clean before we can make the decision.’ Certainly, data science is one of the ingredients, but often people think it's the main ingredient. As they get through it, they realize, no, it's just one of several.”

DATA-DRIVEN DECISION-MAKING^xxvii	DECISION-DRIVEN DATA ANALYTICS
Anchor on available data.	Anchor on the critical decision.
Find a purpose for data.	Find data for your purpose.
Start from what is known.	Start from what is unknown.
Empower data scientists.	Empower decision makers.

The decision framework model helps determine what data to incorporate. A well-designed decision model identifies uncertainty and where to add data. When making decisions, it takes away arbitrary limitations based on available data, directs leaders to acquire the most valuable data, and saves valuable time and resources when acquiring, cleaning, and managing irrelevant datasets.^xxviii

Hyper-focus on data often crowds-out decision making, as Pratt explained:

Organizations want to be more data driven, so data science is where they start out. Then they realize that data driven is just a proxy for better business outcomes. Better business outcomes come from better business choices, and that's DI. Data is only one piece of that.”

Figure 7 Starting with the data may be seen as pragmatic but is misguided^xxix

DI requires good decision making and good decision-making processes. A framework is more important than more data when turning ML predictions into decisions, especially if automation is involved. ML can scale bad decisions just as effectively as it scales good decisions.^xxx