Managing Expectations With Big Data




A commonly touted virtue of big data is that it enables better decision-making by providing tools for more efficient, informed choices about how to operate a business, government agency, or even a political campaign. The reason is obvious: a necessary (though not sufficient) condition for good decision-making is having access to relevant information, and big data provides larger and more diverse troves of information than would otherwise be available, whether about consumers, voters, industrial processes, or real-time operations.

But big data can start to look like a bad investment when it fails to deliver on outsized expectations. When we hold big data-related tools up to an impossibly high standard of certainty or success, disappointment with those tools becomes inevitable; the temptation grows to throw out the proverbial baby with the bath water. But the problem is with our expectations, not the new tools.

Consider an analogy. In our day-to-day lives, the choices we make depend on information about the things around us, such as the weather, our neighbors, family, and co-workers. There are two noteworthy things about our choices in such contexts.

First, it is clear that, thanks to the limitations of time, energy, and our senses, the information to which we have access is either imperfect or incomplete. We can never be certain what the weather will be like or how somebody will react to something. We rarely, if ever, have all the information.

Second, our choices tend nevertheless to be successful—they usually work. Whatever information we do have is good enough for most of us, most of the time. From this point of view, one could not hope for a more reliable “data set” than what is presented in ordinary life; however imperfect, it is sufficient for practical purposes.

In everyday life, we rarely, if ever, have complete certainty, but that does not prevent us from acting or doing so successfully most of the time. When we do make bad choices, we tend not to lose faith in our ability to acquire information as such (whether from our senses, the testimony of others, etc.). Instead, we try to learn from our mistakes and make better decisions next time round, given the information at our disposal. This is because on some level we assume (correctly) that we could always make the wrong choice, either because of human error, or imperfect information.

Managers, entrepreneurs, and CEOs are in similar positions. They make decisions on a daily basis, and these require information about and have bearings on things far beyond the scope of what is usually presented to us in ordinary life—such as concerning the quality and quantity of employees or massive industrial machinery. Ideally, knowledge gleaned from large and diverse data sets—big data—gives us more and better information on these kinds of things and events, about which we previously had only misleading, partial, or simply no information.

There is, however, a temptation to view such new tools as providing, over and above this, a magical key to infallible decision-making. But really what big data tools provide are the technological means to extend our human faculties into new realms. They are not the technological means to make our human faculties superhuman.

It would be strange if more and better information about such unwieldy things as the efficiency of large industrial systems and consumer or voter preferences could provide us with more certainty than is available to us in ordinary life. If this seems implausible, consider another analogy.

The microscope allowed (and continues to allow) scientists to extend their powers of observations into hitherto invisible domains of reality. It did not thereby make their powers of observations perfect or unimpeachable. However fruitful, the scientist’s perception of microscopic things is no more unimpeachable than our ordinary perception, insofar as both depend on human interpretation and potentially false or misleading data.

Scientific knowledge may—or may not be—more precise, possibly even more reliable than common sense knowledge. But in any case, both involve human, and therefore imperfect capacities. And both involve human, and therefore, fallible choices that rely on intuition, instinct, and even guesswork. The good news is that these capacities and these choices often do succeed.

The same holds for big data. Such new technologies add to our informational tool kit, but no single technology should be held to standards exceeding what is possible in the secular realm. In ordinary life, as in scientific and business contexts, perfect information is impossible; there is no magical key to perfect decision-making, since decision-making is ultimately a human affair.