The Unfolding Frontier– the Data-Driven Economy

April 30, 2014

With the forthcoming release of the long-awaited report on "Big Data" from White House Counselor John Podesta, it’s easy to get lost in the politics and other distractions that come from any type of Washington report. Regardless of what the report ultimately says, it touches on the largest unfolding economic frontier this country (and the world) has discovered in generations—the data-driven economy. Regardless of who you are, what you do, and where you do it, no country, community, or industry will be left unchanged by the data-driven innovations that are happening today.

That’s why the Podesta report’s contents, as well as the the subsequent analyses and debates by the public and policymakers, is so important. If we choose to look at the future with fear and trepidation, rather than with hope and opportunity, we risk stifling the very innovations that could truly change all of our lives for the better. 

It’s important for the debates to focus on the value, reach, and impact of the data-driven economy and its components. Here are some facts that policymakers, business leaders, consumers, and everyone in between need to be aware of:

  • According to a recent McKinsey report, global data generation will increase by 40% a year. Nearly 80% of existing data are thought to be copies. Virtually all of this information is now in digital form.
  • Improved data usage could soon generate $3 trillion in additional value each year across the globe in just seven industries (education, transportation, consumer products, electricity, oil and gas, health care, and consumer finance). Of this, $1.3 trillion would benefit the United States annually. More than half of this value will come in the form of enhanced consumer surplus.
  • Information differs from other resources in fundamental ways. First, data can be used simultaneously by many people without diminishing their value to any party. Second, technology makes it easier to store, search, copy, and transmit data to multiple parties. Third, it is easy to translate digital data from one form into another.
  • Information systems are improving at a geometric rate. This will deliver dramatically better capabilities in the future. Companies need to anticipate these new capacities so that they can take advantage of them.
  • Cloud computing has increased the power of the information system. The ability to lease inexpensive storage and processing power makes it much easier for all companies to use data. Sophisticated data strategies no longer require large up-front capital costs and deep expertise in computer maintenance. Even the smallest companies now have access to the fastest servers and most sophisticated processing power at affordable rates. Usage can also be scaled up or down as needed, transforming a major fixed cost into a variable one.
  • With these breakthroughs, data will become less expensive, deeper, and more ubiquitous over time. It will be increasingly possible to design affordable systems that capture whatever data are useful, analyze them for meaning, transfer knowledge to where it is needed, and act on it, often without direct human involvement. Competitiveness will depend less on a company’s access to data and more on its ability to integrate the data into a business strategy.
  • Data enables us to solve a growing number of problems, including many that were previously unimagined. Companies that want to use Big Data to improve their internal operations will need to find the key issues holding them back. Companies seeking to develop new products or services need to identify unsolved problems among their customers. Some of the most valuable uses of data may not be identified until well after they are collected.

Another recent study by McKinsey[1] found that Big Data is:


  • Becoming a factor of production in every industry and business function;
  • Creating value by enabling transparency, experimentation, customization, decision-making through automated algorithms, and new business models and products;
  • Becoming a key basis for differentiating companies;
  • Creating new waves of productivity growth and consumer surplus;
  • Affecting all sectors, but some will benefit more than others;
  • Contributing to a large shortage of talent; and
  • Requiring progress on policy, technology and organizational issues.

McKinsey also found Big Data’s impact in a particular industry will be influenced by: 

  • The amount of data available within each firm;
  • The current variability in performance among firms;
  • The higher degree of customer and supplier intensity, which gives companies more ways in which to use data to properly segment markets;
  • The data intensity of each transaction; and
  • The frequency with which leaders and laggards exchange market leadership.
  • McKinsey’s report estimates a national shortage of 140,000 to 190,000 data experts in addition to 1.5 million managers and analysts. However, companies are likely to develop tools that make data more accessible to people with fewer skills.
  • Data will almost certainly result in greater transparency. Companies will know a lot more about their customers; customers will also know more about their companies. Transparency could lead to a more competitive environment in which margins are reduced. It could also produce stronger business relationships in which companies give customers exactly what they want, exactly when they want it.
  • Unfortunately, the sheer size of today’s data flows can present companies with a false sense of certainty. Common problems include biases in the data, an inability to balance the cost of inaction with overreliance, and undue confidence in the stability of data correlations.
  • Firms must deal with the challenge of legacy systems. Existing data are often stored in different formats and in different places within a firm, and new systems may be incompatible with older ones. Some of the most valuable uses of data require them to be shared between institutions, possibly even between competitors. These efforts likely require new understandings about permissible use and security.
  • Collecting data alone simply generates cost. Data need to be transformed into knowledge to be useful. The wise use of data often requires the same degree of business restructuring that accompanied previous efforts to improve quality programs, processes, and supply chains. Each centered on the better collection and use of data. Despite the innate appeal of these efforts, many companies struggled to implement them because they also necessitated significant institutional change.
  • Without common understanding, the benefits of data-driven innovation will be delayed. Yet too much government interference could also limit data’s promise. Policy issues that need to be resolved include standards, ownership, privacy protections, and security obligations.

Recent headlines, news stories, and debates have unfortunately made data a four-letter word. Sadly, in that process it has turned data into another such word: “fear.” In truth there is far too much good that it can avail everyone. Unfortunately, that potential has been lost in the larger narrative over the past few months. 

It is our hope that the Podesta report will take in the full promise and potential that the data-driven economy offers. As the debates continue to unfold about the future of data (as they should), it is important to recognize that there are facets and responsibilities that come with any breakthrough innovation. Those facets should be at the epicenter of decisions because if they are, we’re going to have a wider frontier of opportunity and potential that will benefit more people in imagined and unimagined ways. The future is always promising when that happens. 

By foreclosing on opportunity, as fear often does, we risk never allowing that future to arrive. 

[1] Big Data: The Next Frontier for Innovation, Competition, and Productivity.