How Big Data Can Improve Health Care—If We Let It

May 22, 2014

The growing ease with which even the hidden data can be collected stored and analyzed will transform many industries. Perhaps the most important of these is health care. This sector already accounts for around 15 percent of the nation’s economy. Rapidly rising costs and an aging population will cause medical expenses to continue to rise and threaten the nation’s fiscal health. Yet, despite high costs, America’s health care system has not delivered better performance.

The U.S. health care system is largely determined by government policy. States license medical professionals and determine their permissible scopes of practice. The Food and Drug Administration carefully screens drugs and medical devices. The federal government sets out coverage policies and reimbursement rates for its two main programs, Medicare and Medicaid. Because of these programs’ size, the private sector often follows their lead.

In the future, medical care will increasingly be focused around information. With the help of medical apps, patients will often diagnose themselves. The Internet will give them a separate and sometimes more accurate source of information in addition to their doctor. Software will allow attendants and nurse practitioners to perform work that now requires a doctor or specialist. Artificial intelligence systems will help providers make the correct diagnosis faster. And patients’ care will increasingly be informed by both vast databases of other cases and by their own individual histories.

A McKinsey report recently estimated that better use of data could reduce U.S. health care costs by $300 to $450 billion per year or 12-17 percent of current costs. This transformation is already occurring. It found 200 companies created since 2010 to develop new tools to analyze existing health care data. Uncertainty over its speed and benefits depends less on continued advances in science and information technology and more on the degree to which existing institutions accommodate or impede it.

Both the speed and cost of genetic sequencing are falling rapidly. The specific makeup of each individual, as well as of the particular viruses and bacteria that inhabit it, is now accessible. Partial genetic testing is now routine for patients thought to have specific conditions. Soon everyone will have their full genome decoded.

When combined with electronic medical records, this will lead to the construction of massive data bases that allow doctors to search out the interrelationships between specific genetic variations and individual drugs and outcomes. Hospitals that use tools provided by EPIC, an electronic medical records provider, can access the benchmark and reference information from the records of all other Epic customers. The vast size of the data bases will allow researchers to find correlations in much smaller groups of patients. The result will be truly individualized medicine.

The rise of online information centers like and social groups like allows patients to do their own research about diseases and connect with similarly situated people. makes it possible for patients to combine medical information confidentially in an effort to encourage and speed research efforts focused on their disease. The ability to digitize information also facilitates telemedicine, breaking down regulatory jurisdictions and reducing the cost of getting specialized medicine to rural areas.

A large host of devices exists to collect health related information about each individual. Wearable technology such as Fitbit, continuously collects data on heart rate, blood pressure, exercise, and sleep patterns. helps hospitals track specific high risk patients by monitoring whether they have taken their medicine, experienced sudden weight gain, or changed their routines. More sophisticated devices already exist that would allow individuals to check for ear infections, take an electrocardiogram at home, or write their own eyeglass prescription with the help of their iPhone.

Artificial intelligence will increasingly help professionals make sense of all this information. IBM’s Watson supercomputer is helping to make diagnoses by keeping track of the growing volume of medical literature. And information on specific providers will help patients find the best care. The large amount of provider information recently released by the government has generated numerous stories about the disparity between individual doctors. helps patients find and compare doctors.  

But these gains will require significant restructuring of the medical industry. Dr. Eric Topol has argued that:

Medicine is remarkably conservative to the point of being properly classified as sclerotic, even ossified. Beyond the reluctance and resistance of physicians to change, the life sciences industry … and government regulatory agencies are in a near paralyzed state, unable to break out of a broken model of how their products are developed or commercially approved.

Peter Huber has echoed these thoughts: “[t]he system is currently frozen in the headlights. It can’t handle the complexity and torrents of data that now propel the advance of molecular medicine. It isn’t just lagging behind the new science; it has fallen so far behind that it now obstructs it at almost every turn.”

Whether this changes depends on the willingness of individuals to demand access to new technology, the ability to draft new legislation and regulations, and a decision to put more of the decision making and costs in the hands of the patients themselves.