Myths of Measurement

December 19, 2012

BCLC’s fall events -- the Global Conference and the Citizens Awards -- were an inspiration. Personally, I couldn’t help but be inspired by so many companies tackling the world’s problems so thoughtfully. Interestingly, among the many topics companies cover, I noticed a common thread: measurement is sorely needed in CSR, and there is a lot of ambiguity about the best way to measure success.

As a self-described “data nerd,” many of these issues are quite familiar to me and I thought some advice might be helpful.

In particular, I think there are three common “myths” that impede good measurement of corporate responsibility initiatives. These aren’t the only issues (by far), but they do get in the way of attempts to measure impact. If we can get past these three myths, I think measuring impact would be a great deal easier.

Myth #1: Certain things just can’t be measured.

This common wisdom is found throughout the business community, not just in CSR. The idea shows up in many different guises – one often hears about “intangible effects” and “incalculable benefits.”

Certainly many phenomena are complex and difficult to measure – yet everything can be measured. In all of my experience gathering quantitative and qualitative data, I’ve found that there is no human interaction so irrational or elusive that it couldn’t be measured. Historically, researchers have created all manner of innovative methods to explore difficult, “soft” concepts. For instance, in 1966 Eugene Webb pioneered unobtrusive measures to “see” social interactions without interfering with them. No matter if it’s an idea, an action, an interaction, or an artifact – if it exists, someone can find a credible way to track it.

I’ve found that there is no human interaction so irrational or elusive that it couldn’t be measured.

Myth #2: Certain things can’t be quantified.

Another common business myth is that certain effects can’t be translated into numbers. This idea is often bandied about when emotional or personal information is the object of research. Like Myth #1, it shows up in different guises. One hears of social value that cannot be translated into a monetary value, or social impacts that go “beyond the numbers.”

The problem with this thinking is that anything can be translated into numbers. Quantification is just translation from one language into another.  Math is a particularly logical and strict language, but like any language it can express varied concepts. So while it’s not always easy to translate a phenomenon into numbers, there is always some way to do it.

When professionals realize this, it takes away the fear of applying numbers to social impact, and at the same time it frees us to be analytical about this topic. When you start to measure “soft” concepts like social impact using very precise methods, then they become increasingly concrete. And when they become concrete, it becomes easier to justify your work on these issues, and to understand exactly how to implement the best programs to tackle them.

When you realize that quantification is just translation from one language to another, it takes away the fear of applying numbers to social impact.

Myth #3: New measurement requires an expert technique

The final common myth is that a measurement problem is likely a technical problem. Many people think that solving a measurement issue requires a statistician or a scientist. Frankly, these professionals will accelerate the process, but rarely does a measurement problem absolutely require them. Almost every measurement problem can be taken apart by an intelligent person, and every measurement technique rests on straightforward, logical principles.

With very few exceptions, any smart person should be able to fundamentally understand a measurement issue after a couple days of active research. And yet, I think the common assumption is that these problems can only be solved after years of training in math, economics, or statistics. Like any issue, a measurement issue starts by a person breaking it down into its constituent parts, and that process holds whether you are a Research Manager with a PhD in statistics, or you’re an intern just starting out.

Like any issue, a measurement issue starts by a person breaking it down into parts.

Beyond the Myths

Getting past these three myths might help the field of corporate responsibility to move forward. When individuals realize that measurement doesn’t require a PhD to figure it out, maybe many unmeasured practices will be better tracked and analyzed. When professionals feel less intimidated by quantification, maybe more of them will realize that numbers can provide useful specificity, even when measuring “intangible” phenomena like thoughts and feelings.

Overall, what I think the field needs to realize is that measurement of social impact is completely attainable – and it can even become routine. Great new strategies are waiting to be discovered, and all it takes is for the profession to conduct new analyses to uncover them.