Use the Right Data

Some of the most serious and prevalent problems that plague modern business result from using the wrong data to make decisions, measure outcomes, and incent performance. Recently, business journalist Geoff Colvin wrote, "In business as in life, be careful what you wish for. I know a company that wished for a better return on equity. What could be wrong with that? It paid its executives according to that measure, and man, did they deliver. In some years the firm had the best ROE in the industry. It was winning big time. The firm was Lehman Brothers, now dead because managing for ROE caused executives to overborrow…. Wishing for the wrong thing- managing for the wrong ratio- killed the company."

The cautionary tale of Lehman Brothers is just one among many to come out of the Great Recession. These days, smart business leaders are meticulously careful to use the right data.

We are obsessed by numbers and have become increasingly good at measuring all manner of things. The July-August 2009 issue of Harvard Business Review states, "Data, computing power, and mathematical models have been transforming many realms of management from art to science. But the crisis exposed the limitations of certain tools. In particular, the world saw the folly of reliance by banks, insurance companies, and others on financial models that assumed economic rationality, linearity, equilibrium, and bell-curve distribution. As the recession unfolded, it became clear that the models had failed badly."

The measurement tools and models are not themselves necessarily flawed. Business leaders simply need to become more adept at comprehending and using the data they generate. HBR argues, "…. decision makers in every industry must take responsibility for looking inside the black boxes that advanced quantitative tools often represent and understanding their functioning, assumptions, and limitations."

Consider the incredibly controversial issue of executive and, specifically, CEO compensation. Duke University business professor Dan Ariely points out that numerous studies demonstrate that people will behave based upon whatever measures we use to evaluate them. It seems too simple to contemplate but, says Ariely, "Human beings adjust behavior based on the metrics they're held up against. Anything you measure will impel a person to optimize his score on that metric. What you measure is what you get. Period."

Chief executives are overwhelmingly evaluated based on a single data point: the value of their company's stock. Even measuring CEOs against several years worth of stock returns does not necessarily incent them to consider the long-term health of the enterprise they lead: they are still obsessed by stock price. It is not surprising, therefore, that because they are compensated based on that one measure, most CEOs spend an inordinate amount of time considering and working towards an improved stock price.

Professor Ariely says, "To change CEOs' behavior, we need to change the numbers we measure. Stock value metrics that focus on the long term are a start, but even more important are new numbers that direct leaders' attention to the real drivers of sustainable success. What are those numbers? …. How many new jobs have been created at your firm? How strong is your pipeline of new patents? How satisfied are your customers? Your employees? What's the level of trust in your company and brand? How much carbon dioxide do you emit?"

Geoff Colvin asserts that businesses should evaluate performance using a new metric, called "EVA momentum." Economic value added, or EVA (a measure used by some companies) is essentially profit after charges for all the factors of production, and an improvement in EVA presumably results in increased value. Yet some business thinkers believe EVA can still be manipulated. EVA momentum is defined as the change in EVA divided by the prior period's sales and, the argument goes, simply cannot be tinkered with. Consultant Bennett Stewart says, "It's the only performance metric where more is always better than less. It always increases when managers do things that make economic sense."

Even at the level of macroeconomics and public policy we see much current discussion about the data that informs decision making. Nobel prize-winning economists Joseph Stiglitz and Amartya Sen produced a recent study that blames disproportionate focus on growth in the form of gross domestic product- the quantity of goods and services produced in the economy- for contributing to the world-wide recession. An unhealthy fixation on G.D.P. causes governments to overlook such problems as joblessness and environmental degradation, which are also important quantifiers of the overall health of the economy. Stiglitz says, "If you don't measure the right thing, you don't do the right thing," and he advocates for more attention on such benchmarks as income and consumption, availability of health care, and quality of education.

Temple University mathematics professor John Allen Paulos wrote an article recently in the New York Times Magazine called: "Metric Mania: Do we expect too much from our data?" Dr. Paulos says, "In the realm of public policy, we live in an age of numbers…. The problem isn't with statistical tests themselves but with what we do before and after we run them." He argues that measures in such areas as school performance and health care can be second-guessed, but that, "This doesn't mean we shouldn't be counting…. it does mean we should do so with as much care and wisdom as we can muster."

Albert Einstein supposedly said, "Not everything that can be counted counts, and not everything that counts can be counted." What measures do you use in your business to make decisions, assess performance, and reward behaviors? Are you careful and wise in your use of data, or do you rely on certain metrics just because you've "always done it that way"? The answers to these critical questions are essential to the future success of your business.