Regulatory Moneyball, Values and Cost-Benefit Analysis

Cass Sunstein has been whetting appetites for his new book, Simpler: the Future of Government (out April 9) by doling out tasty samples of its content (see e.g. here and here). One of these is an essay for Foreign Affairs, "Regulatory Moneyball". The title is borrowed from Michael Lewis' superb Moneyball: the Art of Winning in an Unfair Game, about Billy Beane's use of statistics to get the jump on baseball clubs who relied on hunches and instinct.

In this essay, Sunstein deals with one of the criticisms of cost-benefit analysis: that decisions to regulate cannot be reduced to a balance sheet of monetized costs and benefits; values are inevitable. Here is his response:

Some might object that debates about regulation are really about values, not facts. According to this view, when people disagree about a rule that would protect clean air or increase highway safety, it is because of what they most value, not because of disagreements about the evidence. On some of the largest issues, values and predispositions do play a critical role. At the same time, it is easy to overstate the point. For example, most people’s values do not lead to a clear judgment about whether to require rearview cameras in cars. Values alone cannot guide the decision about whether to reduce levels of ozone in the ambient air from 75 parts per billion to 70 parts per billion or, for that matter, to 20 parts per billion. To evaluate such proposals, factual evidence is indispensable.

Now, no one could disagree that where costs and benefits are identifiable and available they should play a part in administrative decision-making. And it is true that values alone cannot guide regulatory decisions. But values are hugely important. Their role cannot be downplayed and we should be alive to the risk that they could be concealed by resort to cost-benefit analysis.

To play with Sunstein's example, who are the beneficiaries of the decision to reduce ozone levels? People in the territory of the regulator (say, the United States, for the Environmental Protection Agency), or people on a broader scale, continental or international? Are the beneficiaries currently living, or should we also account for future generations? In terms of costs, how do we calculate them: should historic advantages that individuals and enterprises have reaped from laxer controls be weighed in the balance? And what costs should we consider: does harm to wildlife count? Reducing regulation to a dollar figure only conceals these very important, values-based choices.

Sunstein recognizes a similar difficulty when discussing how to monetize human lives:

Suppose that across large populations, workers who are subject to a mortality risk of one in 100,000 generally receive a wage bonus, or premium, of $90. That would suggest a “value of a statistical life” of $9 million. Relying on evidence from the labor market, then, the federal government would spend no more and no less than $90 per individual to eliminate that risk.

But these simple approaches would leave many questions unanswered. For example, should children’s lives be valued more, less, or the same as the lives of adults? More precisely, how should the government treat statistical risks faced by children? Parents would be willing to spend a lot to reduce risks to their children; shouldn’t their wishes count? The other end of the age spectrum raises similar questions. Suppose that a rule would mostly extend the lives of the elderly by a short time: for example, an air-pollution rule whose main effect would be to add a few months to the lives of people over the age of 80. Should agencies give a lower value to the lives of old people because the effect might prolong lives by only a matter of months? Would those few extra months justify a high cost if the same amount could extend a younger person’s life by decades? Or is that difference irrelevant?

Sunstein's answer to this question appears to be (we will have to wait for the book to be sure!) along the following lines:

It is perfectly appropriate for agencies to take unquantifiable factors into account. But excessive regulation is a genuine concern, and agencies should not use their authority to consider qualitative factors as a license to do whatever they like. While I was at OIRA, the Obama administration took a number of steps to ensure a disciplined approach. The first step was to promote accountability by recommending that all significant regulations be accompanied by a simple table that offered three things: first, a clear statement of both the quantitative and the qualitative costs and benefits of the proposed or final action; second, a presentation of any uncertainties; and third, similar information for reasonable alternatives to the action. In a related step, OIRA required agencies to include a clear, simple executive summary of any new rules, explaining what they were doing and why and offering a crisp account of the costs and benefits, both quantitative and qualitative. Many federal rules are extremely long and complex, and it is hard for people to know what they are trying to do and why. A clear summary can help a great deal.

Is this really an answer to the problem I identified earlier? Sunstein would have these factors considered separately, but the problem is that they infuse the very identification of the relevant costs and benefits. A "disciplined approach" should be applied across the board, to quantifiable factors as well as the unquantifiable. Otherwise, regulatory moneyball risks relying too much on hunches and guesses.  If these are hidden, we are much worse off.

Indeed, Sunstein's response might raise another problem. If both quantitative and qualitative costs and benefits have to be included as a matter of course, is this really purely stats-based "Regulatory Moneyball" at all? Just because hunches and instincts are written down doesn't make them any less hunches and guesses.

Of course, one can draw a lesson like this from the original Moneyball. Beane experienced a lack of success in play-off games. The law of large numbers gave him a statistical edge over a whole season. But in a condensed space, "big-game" qualities became important, the very hunches and guesses about players' characteristics that Beane derided. The lesson from Moneyball may then be that numbers are important and a useful way of cabining "softer" factors, but not the be all and end all. Numbers alone should not dictate every action of a decision-maker. It will be interesting to see what conclusion Sunstein arrives at in Simpler, which I am very much looking forward to reading.

P.S. On the relationship between financial numbers and politics, this recent exit interview given by Canada's Parliamentary Budget Officer, Kevin Page may be of interest.