Cost-Benefit Analysis: Wonder Tool or Mirage?

Cost-Benefit Analysis: Wonder Tool or Mirage [i]? Which is it? At the highest levels, the US government maintains that cost-benefit analysis is indeed helpful to sort out potential good v. bad of "significant regulatory action." [ii]  September 17, 2003 OMB Circular A-4 "guidance to heads of executive agencies" says,

A good regulatory analysis is designed to inform the public and other parts of the Government (as well as the agency conducting the analysis) of the effects of alternative actions. Regulatory analysis sometimes will show that a proposed action is misguided, but it can also demonstrate that well-conceived actions are reasonable and justified.

Benefit-cost analysis is a primary tool for regulatory analysis[2]. Where all benefits and costs can be quantified and expressed in monetary units, benefit-cost analysis provides decision makers with a clear indication of the most efficient alternative, that is, the alternative that generates the largest net benefits to society (ignoring distributional effects). This is useful information for decision makers and the public to receive, even when economic efficiency is not the only or the overriding policy objective.

It is not always possible to express in monetary units all of the important benefits and costs. When it is not, the most efficient alternative will not necessarily be the one with the largest quantified and monetized net-benefit estimate. In such cases, you should exercise professional judgment in determining how important the non-quantified benefits or costs may be in the context of the overall analysis. If the non-quantified benefits and costs are likely to be important, you should carry out a "threshold" analysis to evaluate their significance. Threshold or "break-even" analysis answers the question, “How small could the value of the non-quantified benefits be (or how large would the value of the non-quantified benefits need to be) before the rule would yield zero net benefits." In addition to threshold analysis you should indicate, where possible, which non-quantified effects are most important and why. (p. 2, [2]: references E.J. Mishan’s book, Cost-Benefit Analysis, 1994)

There are many economists who believe cost-benefit analysis to be very helpful in government rulemaking and other regulatory efforts. But many others are not so sure that detailed cost-benefit analysis is either warranted or even helpful. Some argue that far from being helpful, such analysis is in fact often harmful to disadvantaged individuals and groups, and to the environment.

The latest manifestation of long-standing criticism of cost-benefit analysis, comes in the form a book by Frank Ackerman and Lisa Heinzerling titled Pricing the Priceless: Knowing the Price of Everything and the Value of Nothing, 2004.

In the fall of 2004, Carrie McLaren of Stay Free! Magazine interviewed Frank Ackerman and Lisa Heinzerling about Priceless: http://www.stayfreemagazine.org/archives/23/priceless.html

… Throughout Priceless, Ackerman and Heinzerling show how regulators have determined costs for anything from a case of chronic bronchitis ($260,000; EPA, 1997), to the preservation of national forests ($219,000; OMB, 2002), to IQ points ($8,346 each; EPA, 2000).

The authors oppose cost-benefit analysis and consider it too inherently biased to base decisions on. Yet in their critique they remain level-headed and clear, pointing out blatant errors in logic and calculations--criticisms that should be of value even to cost-benefit's advocates. …

Frank Ackerman is an economist at the Global Development and Environment Institute at Tufts University. Lisa Heinzerling is a professor at the Georgetown University Law Center and has clerked for Judge Richard Posner and Justice William Brennan. We talked by phone in July 2004, and I was immediately taken with both of them.--Carrie McLaren ….

STAY FREE!: Are there any critics of cost-benefit analysis who don't share your politics? Any conservatives oppose it?

ACKERMAN: Conservative policy wonks generally love this stuff.

STAY FREE!: How do mainstream environmental groups respond to cost-benefit analysis? Are they responding with their own numbers: "if you can't beat 'em, join 'em"?

HEINZERLING: Good question. Certainly when Newt Gingrich's Contract with America came to Washington, environmental groups united against a so-called "super mandate" that would have required cost-benefit analysis for health and environmental rules. And so you have that fairly recent experience when the environmental community was united against cost-benefit analysis. Today I hear among environmental groups some sense that, "well, this is inevitable, so let's try to make it as good as we can."

STAY FREE!: What about "bioeconomics"? I've read that some environmentalists have been arguing for putting a price on natural resources in order to deter corporations from plundering the environment without paying. What has become of this idea?

ACKERMAN: There have been little successes here and there, but the problem is that the market values of sustainable uses of natural resources are often much less than the values of damaging uses. What happens in, say, the Amazon, where it turns out that the value of preserving the trees for tourism and sustainable industries is nowhere near the short-term value of clear-cutting, selling the timber, and farming the land? To take another example, if you wanted to put a price on whales, you could add up the money people pay for whale-watching trips, but it turns out that's fairly small. It's easy to believe that commercial exploitation of whales could produce a lot more money than the revenues of whale-watching trips.

My reaction to those who want to save nature by adding up its market value is, more power to them, but they're not going to get us nearly as far as we need to go.

STAY FREE!: You would prefer to get rid of cost-benefit analysis entirely, wouldn't you?

HEINZERLING: We think it's fundamentally flawed and that refinements aren't going to help in a meaningful way.

ACKERMAN: The people who are pushing it are not just relying on its intrinsic flaws; they're often cheating in their calculations. Environmental groups fighting a rear-guard action against cost-benefit analysis can always find ways in which the benefits numbers are too small, but that never wins the war.

STAY FREE!: It reminds me of the divide between police estimates of the crowd size at a protest verses the organizers' estimates. The two sides are never going to agree on the numbers, so it boils down to politics. To play devil's advocate, proponents of cost-benefit analysis argue that obviously some ways of preserving our environment or our health are cheaper and better than others, so can't cost-benefit analysis help with that?

HEINZERLING: It doesn't necessarily help with that. What might help is setting a goal and then thinking about creative ways to get to that goal most cheaply. In some contexts, that might mean labeling a product rather than banning or restricting it. In other cases, when you're talking about pollution, it might mean allowing emissions trading in that pollutant rather than requiring a particular control technology.

ACKERMAN: From the beginning of modern environmental regulation in the early 1970s right to the present, there has been continuous discussion about the best, most innovative ways to regulate--a search for cheaper control technologies, simpler forms of record keeping and so on. I'm not convinced that cost-benefit analysis does anything to accelerate that process. The case for cost-benefit calculations so often depends on a strange rewriting of the past, as if the EPA was once run by Stalinist bureaucrats who delighted in capriciously spending money, and so now we have to bring in economic analysis to undo the damage. If you were alive then, or if you've read about the period, you know that this legendary era of extravagance never happened. So cost-benefit analysis is presented as solving a desperate problem that never actually occurred.

STAY FREE!: How do you respond to economists who argue that cost-benefit analysis helps in making difficult decisions? If closing off a particular waterway is going to harm a couple of types of fish but will protect a couple of other species, or it will devastate the livelihood of one community but not another, can't cost-benefit analysis help with making the best choice?

ACKERMAN: Making difficult decisions is what government and the courts have always done, and there's no evidence that they have failed for lack of a magical mathematical formula.

HEINZERLING: One of the subtle things that's also lost when you decide things according to this formula is any sense of loss or tragedy. Martha Nussbaum, a philosopher at the University of Chicago, has pointed out that if you reduce everything to numbers you can easily think nobody was hurt by a regulation. It may be, in some cases, that you can't please everybody and at the end of the day somebody is hurt, but cost-benefit analysis completely papers over that fact. Sometimes in human situations when you realize that people are being hurt you can actually come up with a solution that you wouldn't have thought of if you were pretending that you were just trading money around.

STAY FREE!: So do you think cost-benefit analysis is going to be around for a while?

HEINZERLING: Yes, unfortunately. There are too many people who make their living off of it [laughs]. There are too many think tanks. It's a huge industry.

If you enjoyed these remarks, you might also like this from OMB Watch, 3/21/2005, titled "Is Cost Benefit Analysis Needed?" The article concludes,

[C]ost-benefit analysis is not only a weak tool for determining public protections, but its "impartial" calculations can have severe and damaging impacts. In no way is it a blind arbitrator, equally weighing both sides of an issue. Rather it is a political tool, weighted to favor the regulated community that does not adequately address our regulatory priorities.

Finally, I have carried similar criticisms forward in social science circles and broader circles in the Forest Service for many years. My latest criticism is packaged as "Top Ten Reasons why Cost Benefit Analysis Fails in Public Choice Settings."

Endnotes

[i] "Cost-Benefit Analysis: Wonder Tool or Mirage?" titled borrowed from Dec. 1980 Congressional Report, cited in my late-1980s "Economic Advice for Forest Managers."

[ii] OMB A-4 Note: Significant regulatory action is defined by Executive Order 12866.   OMB Circular A-4 pertains in particular to two sections of EO 12866: Section 6(a)(3)(c) and Section 3(f)(1), but also references other authorities.

Comments

[Tracy W]

Out of curiousity, what would you substitute for CBA as a guide to policy-making?

[Dave Iverson]

Tracy W,

Personally, I'd use Herbert Simon and James March's work on bounded rationality. Begin with March's "A Primer on Decision Making: How Decisions Happen." I'd compliment it with work from Karl Weick on "Making Sense of the Organization" and Weick with Kathleen Sutcliffe on "Managing the Unexpected." I'd compliment it with works from Margaret Jane Radin and Elizabeth Anderson on understanding value in pluralistic frames. See Anderson's "Value in Ethics and Economics." See Radin's "Contested Commodities." I would have people begin to understand the power of leadership as Ronald Heifetz and others teach it -- i.e. "Leadership Without easy Answers."

There are no quick fixes available to those who frame things more broadly than allowed in the narrow confines of monetized CBA. Finance and economics play important parts in decision making even when such are viewed as but one aspect of a broader whole. But we pluralists believe that practitioners ought not to make the "category mistake" of thinking all can be reduced to the narrowly framed rationality often posited by practitioners of CBA.

[Dave Iverson]

People who want to wade through fewer books, to begin with than identified in my previous comment, might start a journey into political decision making with Deborah Stone's "Policy Paradox: The Art of Political Decison Making."

[Tracy W]

I am stuck with a relatively small available library and expensive delivery fees from Amazon (I hate that part about living in NZ). What are the ideas in those books you'd use to inform policy-making? Could you pretty please give me a quick summary?

[Tracy W]

I mean summary of the names of the ideas in those books, I might well recognise them once told.

[Dave Iverson]

Tracy,

I'm away on business this week, but will try to develop some of the ideas more fully in the weeks and months to come. They all come from political science, organizational psychology and philosophical reasoning far from the narrow confines of economics.

Another important book:

Robert Jervis' "System Effects: Complexity in Political and Social Life"