« Fierce Conversations | Main | Roadless: A Rule to Sue By »

May 05, 2005

Cost-Benefit Analysis: Wonder Tool or Mirage?

Cost-Benefit Analysis: Wonder Tool or Mirage [i]? Which is it? At the highest levels, the US government maintains that cost-benefit analysis is indeed helpful to sort out potential good v. bad of "significant regulatory action." [ii]  September 17, 2003 OMB Circular A-4 "guidance to heads of executive agencies" says,
A good regulatory analysis is designed to inform the public and other parts of the Government (as well as the agency conducting the analysis) of the effects of alternative actions. Regulatory analysis sometimes will show that a proposed action is misguided, but it can also demonstrate that well-conceived actions are reasonable and justified.

Benefit-cost analysis is a primary tool for regulatory analysis[2]. Where all benefits and costs can be quantified and expressed in monetary units, benefit-cost analysis provides decision makers with a clear indication of the most efficient alternative, that is, the alternative that generates the largest net benefits to society (ignoring distributional effects). This is useful information for decision makers and the public to receive, even when economic efficiency is not the only or the overriding policy objective.

It is not always possible to express in monetary units all of the important benefits and costs. When it is not, the most efficient alternative will not necessarily be the one with the largest quantified and monetized net-benefit estimate. In such cases, you should exercise professional judgment in determining how important the non-quantified benefits or costs may be in the context of the overall analysis. If the non-quantified benefits and costs are likely to be important, you should carry out a "threshold" analysis to evaluate their significance. Threshold or "break-even" analysis answers the question, “How small could the value of the non-quantified benefits be (or how large would the value of the non-quantified benefits need to be) before the rule would yield zero net benefits." In addition to threshold analysis you should indicate, where possible, which non-quantified effects are most important and why. (p. 2, [2]: references E.J. Mishan’s book, Cost-Benefit Analysis, 1994)

There are many economists who believe cost-benefit analysis to be very helpful in government rulemaking and other regulatory efforts. But many others are not so sure that detailed cost-benefit analysis is either warranted or even helpful. Some argue that far from being helpful, such analysis is in fact often harmful to disadvantaged individuals and groups, and to the environment.

The latest manifestation of long-standing criticism of cost-benefit analysis, comes in the form a book by Frank Ackerman and Lisa Heinzerling titled Pricing the Priceless: Knowing the Price of Everything and the Value of Nothing, 2004. In the fall of 2004, Carrie McLaren of Stay Free! Magazine interviewed Frank Ackerman and Lisa Heinzerling about Priceless: http://www.stayfreemagazine.org/archives/23/priceless.html


… Throughout Priceless, Ackerman and Heinzerling show how regulators have determined costs for anything from a case of chronic bronchitis ($260,000; EPA, 1997), to the preservation of national forests ($219,000; OMB, 2002), to IQ points ($8,346 each; EPA, 2000).

The authors oppose cost-benefit analysis and consider it too inherently biased to base decisions on. Yet in their critique they remain level-headed and clear, pointing out blatant errors in logic and calculations--criticisms that should be of value even to cost-benefit's advocates. …

Frank Ackerman is an economist at the Global Development and Environment Institute at Tufts University. Lisa Heinzerling is a professor at the Georgetown University Law Center and has clerked for Judge Richard Posner and Justice William Brennan. We talked by phone in July 2004, and I was immediately taken with both of them.--Carrie McLaren ….

STAY FREE!: Are there any critics of cost-benefit analysis who don't share your politics? Any conservatives oppose it?

ACKERMAN: Conservative policy wonks generally love this stuff.

STAY FREE!: How do mainstream environmental groups respond to cost-benefit analysis? Are they responding with their own numbers: "if you can't beat 'em, join 'em"?

HEINZERLING: Good question. Certainly when Newt Gingrich's Contract with America came to Washington, environmental groups united against a so-called "super mandate" that would have required cost-benefit analysis for health and environmental rules. And so you have that fairly recent experience when the environmental community was united against cost-benefit analysis. Today I hear among environmental groups some sense that, "well, this is inevitable, so let's try to make it as good as we can."

STAY FREE!: What about "bioeconomics"? I've read that some environmentalists have been arguing for putting a price on natural resources in order to deter corporations from plundering the environment without paying. What has become of this idea?

ACKERMAN: There have been little successes here and there, but the problem is that the market values of sustainable uses of natural resources are often much less than the values of damaging uses. What happens in, say, the Amazon, where it turns out that the value of preserving the trees for tourism and sustainable industries is nowhere near the short-term value of clear-cutting, selling the timber, and farming the land? To take another example, if you wanted to put a price on whales, you could add up the money people pay for whale-watching trips, but it turns out that's fairly small. It's easy to believe that commercial exploitation of whales could produce a lot more money than the revenues of whale-watching trips.

My reaction to those who want to save nature by adding up its market value is, more power to them, but they're not going to get us nearly as far as we need to go.

STAY FREE!: You would prefer to get rid of cost-benefit analysis entirely, wouldn't you?

HEINZERLING: We think it's fundamentally flawed and that refinements aren't going to help in a meaningful way.

ACKERMAN: The people who are pushing it are not just relying on its intrinsic flaws; they're often cheating in their calculations. Environmental groups fighting a rear-guard action against cost-benefit analysis can always find ways in which the benefits numbers are too small, but that never wins the war.

STAY FREE!: It reminds me of the divide between police estimates of the crowd size at a protest verses the organizers' estimates. The two sides are never going to agree on the numbers, so it boils down to politics. To play devil's advocate, proponents of cost-benefit analysis argue that obviously some ways of preserving our environment or our health are cheaper and better than others, so can't cost-benefit analysis help with that?

HEINZERLING: It doesn't necessarily help with that. What might help is setting a goal and then thinking about creative ways to get to that goal most cheaply. In some contexts, that might mean labeling a product rather than banning or restricting it. In other cases, when you're talking about pollution, it might mean allowing emissions trading in that pollutant rather than requiring a particular control technology.

ACKERMAN: From the beginning of modern environmental regulation in the early 1970s right to the present, there has been continuous discussion about the best, most innovative ways to regulate--a search for cheaper control technologies, simpler forms of record keeping and so on. I'm not convinced that cost-benefit analysis does anything to accelerate that process. The case for cost-benefit calculations so often depends on a strange rewriting of the past, as if the EPA was once run by Stalinist bureaucrats who delighted in capriciously spending money, and so now we have to bring in economic analysis to undo the damage. If you were alive then, or if you've read about the period, you know that this legendary era of extravagance never happened. So cost-benefit analysis is presented as solving a desperate problem that never actually occurred.

STAY FREE!: How do you respond to economists who argue that cost-benefit analysis helps in making difficult decisions? If closing off a particular waterway is going to harm a couple of types of fish but will protect a couple of other species, or it will devastate the livelihood of one community but not another, can't cost-benefit analysis help with making the best choice?

ACKERMAN: Making difficult decisions is what government and the courts have always done, and there's no evidence that they have failed for lack of a magical mathematical formula.

HEINZERLING: One of the subtle things that's also lost when you decide things according to this formula is any sense of loss or tragedy. Martha Nussbaum, a philosopher at the University of Chicago, has pointed out that if you reduce everything to numbers you can easily think nobody was hurt by a regulation. It may be, in some cases, that you can't please everybody and at the end of the day somebody is hurt, but cost-benefit analysis completely papers over that fact. Sometimes in human situations when you realize that people are being hurt you can actually come up with a solution that you wouldn't have thought of if you were pretending that you were just trading money around.

STAY FREE!: So do you think cost-benefit analysis is going to be around for a while?

HEINZERLING: Yes, unfortunately. There are too many people who make their living off of it [laughs]. There are too many think tanks. It's a huge industry.

If you enjoyed these remarks, you might also like this from OMB Watch, 3/21/2005, titled "Is Cost Benefit Analysis Needed?" The article concludes,

[C]ost-benefit analysis is not only a weak tool for determining public protections, but its "impartial" calculations can have severe and damaging impacts. In no way is it a blind arbitrator, equally weighing both sides of an issue. Rather it is a political tool, weighted to favor the regulated community that does not adequately address our regulatory priorities.
Finally, I have carried similar criticisms forward in social science circles and broader circles in the Forest Service for many years. My latest criticism is packaged as "Top Ten Reasons why Cost Benefit Analysis Fails in Public Choice Settings."

[i] "Cost-Benefit Analysis: Wonder Tool or Mirage?" titled borrowed from Dec. 1980 Congressional Report, cited in my late-1980s "Economic Advice for Forest Managers."
[ii] OMB A-4 Note: Significant regulatory action is defined by Executive Order 12866.   OMB Circular A-4 pertains in particular to two sections of EO 12866: Section 6(a)(3)(c) and Section 3(f)(1), but also references other authorities.

Posted by Dave on May 5, 2005 at 04:30 PM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Cost-Benefit Analysis: Wonder Tool or Mirage?:


Posted by: Tom Mitchell

This note deals with the comment on cost-benefit analysis by Mr. Iverson on May 5, 2005.

I would like to tie Dave's comment to the set of comments dealing with analysis to support Forest Planning that Mr. Rupe and I have been passing back and forth.

I remember taking a course where we dealt with developing cost-benefit equations. Each equation became longer and longer as the class attempted to take into consideration more and more items - some of which had no market and therefore no agreed upon "value" - and include them as benefits and/or taking "negative" benefits (like pollution) and adding them to costs. It became a real game with the winner being the most creative in their ability to quantify and then argue for the things they quantified. Training to be a lawyer helps in these situations - making a case for why your guilty client is really innocent. An interesting game, but many of us weren't sure how useful it was for guiding management of natural resources.

In the early phase of implementing the 1979/1982 planning regulations, after it was decided that the major analysis for planning would be linear programming, one of the Forest Service economists, Charlie Palmer, took a look at the "objective function" of these models. For the most part, the "final" objective function was maximization of Present Net Value ... which is very similar to maximizing the Benefit/Cost ratio. For commodities such as timber and range, there were "market" prices that could be used. For some "products", like a day's worth of hiking, you could, perhaps, identify a shadow price. But it was clear that developing shadow prices for all goods and services including protection of the productive capability of soil and water were difficult to impossible for most in the Forest Service.

Based upon this, Charlie, working with others, worked to establish "cost efficiency" rather than economic efficiency as the requirement for planning. Seems a small difference, but really not. Economic efficiency, much like Cost/Benefit analysis, relies on putting prices on everything; in the case of using LP that was dealt with by having an objective function of maximizing present net value.

On the other hand, cost efficiency means, for those things that we can reasonably value such as timber and range, we will use prices - so that the timber harvest schedule and intensity of management (e.g., thin or no thin) would be decided based upon the discounted value of the timber produced versus the costs to produce it. For other things like providing for a quality wilderness experience or habitat for wildlife that can be viewed by visitors, enhancing their enjoyment, analysis would be what was termed cost effective analysis. That is, you don't place a value on the output, instead, you try to find the way to achieve that output in the least cost manner. (Sortuv as you jump out of an airplane, you remember that the company that got the contract for the parachutes was the lowest bidder.)

In this way, forests doing their plans, didn't have to try to place a dollar value on all that they thought should be produced. They could include in the plan ... including practices with real dollar costs ... those things that could not be easily valued and use analysis to find the least cost way of achieving this.

Based on Charlie's work, cost efficiency replaced economic efficiency in forest planning and in that way, attempted to deal with the vagaries of Cost/Benefit analysis when dealing with Congress, et. al.

To some of us, this was a good stroke of business. But then it became obvious, at least to some, that this type of thing was difficult to nearly impossible to represent well in the form of "constraints" within a linear programming model.

At this point, an economic concept introduced by Kornoi, an economist who worked in a non-market economy, was used by some of us. That was, that the duty of a planner/analyst is to set before the decisioin maker only those alternatives which were "superior" to others... again for those who didn't read my earlier comment a "superior" alternative is one that produces the same outputs at less cost or one that produces more at the same cost.

This concept was incorporated in Forest Planning through the use of what were defined as "goals" in the earlier regulations and now defined as "forest conditions". That is, specifying the forest conditions that were desired and using these in the forest plan as the targets for all planned actions as well as within the linear programming model used for analysis of the final plan. "Desired" here included forest conditions that provided habitat for all native wildlife species, that provided for protection of soil and water resources, that maintained aquatic and riparian ecosystems, and that produced mixes of products including timber, range, all types of forest recreation, and visual quality.

These forest conditions were used to make sure that all of the legal minimums were being met and that the productivity of soil and water would be sustained through time. They also provided some assurance of the health and viability of all involved ecosystems.

They were then used to create "options" within FORPLAN - sortuv built into the prescriptions or "linked" prescriptions and for the most part, not in constraints. In this way, the cost efficiency analysis required for forest planning included all of the legal minimums and health/viability of all ecosystems involved. Of course, there were many other "prescriptions" that were possible (and another model was used to investigate these), but these would not achieve the desired forest conditions even with very creative use of constraints.

In effect, we discarded all of the ways of managing the forest that would violate legal minimums and only provided alternatives to the Forest Supervisor composed of achieving specific forest conditions in specific locations that would meet at least minimum requirements on sustainability and ecosystem health/viability.

AND ... because maximization of Present Net Value was the objective function, we could say that each alternative was the most cost efficient way to achieve the specific mix forest condtions and thereby a specific mix of goods and services in each alternative while meeting the considerations for sustained yield, protection of soil and water productivity, etc. In short, this set of alternatives were "superior" alternatives in Kornoi's terminology.

In other words, it met the requirements for economic analysis without trying to put a price on such things as sustaining a quiet place for re-creation of your spirit and soul or for a lovely picnic with your family.

Tom Mitchell | May 6, 2005 9:21:46 AM

Posted by: Tom Roberts

As a young range con in BLM - in the late 1970's I played the Cost-Benefit ratio game, at that time to evaluate range improvements- but it wasn't long, not that I didn't realize it before, that I was convinced that it was shakey(?) because of what could be done with it. While we no longer use it, certainly for fairly low cost range improvements, the point is made or was made, that although flawed, it makes an attempt to put a common denominator of different goods or values so that they could be compared. Indeed, I used to discuss the value of a small game or big game hunter day, with the Outdoor Writer for the Salt Lake Tribune - knowing that that I didn't eat much differently when hunting than I did going out to the field and often carried the same (un-shot up) box of shells from the previous years. It didn't make sense always, but it was part of the proces and did/does make the different goods or services comparable when looking at potential projects. I think thankful, is how I would describe my feeling about continued use of C/B ratios for most projects that BLM does. Having seen them used (or developed them), I know how they can be used - again realizing and understanding that it is an attempt, albeit an attempt to put things on an even playing field to be compared.

Tom Roberts

Tom Roberts | May 19, 2005 3:34:46 PM

Posted by: Dave Iverson

Tom R,

Yep.. Cost-benefit Analysis "is" an attempt, albeit perverse, to put things on a level playing field.

Problem is, it's a field set up for business play. The base theory behind it all is that of free markets, atomistic consumers, non-monopolistic providers of marketable goods, etc.

It was adopted by government practitioners over the strong opposition of many, and has proven lasting power in spite of continued criticism. Few of the underlying assumptions prove up, even when tweaked by some to be better suited to the stuff of government rather than business.

But there is strong sentiment to put a price on pracitically everything in our consumer-oriented culture. A culture that seems to be ever more indebted (public and private), and ever more spiritually impoverished as time goes by.

So cost-benefit analysis continues to be promoted, and recently there seems to be renewed interested in pricing out (and marketing) environmental services. Some believe this will be the magic bullet to put the environment on an even par with traditionally marketed goods and services. I am among those who believe this to be but the latest in a long string of folly regarding cost-benefit analysis. But we'll leave that for a follow-up post. (Or maybe we've dealt with it enough in this one.)

And we'll leave the broader notion of "framing" for other posts, wherein we can deal with the tie between the mythology of "free markets" and pushes to run the government more and more like a business.

Dave Iverson | May 26, 2005 8:43:15 AM

Posted by: Allen Jerome

Very interesting discussion. I am an aerospace engineer working on fuel combustion. So far fuel efficiency with no consideration to the environmental impact of the fuel used to be the prime concern in our research. The department of environmental engineering used to do separate research to control pollution. Last year we took an initiative to do combined research with environmental engineering department to do research on increasing fuel efficiency having environmental impact as a consideration.

Allen Jerome | Jan 13, 2012 9:50:45 PM

The comments to this entry are closed.