Flyvbjerg, Skamris Holm and Buhl: Déjà vu All Over Again? Special Report No. 5
Leroy W. Demery, Jr. • Michael D. Setty • Updated January 2007
Copyright 2003–2007,
Publication in April 2005 of a new study by Bent Flyvbjerg, Mette K. Skamris Holm and Søren L. Buhl brought back memories of an earlier work by the same authors.
The 2002 paper by Flyvbjerg et al., Underestimating Costs in Public Works Projects: Error or Lie? The 2005 paper, How (In)accurate Are Demand Forecasts in Public Works Projects? The Case of Transportation, was also published by the APA Journal.
Flyvbjerg (pronounced much like FLEW-byair; rhymes with "Pierre") and Buhl are professors at Aalborg University, Denmark. Skamris Holm is a planner with Aalborg Municipality.
The 2002 paper analyzed construction cost estimates for large public works projects using statistical methods. Flyvbjerg and his collaborators demonstrated a taste for very strongly worded allegations that they failed to document.
Underestimation cannot be explained by error and is best explained by strategic misrepresentation, that is, lying. The policy implications are clear: legislators, administrators, investors, media representatives and members of the public who value honest numbers should not trust cost estimates and cost-benefit analyses produced by project promoters and their analysts.
A cynic might describe the above as a flagrant strategy to garner attention, name recognition, publicity for upcoming projects – and to discourage criticism. The 2002 paper attracted much attention and stirred significant commentary, but little formal criticism followed. This does not surprise us. Many academics have a strong vested interest in their pet theory – that costs are systematically underestimated and utilization overestimated for large public works projects – and have little interest in criticism or “alternative” explanations. This, as we demonstrated in a previous post, has led to a situation where a paper with significant analytical flaws (Flyvbjerg et al. 2002) was followed by one with even greater flaws (Flyvbjerg et al. 2005). Nonetheless, a “rerun” of the 2002 media frenzy is all but certain.
This article reviews the issues related to Flyvbjerg et al. (2002) – and the ensuing media circus. As usual, we invite original commentary and feedback for posting here on; we may, of course, choose to post a subsequent response.
Q: Why do very large public-works projects almost always cost more to build than predicted before the start of construction?
A: Because taxpayers would probably not have it any other way.
The exercises that follow were written in order to summarize various issues related to infrastructure project cost estimation. They make reference to various points of "engineering economics," the field that deals with estimating cost, and benefits, for various projects. We try to provide a generous supply of links here at , but we strongly recommend an old-fashioned paper textbook, such as Park and Jackson (1984). The issues summarized below are very important to transit supporters, and so it is worthwhile to check out this or some other engineering-economics text. (For a brief overview, see
1)  Estimation costs money – and sometimes, a lot of money
Planners, designers, architects and engineers do not work for free. Cost estimation for certain projects – e.g. houses, for example  – is routine because the projects are relatively small, many get built and construction is straightforward. Also, "site-specific" issues seldom lead to dramatic cost increases.
However, costs for other, larger projects – roads, bridges, tunnels, airports transit systems and so forth – must be estimated on a project-by-project basis. Such facilities are built infrequently, and so there is not as much data from previous projects as there is for, say, single-family houses. Each project is large and complex, construction may involve unanticipated difficulties and delays and site-specific problems (soil conditions, endangered species) may lead to large cost escalations. Cost estimation is a challenging exercise – and costs accordingly.
2)  Precision: the more you want, the more you pay
The price one pays for cost estimation is generally stated as a percent of total project cost. "Textbook" examples are generally based on a project cost of $1 million.
"Back of envelope" (aka "ballpark" or "seat-of-the-pants") estimation costs a tiny fraction of the total project outlay (less than $1,000 based on the $1 million example). However, such an estimate is no more precise than "plus or minus" 50 percent. In other words, the "actual" construction budget might easily prove 50 percent greater than estimated.
One step up is an "order-of-magnitude" estimate – precise to within 30-50 percent, but seldom more precise than "plus or minus" 15 percent. That should cost about 0.2 percent of the total ($2,000 based on the $1 million example). However, substantial cost escalation remains possible. Accountants will want better numbers for budgeting purposes, and contractors will need much better numbers before they will risk a fixed-price bid (even firms known for “overdesign” and large cost overruns will say this).
An estimate adequate for budgeting purposes, precise to "plus or minus" 10 percent, will cost about 1.5 percent of the total cost (about $15,000 based on the $1 million example). An estimate good enough for a fixed-bid requires "plus or minus" 5 percent precision – and will cost about four percent of the total ($40,000 based on the $1 million example).
Most readers will recognize the absurdity of spending large sums to estimate costs prior to the decision to proceed. Matthew Hisrich and Jen Melby of Ohio’s Buckeye Institute evidently did not:
Policymakers may choose to fund a less expensive estimate so as not to consume a large portion of their budget for a project before ground is even broken. This allows them to avoid political heat over the cost of a study. This applies to the study of other alternatives, as well. Accurate estimates of the costs to build both what agency heads want to build, as well as what they would prefer not to, would eat up a great deal of funds.
(Hisrich, Matthew, and Jen Melby. 2003. Ballooning Transit Costs.
The passage above suggests extreme naïvety – or a poorly concealed agenda. Hisrich and Melby appear unwilling to acknowledge that large sums – perhaps in the tens of millions – cannot reasonably be spent for cost estimation during early planning. They fail to explain how such outlays might be justified in public-policy terms.
(That Hisrich and Melby are not naïve soon becomes apparent upon comparison with this article. Agendas tend to become obvious over time.)
Assume that five different options are under consideration during early planning. Only one, at most, will be built. It is not likely that anyone would spend $40,000 x 5 = $200,000, or 20 percent of the project budget, for "top of the line" cost estimation for all five "alternatives" (based on the $1 million example). It is true that some of the alternatives may be similar enough so that the total "cost of estimation" is less than stated above, but the issue does not go away.
Nor does the issue go away if the project budget is $100 million rather than $1 million, and the "percentage of final cost" factors are be lower than those used above.
Certain firms known for "gold plating" of projects might enjoy collecting the cash for "top-of-the line" cost estimation during early planning – but taxpayers would certainly disapprove. This, given five alternatives, might cost 10-15 percent of the project total. This works out to $10-15 million, a fraction of the $100 million budget but still a large amount of money. Hisrich and Melby may pretend otherwise, but politicians foolish (or corrupt) enough to spend such sums for inappropriate cost estimation would get voted out of office.
3)   Contingency allowances
A "contingency allowance" is an additional amount, based on some percentage of the estimated cost, generally added as the "final step."
A contingency allowance does not increase the precision of the original estimate, and cannot guarantee against cost overruns. It merely increases the probability that the estimate will fall at (or below) the "actual" cost.
It is possible to quantify this probability, and this has been publicized as part of cost estimates for planned "mega-projects" in Seattle and elsewhere.
For example: "There's an 85-percent chance that the cost to build Project X will fall within 10 percent of the estimate."
Or, in other words: "There's an 85-percent chance that the cost to build Project X will fall between $90 and $110 million."
4)  Site-specific unknowns
It is next to impossible to identify, and get a handle on, everything that might affect the cost of a large and complex construction project – at least not at reasonable cost. This fact tends to guarantee that cost escalation will afflict large projects. Unlike houses, large projects are not built everyday, and so less data from "previous" projects is available. As a practical matter, "unanticipated" and "unknown" factors will almost certainly turn "plus or minus" into a "plus." (The trick is to keep the "plus" tendency under control.)
One could (in theory) spend very large sums on pre-construction studies and come up with history's most precise cost estimate. Certain firms might be happy to oblige. However, this would amount effectively to full-scale preliminary engineering – prior to a decision to proceed. This, with the qualified exception noted below, rarely occurs in the real world – simply because of cost. No one conducts preliminary engineering for several different "alternatives" or "options" in a situation where only one, at most, will be built – again, because of cost.
5)  The Downtown Seattle Transit Tunnel: An exception that proves the rule
During the 1980s, the erstwhile Municipality of Metropolitan Seattle (Metro) was given a significant and unusual "advantage" that permitted a much more precise "pre-construction" cost estimate than otherwise possible. The Urban Mass Transit Administration, wanting very much to advance a major "non-rail" project, permitted Metro to begin preliminary engineering during the "alternatives-analysis" phase. Metro was permitted to pay for "top-of-the-line" cost estimation during "early planning," and therefore had much better information than it otherwise would have for cost-estimation purposes.
6)  "Padding" the estimate
Ethical planners and civil engineers try to base cost estimates on facts, data – and professional judgment. Everyone knows that the unknown and the unanticipated can and do increase costs. "Padding," or arbitrary increases without supporting facts, is known to occur but is frowned upon. This amounts to unjustified escalation of a prospective capital budget.
Would anyone pay a 20 percent surcharge on a restaurant tab to cover the "estimated cost" of "seconds" that "most customers" order?
Perhaps bureaucrats attract greater sympathy: Would anyone pay a $1,000 surcharge on an initial driver's license to cover "estimated fines" for "traffic violations" that "most drivers" commit during their lifetime?
We are confident that few readers would accept bill padding by restaurants, or a state licensing bureau. We are similarly confident that most readers would not accept it from planners and engineers, either.
Designers do, of course, make allowances for safety. A colleague reminds us that engineers must apply a "safety factor" to allow for errors and misjudgments, and that this is a matter of professional judgment. If "accuracy" is improved by reducing "safety factors," the result is more risks as corners are cut.
Another colleague reminds us that a lot of real-world problems are not "cut-and-dried" with "sample problems in the back of the textbook." Solving problems in real life requires "a good measure of common sense, professional judgment and, sometimes, a bit of luck."
The same colleague emphasizes the need for professional judgment: a lot of facts are not published, and a lot of published "facts" are pure fiction.
7)  "Why can’t planners and engineers take past experience into account and adjust cost estimates accordingly?"
The answer: Because there is no systematic, ethical – and legal – way to do this (at early 2005, although strategies have been proposed).
Referring to section 3 above, when one pays for an estimate (remember: the more precision you want, the more you pay), the result is based on available facts and information, allowances for safety as appropriate, and professional judgment. Basing an estimate on anything else amounts to unjustified inflation of a prospective capital budget.
One might ask, for example, "What if there's an earthquake fault under that proposed project; aren't you going to take that into consideration?"
The answer, absent some evidence that a fault exists, or that the area is a seismic "hot spot" where previously-unknown faults have been discovered, may be short, sour and to the point: “No.”
One might also ask about "all those cost overruns" elsewhere. If so, the estimator may ask for details that are pertinent to the project in question. In other words, s/he will want specific details of what caused "those" overruns and why these factors might affect this project. Absent such information, the estimator will defend the result as the best possible, given available information, reasonable safety factors, and best professional judgment (and, of course, the time paid for).
8)  "Why not increase the contingency allowance?"
This idea tends to be a non-starter for a couple of reasons.
First, contingency allowances for various types of projects are established by statistical studies of previous experience. Such information may not be available for large projects that are built relatively infrequently, or are of a type or scale that has not been built before.
Second, some colleagues say that a large contingency allowance is not a good idea because it provides an inviting "slush fund" for unscrupulous contractors and consultants, who might use a variety of tactics to attempt to secure a share of the extra money.
9)  "Why do consultants avoid overestimation?"
Assume that a project were estimated to cost $200 million, and this estimate was carried through planning until the day bids were opened – and the lowest bid proved to be $100 million.
The project manager would certainly celebrate. Then, after the hangover wore off, s/he might sue, and might even call the county prosecutor.
As a practical matter, the enterprise that conducted initial planning was probably the one retained to manage subsequent work, and perhaps even construction. A contract was probably signed that outlined how much would be paid for these services.
Planning and management fees may be based in part on anticipated total project cost. The consultant risks little (other than ire) if the project ends up costing more than estimated. But the firm will need a good explanation if it costs significantly less, and its fees were based on the original estimate to any significant degree. In extreme cases, lawsuits and prosecution for fraud may result. Therefore, even the most avaricious of consultants avoid overestimation.
10. Flyvbjerg et al. (2002): the media frenzy
. . . Underestimation cannot be explained by error and is best explained by strategic misrepresentation, that is, lying. . . .
Well-known transit critic Wendell Cox quickly embraced the above; however, it was not clear whether Cox and similar critics actually read the paper.
Flyvbjerg et al. (2002) received its first major media publicity in The New York Times ("Study Finds Steady Overruns in Public Projects," by Michael Wilson, July 11, 2002).
The New York Times online archives requires a fee for any story more than seven days old. A similar article from the Seattle Post-Intelligencer is here
Cost overruns for large public works projects have stayed largely constant for most of the last century . . . estimates are no more accurate now than they were 90 years ago.
--That is a rather sweeping statement, given that "large public works projects" have grown larger, more complex and have incorporated much more "innovative technology" over the past 90 years.
--Ninety years ago, the problem was much worse. A report on the construction of New York City's first subway line, in the collection of a colleague, states that public works projects have a history of costing twice the original estimate, and mentions the Croton Water Works as an example.
When predicting the expenses of transportation projects, economic forecasters and project promoters in the United States, Europe and elsewhere have made underestimates costing the public hundreds of billions of dollars.
--This scrambled rhetorical flourish, taken literally, asserts that the estimates – the actual estimates – cost "hundreds of billions" more than they should have. We doubt that Flyvbjerg et al. intended to advance such absurdity.
The Channel tunnel between England and France came in 80 percent over budget;" initial estimates for the Holland Tunnel in New York, "completed in 1927 at a cost of $48 million," were "52 percent lower.
--These and other "overruns" surprised no one, given that public works projects have a history of costing more than the original estimate. The central point of Flyvbjerg et al. (2002) makes sense if, and only if, one believes:
1.) "Actual" costs could have been predicted prior to the start of construction.
2.) The projects would not have been built if the "actual" cost had been known. This is certainly true for some, but probably not many.
3.) The projects, as built, were not worth the "actual" cost of construction.
(Cox and others typically argue all three points above with reference to public transit.)
--The typical "post-construction" retrospective tends to read as follows:
In March, 1900, when the Mayor with appropriate ceremonies broke ground at the Borough Hall, in Manhattan, for the new road, there were many well-informed people, including prominent financiers and experienced engineers, who freely prophesied failure for the enterprise, although the contract had been taken by a most capable contractor, and one of the best known banking houses in America had committed itself to finance the undertaking.
In looking at the finished road as a completed work, one is apt to wonder why it ever seemed impossible and to forget the difficulties that confronted the builders at the start.
(The New York Subway: Its Construction and Equipment, Interborough Rapid Transit Co, 1904, Introduction, see here, ; the remainder of this page makes fascinating reading.)
Or, as Julie Hoover put it at an American Public Transit Association (APTA) conference:
Does anyone in this audience seriously believe Portland officials would have voted differently if they thought the cost of the [first light rail] Project was $210 million rather than the $172 million originally projected?
If polled, would decision makers anywhere be more concerned with ridership on opening day than with ridership, urban development, air quality and quality of life generally ten or even twenty years later?
Who here thinks that [Atlanta] Mayor [Andrew] Young might be alarmed by the Pickrell report, and wish that Atlanta had opted for an all-bus system if he had only known from the beginning that the costs of rail would be higher?
Indeed, it would appear that most U.S. cities are happy with their rail systems because twenty-seven extensions are currently in some stage of active development.
(Ridership and Cost Forecasting, by Julie Hoover, paper presented at the APTA Rapid Transit Conference, Vancouver, BC, 1990. Not available online. "The Pickrell Report" refers to Urban Rail Transit Projects: Forecast vs. Actual Ridership and Costs, Urban Mass Transit Administration Report, U.S. Department of Transportation, 1990, authored by Dr. Donald H. (“Don”) Pickrell, Chief Economist, John A. Volpe National Transportation Systems Center, Cambridge, MA, see here .)
Returning to the New York Times article: The solution prescribed by Flyvberg et al. (2002) is more openness in the estimation process.
The more public awareness and participation, the more accurate the estimate might become.
--We at  are strong supporters of public awareness and participation. We also think that the public should be more involved in early planning for large projects such as airports, highways and transit. We think this will lead to greater understanding of costs and risks before "megaprojects" are approved. But we do not believe that "increased public awareness and participation" will increase the "accuracy" of estimates – there is no causal link between the two.
11. Builders don’t talk to Flyvbjerg (so he says)
Flyvbjerg told the news media that he had difficulty getting "builders" to discuss the issue. 'People run away screaming,' he said. “It doesn't look good for the profession.'"
--A study like Flyvbjerg's is potentially very useful – provided the author had a different objective than Flyvbjerg apparently did. It might even lead to development of that "systematic, ethical – and legal" way to take previous experience into account when estimating costs for large public-works projects. But Flyvbjerg was not concerned with this. His apparent primary interest was selling his latest book.
Note that the New York Times avoided Flyvbjerg's provocative language about "lying." But Wendell Cox, as we will describe in due course, did exactly the opposite with his "Liar, Liar" campaign.
12. Anti-rail crib clubs (excuse us, “think tanks”) weigh in
Once the denizens of various anti-transit, anti-urban "think tanks" learned of the Flyvbjerg paper, they began working quadruple-overtime, grinding out propaganda. Remember that the New York Times article described above appeared on July 11, 2002.
A condensation, also dated July 11, was posted here by the National Center for Policy Analysis. Readers curious about what NCPA supports might check out the link here ; the Board of Directors listing is here the NCPA Experts list is here and the NCPA staff list is here . Note the prominence of Pierre (“Pete”) du Pont
Another example, titled "True Cost of Rail," see here , was posted on the "Ax the Tax" website, see
here Ax the Tax is a project of Orlando talk-show host Doug Guetzloe (see here and here ; click "DOUG GUETZLOE’S BIO" at left), he is noted for his anti-tax and very anti-rail views (see here ; click "Orange County Transport Tax," "Light Rail
Battle 1999" and "Commuter Rail"at left). Orlando Weekly News writer Jeff Billman described a school district "screwup" that gave "Ax the Tax blowhard Doug Guetzloe an opportunity to jump back in the spotlight and do what he does best -- bitch about taxes" (see here ; scroll down to "Education: Fickle Funding;" broken link at 2007.1.13).
We note that other websites and weblogs posted the New York Times article; we doubt that the "Coalition for a Livable West Side" (New York City) (see here; broken link at 2007.1.13; home page is here ) can fairly be described as a "conservative" or "right-wing" think tank.
As we noted before, the New York Times avoided Flyvbjerg's provocative language about "lying." But Wendell Cox went overboard.
13. Wendell Cox exploits "the" Flyvbjerg quote
We repeat the following direct quote from the abstract of Flyvbjerg et al (2002)::
. . . Underestimation cannot be explained by error and is best explained by strategic misrepresentation, that is, lying. . . .
Certain aspects of the typical Wendell Cox analysis, based on quotes from Flyvbjerg et al. (2002), proved amusing First, as a number of delightful essays make clear (e.g. Frankfurt 2005), there is a crucial difference between "lying" and another form of deception, which we will call "B.S.-ing" (numerous examples on Usenet, particularly at misc.transport.urban-transit).
In order to lie, one must have some prior knowledge or belief about the "truth." Otherwise, one may certainly "deceive" but cannot "lie." Nor, we are advised, can one be convicted of "perjury" (at least in the U.S.) if one can prove, in court, that one could not have known that the statement at issue was false – even if it actually was so. (We are also advised that this strategy carries the risk of a contempt of court citation – but that is not the same thing as perjury.)
On the other hand, "B.S.-ers" are not concerned with truth or falsity. Their primarily concern is to convince other that they have the facts; that that they're the person in the know, that they're the ones to turn to for the "real story."
Lying is an overt act (for which one may be prosecuted and "put away"); "B.S.-ing" is more of a performance (and is also difficult to prosecute). Thus the term "B.S.-artist."
Flyvbjerg and his colleagues (2002) commit a serious conceptual error (at least from the viewploint of U.S. readers) by using the word "lying," or euphemisms meaning the same thing. It implies that readers will find clear evidence that planners knew the "truth" – either the actual project cost (which, as a practical matter, is impossible), or the fact that the projects would cost more than estimated (which is widely known). But no such thing appears in the paper.
14. “Liar! Liar!” trumpets Wendell Cox
Wendell Cox penned an article titled “Liar, Liar,” posted on July 8, 2002 on the "Tech Central Station" website ("Where Free Markets Meet Technology," home page is here ; information about website host James K. Glassman
is here ). Note the publication date, three days prior to the New York Times article of July 11, 2002; unfortunately, this is no longer available online.
Cox pontificates as follows:
The use of deception and lying as tactics in power struggles aimed at getting projects started and at making a profit appear to best explain why costs are highly and systematically underestimated in transportation infrastructure projects.
There is a very simple incentive for large contractors to lie. It is profitable. The reward is more money. And, there is an incentive for public officials to believe the lie. The 'Edifice Complex,' which drives so many elected officials to build virtual monuments and to seek 'legacy' is served by underestimating costs, since less costly projects are simpler to sell to governing bodies and the electorate than more costly projects.
This sounds plausible, but Cox then demolishes his case in the next paragraph:
My personal experience confirms this. In 1981, I was among the members of the Los Angeles County Transportation Commission voting to build the Los Angeles to Long Beach light rail line. We were given an estimated cost of $140 million by the consultants, who were later selected to oversee building of the route. By the time it was all over, the line cost more than four times the original estimate, and that's after inflation. If we had known in 1981 what the true cost would have been, there is simply no way that the project would have been authorized.
With this passage, Cox, who was an LACTC board member from 1977 to 1985, evidently engages in a bit of "strategic misrepresentation."
It is well known that Los Angeles County voters approved a sales tax in 1980, with the revenue dedicated explicitly for rail construction (after an initial period when the money was used to temporarily lower bus fares). An outline map of a countywide network even appeared on the ballot.
It is also well known that the original $140 million estimate was for a bare bones, single-track line as built in Sacramento. This would have been woefully inadequate for the Blue Line corridor. Double track, high platforms and a short subway were among the features not envisioned originally (stations “should” have been built with platforms long enough for four cars, not just two). In fact, the line narrowly averted a crippling shortage of cars because LACTC included some for the Green Line in the initial order.
Cox certainly knows all this. He writes that:
The large rail projects tend to be built by a very small market of international contractors. The size of such projects makes a truly competitive market problematic, since smaller companies do not have the resources to overcome the substantial financial barriers to entry. On the other hand, road projects tend to be broken up into much smaller components, with healthy competition from small, medium and large companies. But in the less competitive market for mega-projects, governments are more susceptible to the cost-enhancing behaviour [sic] of the large firms.
Well, some governments anyway. French cities divide light-rail construction projects into a number of small contracts, and exercise meaningful supervision.
By 1990, the recently-elected mayor of Strasbourg [France], Mme Catherine Trautmann [curriculum vitae here  ], doubling as the chair of CUS [Commmunauté Urbaine de Strasbourg, the regional authority responsible for transport], had risen to prominence as the leading tramway champion of France. Her passion made all the difference, and it was a stroke of genius for her to understand that the project was so complex that a political body like CUS should not act as a 'maître d'ouevre' (project director). A high degree of flexibility was needed, so the project was entrusted to a private company as a joint venture.
The public transport operator is Compagnie des Transports Strasbourgeois (CTS) [homepage here  ], in which the Transdev group [English-language page here ] has a minority stake. It is not the usual solution in France, but it has proved a blessing in Strasbourg that the operator has been allowed so much freedom. The track record of CTS was excellent, as it had been the city's one and only urban transit operator since 1878.
Ironically, it ran the trams between 1878-1914, and built its own urban and interurban electric tram lines. Mme Trautmann intervened to reinstate CTS both as the maître d'ouvrage (the instance which pays for the project), as well as concessionaire du nouveau réseau -- the franchise-holder to run the new, standard-gauge tramway, as well as the connecting bus network.
Bridges, tracks, infrastructure. . . the entire project was broken up into sub-tasks, each awarded to a specialized consultancy firm under the overall supervation of GETAS (Groupement d'Etudes du Tramway de l'Agglomération de Strasbourg). This GETAS company is allied with the SEMALY consultancy of Lyon [English-language home page here  ], which designed several new French tramway systems, including those of Nantes, Grenoble, Montpellier and Lyon. All SEMALY-designed tramways in France have proven highly successful.
By organising the work in this way, Mme Trautmann built a quality tram line on time, without cost overruns. She was elected twice as mayor, the reward for 'her' tramway. Now aged 51, Mme Trautmann was displaced as mayor in March 2001, when her party lost the elections as France lurched to the right. Thee proven efficiency of tramway-building means that banks will continue to lend money at discount rates, now that they iknow the Strasbourg tramway project is sound. It was financed by the Caisse de Dépot et Consignation, which holds a controlling stake in Transdev (Wansbeek 2003).
Cox also neglects to mention that breaking up large public works projects into small contracts fell out of favor in this country because large firms were seen as more "efficient." Also, public agencies prefer to hire consultants ("independent contractors") rather than spend the money for "in-house" expertise. Cox's article, Playing with Trains, published on August 6, 2002 in the East Valley Times (Mesa, AZ; see here, access to archives requires registration), contained the following "kicker":
As Valley gears up, light-rail ridership elsewhere declines.
The first paragraph reads as follows:
The Valley will soon spend more than $1 billion for the Central Phoenix - East Valley light-rail line. The line was sold to the voters under the promise of reducing traffic congestion. They were misled.
Cox is no shrinking violet: "Light rail ridership elsewhere declines," he says, but offers no examples of his claim. The article repeats previous observations that traffic congestion on freeways parallel to rail lines does not go away, and that Census Bureau figures suggest that the rail market share of work trips is declining. With respect to the latter, Cox contradicts his own argument in a subsequent paragraph:
None of this is to suggest that light-rail systems are unpopular in their communities. In Portland, St. Louis, Dallas and elsewhere, people have found that light rail is a pleasant way to travel to downtown stadiums, arenas and festivals. But as for commuting to work, light rail will provide no relief and fails abjectly in the fundamental promise that induced approval.
This particular Cox effort mentioned the Flyvbjerg (2002) study rather parenthetically:
. . . A new report published by the American Planning Association found that cost escalation occurs in nine of 10 projects, in a study of hundreds of projects North America and Europe. The authors, Danish university professors, coined a technical term for the cause - 'lying.'
Another Cox effort, “Of Rail, Lying and Lemons,” was published "for immediate release" on August 8, 2002 by the Independence Institute (see here ). The Independence Institute is a
conservative Colorado think tank (home page here ). Jon Caldara, a former chair of the
Regional Transportation District (metro Denver) board. is President of the Independence Institute; see here and note that he shamelessly mentions a newspaper item labeling him as Denver's "Best media Manipulator," see here:
Cox concludes:
But there is a more important issue here. The legal equivalent of lying is fraud. Consumer protection laws forbid fraud in commercial transactions and violation can result in civil or even criminal penalties. The citizenry should at least be able to depend upon being treated as well by their government as by the otherwise unscrupulous used-car salesman who values his freedom more than a fraudulently earned commission. Perhaps it is time to apply "lemon" laws to government.
The above is a good example of Cox’s "over-the-top" rhetoric. "Lying," "fraud" and "lemon laws" require legal proof, which (except in a very few cases such as Boston's “Big Dig”) neither he, nor Flyvberg et al., nor anyone else has.
["Lemon laws" are intended to provide relief to consumers who make multiple, unsuccessful attempts to resolve auto or other product defects, see here
Another item, posted on Cox’s website on July 21, 2002, is titled “International Research: Lying about Project Costs: Implications for Florida High-Speed Rail,” see here , cites the Flyvbjerg study as a
reference. (Not much "new" here.)
15. Efforts to control U.S. rail cost overruns: successful
This fact probably won’t appear on Cox’s website anytime soon, but efforts to control cost overruns have produced results – at least for U.S. rail transit projects. As a colleague puts it, recent U.S. projects have been coming in on time and under budget more often than not. By contrast, recent busway and transitway projects have experienced large cost overruns, such as the East Busway in Pittsburgh: $320 million estimate, $515 million bid. Other "real-world" narratives from colleagues:
--Two contractors worked on a subway extension in an East Coast city. One got right to work and completed the job with no notable problems, on time and on budget. The other, well connected politically, used ongoing eminent-domain litigation as an excuse to delay the start of work. The city refused repeated requests for "change orders" and the contractor eventually had to pick up the tab for the delay.
--A company missed the contracted deadline for a railcar rebuilding job by more than a year, because other work kept "getting in the way." This created additional overhead expenses, for which the firm billed the transit agency. It refused payment on grounds that the company had bid the job, and what one bids is what one gets, completion date and all.
--In Seattle, Washington State Ferries decided about 15 years ago to refurbish an existing ferry (the M.V. Rhododendron, see here ) as a lower-cost alternative to buying a new one. Then, once the project was under way (1990), the U.S. Coast Guard submitted a "change order" (so to speak) – replace the superstructure with all-new construction (USCG has authority to do this on grounds of safety). Result: a hefty cost "overrun" -- but the refurbishment project still cost less than a new vessel.
A colleague provided an interesting outline of the life and times of cost estimators who work for the equivalent of the U.S. General Services Administration in one Eastern Seaboard state. Among other things, they verify cost estimates for new construction and rehabilitation projects. The state legislature uses their estimates when deciding how much money to appropriate for various projects. Estimators have a large incentive to be accurate – if legislators get sufficiently irritated by a particular employee's incompetence, s/he is gone very quickly.
During early project design, state cost estimators use information available and "Means Data," a large national database for individual construction-cost elements (e.g. labor, materials), updated periodically and adjusted on a city-by-city basis by the R.S. Means Co (see here
Although these do not appear on Cox's website, or in Flyvbjerg et al. (2002), there are six well-known reasons why "actual" costs differ from "initial estimates," as outlined by a colleague:
1. Architects and engineers discover that initial design concepts were inadequate. This is a chronic problem in the "high-tech" sector because design standards change frequently and may not be known to all specialists. Imagine that "clean room" (microchip manufacturing facility) specs were discovered to be "behind the times" after design work begins. This has happened at least once; an outside consultant, with knowledge of the "state of the art," was hired – and the estimated cost of the clean rooms turned out to be roughly 50 percent higher than originally estimated.
2. The original design concepts were not thought through clearly, or adequately. Problems of this type may arise from personnel changes (e.g. changes in agency leadership), technological advances (e.g. rendering some design elements obsolete) or legal requirements unknown (or not in effect) when the original estimate was prepared.
3. Changes in project scope requested by the "lead agency" that increase project size and complexity. These usually require re-design, and renders some (or all) of the previous work obsolete.
As we have noted previously, the light-rail Blue Line in L.A. is an excellent example of 3., one that is certainly well known to Cox. We are not surprised that he never mentions this.  If he did, he would have to admit that he and other LACTC board members accepted an initial estimate that they should have known was totally unrealistic.  Either that, or he would have to argue that the line, as originally planned, would have been perfectly adequate for Blue Line corridor traffic.
4. Discovery, following completion of initial estimates, of factors that must be accommodated – and tend to exert an upward influence on cost estimates. These include, but by no means are limited, to unstable soils, contaminated groundwater or soil, redefinition of "100-year flood" plains.
A colleague based in Los Angeles recounts two stories that illustrate this problem. The lake at MacArthur Park was drained in 1991 during construction of a Red Line subway extension. The lake had been drained about 20 years previously, and so this step was not expected to involve major problems. However, local officials refused to permit dumping of the water into the local storm-drain system owing to high particulate content. And so, a job planned to require four days actually took four weeks.
Discovery of highly contaminated groundwater near the site of the planned Red Line maintenance facility led to significant construction delays. The water could not be pumped out and dumped into the nearby concrete drainage channel (also known as the Los Angeles River), but had to be decontaminated first. The source was traced to a long-forgotten coal gasification plant that once stood near today's Union Station, during the early years of the 20th century.
5. Changes – usually upward – in the cost of labor and materials during the design period, and during actual construction, particularly if unanticipated problems force delays.
6. Economic conditions. If the economy is weak and there are more bidders than projects, there will be "competitive bidding." When the economy strengthens, and there are more projects than bidders, firms will raise their bids to what they think the market will bear.
Given these factors, it's remarkable when a project comes in on time and below budget. Recent examples in the U.S. include light-rail projects in Salt Lake City and Minneapolis.
16. Conclusions
. . . Underestimation cannot be explained by error and is best explained by strategic misrepresentation, that is, lying. . . .
This provocative assertion by Bent Flyvbjerg and his colleagues in their 2002 paper made good grist for various anti-transit propaganda mills. However, in order to demonstrate "lying," it is necessary to demonstrate some (reasonable) prior knowledge (or belief) regarding “the truth.” (One unable to do so might want to use some word other than "lying" to avoid being sued for libel.) In order to demonstrate "lying," Flyvbjerg et al. would need to conduct a project-by-project review on points 1-6 above (at least) for everything included in his study. This is probably not possible for projects completed many decades ago.
Of course, provocative assertions make useful tools for garnering publicity if one is primarily interested in selling books, securing research grants and so forth. Whether this is fair to say about Flyvbjerg et al. is not clear, but we do note that, judging from available evidence online, they seldom if ever bother to respond to "outside" criticism. The long-term hazards of this practice should be clear.
Frankfurt, Harry G. 2005. On Bullshit. Princeton, NJ: Princeton University Press.
Flyvbjerg, Bent, Mette K. Skamris Holm and Søren L. Buhl. 2002. “Underestimating Costs in Public Works Projects: Error or Lie?” Journal of the American Planning Association 68, 3 (Summer 2002): 279-295 .
__________. 2005. “How (In)accurate Are Demand Forecasts in Public Works Projects?” The Case of Transportation. Journal of the American Planning Association 71, 2 (Spring 2005): 131-145.
Park, William R., and Dale E. Jackson. 1984. Cost Engineering Analysis - A Guide to Economic Evaluation of Engineering Projects. New York: John Wiley & Sons.
Wansbeek, C. J. 2003. “Strasbourg: Interurban tram strategy strengthens city system.” Tramways & Urban Transit 66, 783 (March 2003): 94-97.