Megaprojects: Over budget, over time, benefits less than projected over and over again

[Axulus]

Fascinating paper on megaprojects, typically those projects costing over a billion dollars and/or affecting more than a million people:

Abstract:

This paper takes stock of megaproject management, an emerging and hugely costly field of study. First, it answers the question of how large megaprojects are by measuring them in the units mega, giga, and tera, concluding we are presently entering a new "tera era" of trillion-dollar projects. Second, total global megaproject spending is assessed, at USD 6-9 trillion annually, or 8 percent of total global GDP, which denotes the biggest investment boom in human history. Third, four "sublimes" – political, technological, economic, and aesthetic – are identified to explain the increased size and frequency of megaprojects. Fourth, the "iron law of megaprojects" is laid out and documented: Over budget, over time, over and over again. Moreover, the "break-fix model" of megaproject management is introduced as an explanation of the iron law. Fifth, Albert O. Hirschman's theory of the Hiding Hand is revisited and critiqued as unfounded and corrupting for megaproject thinking in both the academy and policy. Sixth, it is shown how megaprojects are systematically subject to "survival of the unfittest," explaining why the worst projects get built instead of the best. Finally, it is argued that the conventional way of managing megaprojects has reached a "tension point," where tradition is challenged and reform is emerging.

I thought this was especially fascinating:

The Iron Law of Megaprojects

Performance data for megaprojects speak their own language. Nine out of ten such projects have cost overruns; overruns of up to 50% in real terms are common, over 50% are not uncommon. The cost overrun for the Channel Tunnel, the longest underwater rail tunnel in Europe, connecting the United Kingdom and France, was 80% in real terms. The cost overruns for the Denver International Airport were 200%; for Boston’s Big Dig, 220%; and for the Sydney Opera House, 1,400% (see more examples in Table 2)

...

Similarly, benefit shortfalls of up to 50% are also common and above 50% not uncommon, again with no signs of improvements over time and geography.

http://poseidon01.ssrn.com/delivery...080109015076025004099102000084&EXT=pdf&TYPE=2

The whole paper is a good read. Here is also a good interview with the author of the paper:

http://www.econtalk.org/archives/2015/05/bent_flyvbjerg.html

 

[Juma]

And how is this different for smaller projects?

 

[Axulus]

And how is this different for smaller projects?

The paper doesn't give data on smaller projects. However, the author does mention that things are better if you break things up into smaller units in the interview:

I also believe that for some projects, like dams is an example that you mentioned, and maybe energy projects in general--so often dams are often hydroelectric dams, and therefore energy projects, in addition to being water projects, water management projects. If you can deliver things in smaller units, our data indicate that you are much better off. That your risks of losing money and your risk of not delivering the promised benefits are much smaller if you do things in smaller units.

 

[Juma]

My experience, by projects I been in and others i heard of, is that very few would have been started if they had knew the real cost.

 

[Axulus]

My experience, by projects I been in and others i heard of, is that very few would have been started if they had knew the real cost.

I think this part of the paper is relevant (although you yourself aren't making this argument, but may perhaps be alluding to it):

Hirschman’s Hiding Hand, Revisited

One may argue, of course, as famously done by Hirschman (1967a, pp 12–13) that if people knew in advance the real costs and challenges involved in delivering a large project, “they probably would never have touched it” and nothing would ever get built; so, it is better not to know, because ignorance helps get projects started, according to this argument. The following excerpt is a recent and particularly candid articulation of the nothing-would-ever-get built argument, by former California State Assembly Speaker and Mayor of San Francisco, Willie Brown, discussing a large cost overrun on the San Francisco Transbay Terminal megaproject in his San Francisco Chronicle column (27 July 2013, with emphasis added):

“News that the Transbay Terminal is something like $300 million over budget should not come as a shock to anyone. We always knew the initial estimate was way under the real cost. Just like we never had a real cost for the [San Francisco] Central Subway or the [San Francisco–Oakland] Bay Bridge or any other massive construction project. So get off it. In the world of civic projects, the first budget is really just a down payment. If people knew the real cost from the start, nothing would ever be approved. The idea is to get going. Start digging a hole and make it so big, there’s no alternative to coming up with the money to fill it in.”

Rarely has the tactical use by project advocates of cost underestimation, sunk costs, and lock-in to get projects started been expressed by an insider more plainly, if somewhat cynically. It is easy to obtain such statements off the record, but few are willing to officially lend their name to them, for legal and ethical reasons, to which we will return later. Nevertheless, the nothingwould-ever-get-built argument has been influential with both practitioners and academics in megaproject management.

The argument is deeply flawed, however, and thus deserves a degree of attention and critique.

Read the section in the paper to discover the deep flaws in this line of argumentation:

http://poseidon01.ssrn.com/delivery...080109015076025004099102000084&EXT=pdf&TYPE=2

 

[Jimmy Higgins]

And how is this different for smaller projects?

Smaller projects typically have to stay within a reasonable limit of the budget. A problem with larger jobs are poor estimates in the beginning.

 

[Nice Squirrel]

Less than 10% of projects stay within budget, scope and time. Megaprojects never do because of all the unknowns. Good project planners build in chaos and risk into the plan, but in reality they are just educated guesses.

Also the need to keep the bids low also means unrealistic promises from unscrupulous contractors.

For example, your low-cost steel supplier doesn't have enough qualified workers and you refuse their product and need to find a new source.

 

[Jimmy Higgins]

Less than 10% of projects stay within budget, scope and time. Megaprojects never do because of all the unknowns. Good project planners build in chaos and risk into the plan, but in reality they are just educated guesses.

Also the need to keep the bids low also means unrealistic promises from unscrupulous contractors.

For example, your low-cost steel supplier doesn't have enough qualified workers and you refuse their product and need to find a new source.

Yeah, one of the bigger problems is the contractor's lawyer.

The trouble is you are trying to keep bid prices low, but want to build as much contingency into a bid as you can to keep extras from piling up, but the more contingency you put in there, the larger the unknowns and the higher the bids. And construction contractors should really be forced into the diplomatic arena as they'd be real good at making great diplomatic deals and treaties.

Contractors will under bid a job after looking the contract documents over and knowing exactly where he is going to claim extras before the start of a project. Half the job seems to be tightening up documents so much, it becomes impossible to claim undue extras, but the problem with the law is that the winner isn't who is right about the contract, but who is able to win the argument that they are right.

 

[dismal]

And how is this different for smaller projects?

In my professional career I have been involved with at least 50 or 60 medium scale ($20-$200 million) energy projects and I'd guess about 1 in 5 or 10 runs over budget, and then it's rare to see more than a 20% overrun.

It's not unusual for them to come in below cost as your typical project engineer will tack on a 20% contingency versus what they really estimate it will cost.

It's worth noting that with these sort of projects you are typically doing something that is fairly commonplace, not doing something that is new and different.

This project is a great example of that particular problem:

http://www.popularmechanics.com/tec...e-tunnel-boring-machine-still-stuck-17517311/

 

[Jimmy Higgins]

And how is this different for smaller projects?

In my professional career I have been involved with at least 50 or 60 medium scale ($20-$200 million) energy projects and I'd guess about 1 in 5 or 10 runs over budget, and then it's rare to see more than a 20% overrun.

It's not unusual for them to come in below cost as your typical project engineer will tack on a 20% contingency versus what they really estimate it will cost.

It's worth noting that with these sort of projects you are typically doing something that is fairly commonplace, not doing something that is new and different.

This project is a great example of that particular problem:

http://www.popularmechanics.com/tec...e-tunnel-boring-machine-still-stuck-17517311/

There is absolutely nothing new to that job. It got stuck on a well that wasn't discovered during design. And now they are running into other troubles that aren't exactly unknown. Granted, tunneling with a 57-ft diameter TBM isn't exactly common.

One of my favorite things about Oceans 13 was how quickly they mob'd the TBM from the Chunnel job. The mob and assembly time would have been a year and it'd probably cost them the take in the job.

 

[dismal]

In my professional career I have been involved with at least 50 or 60 medium scale ($20-$200 million) energy projects and I'd guess about 1 in 5 or 10 runs over budget, and then it's rare to see more than a 20% overrun.

It's not unusual for them to come in below cost as your typical project engineer will tack on a 20% contingency versus what they really estimate it will cost.

It's worth noting that with these sort of projects you are typically doing something that is fairly commonplace, not doing something that is new and different.

This project is a great example of that particular problem:

http://www.popularmechanics.com/tec...e-tunnel-boring-machine-still-stuck-17517311/

There is absolutely nothing new to that job.

Yeah, it's just your typical world record size boring machine that lays cement walls behind it which is being redesigned on the fly to be able to handle wet and unpredictable soils beneath a major coastal city.

Who could have imagined there'd be problems?

 

[untermensche]

It seems this author is optimistic because nothing is changing but Obama talks about it differently than previous presidents.

 

[Jimmy Higgins]

There is absolutely nothing new to that job.

Yeah, it's just your typical world record size boring machine that lays cement walls behind it which is being redesigned on the fly to be able to handle wet and unpredictable soils beneath a major coastal city.

I noted a 57-ft TBM is unusual, but it is hardly new science. The hardest part of a 57-ft TBM is finding one, i.e. they had to build one for the project.

Unpredictable soils? There is a field in engineering that deals with the soils and rocks. It was included with this project. They didn't just get someone to build a 57-ft diameter TBM and then plow it into the ground and hope for the best.

Who could have imagined there'd be problems?

The anticipated problems would be noted in the GBR.

 

[dismal]

Yeah, it's just your typical world record size boring machine that lays cement walls behind it which is being redesigned on the fly to be able to handle wet and unpredictable soils beneath a major coastal city.

I noted a 57-ft TBM is unusual, but it is hardly new science. The hardest part of a 57-ft TBM is finding one, i.e. they had to build one for the project.

Unpredictable soils? There is a field in engineering that deals with the soils and rocks. It was included with this project. They didn't just get someone to build a 57-ft diameter TBM and then plow it into the ground and hope for the best.

Other articles I have read suggest the soils are a large problem. I assume you agree there is a problem?

That as routine as using a 57 foot boring machine that lays concrete walls behind it to tunnel under major coastal cities has become there is something not quite right about this particular project?

 

[Loren Pechtel]

And how is this different for smaller projects?

In my professional career I have been involved with at least 50 or 60 medium scale ($20-$200 million) energy projects and I'd guess about 1 in 5 or 10 runs over budget, and then it's rare to see more than a 20% overrun.

It's not unusual for them to come in below cost as your typical project engineer will tack on a 20% contingency versus what they really estimate it will cost.

It's worth noting that with these sort of projects you are typically doing something that is fairly commonplace, not doing something that is new and different.

This project is a great example of that particular problem:

http://www.popularmechanics.com/tec...e-tunnel-boring-machine-still-stuck-17517311/

And megaprojects are normally doing something new. Of course overruns are more likely.

 

[Axulus]

In my professional career I have been involved with at least 50 or 60 medium scale ($20-$200 million) energy projects and I'd guess about 1 in 5 or 10 runs over budget, and then it's rare to see more than a 20% overrun.

It's not unusual for them to come in below cost as your typical project engineer will tack on a 20% contingency versus what they really estimate it will cost.

It's worth noting that with these sort of projects you are typically doing something that is fairly commonplace, not doing something that is new and different.

This project is a great example of that particular problem:

http://www.popularmechanics.com/tec...e-tunnel-boring-machine-still-stuck-17517311/

And megaprojects are normally doing something new. Of course overruns are more likely.

There is a reason to expect more variability from the budget (i.e. the initial estimate would be more likely to be off), but there is no particular reason at first to think such estimates should always be biased much more to _underestimate_ the costs, consistently in time and in geography. If unbiased methods were used, we would expect such projects to be under budget just as often as they are over budget.

 

[Jimmy Higgins]

I noted a 57-ft TBM is unusual, but it is hardly new science. The hardest part of a 57-ft TBM is finding one, i.e. they had to build one for the project.

Unpredictable soils? There is a field in engineering that deals with the soils and rocks. It was included with this project. They didn't just get someone to build a 57-ft diameter TBM and then plow it into the ground and hope for the best.

Other articles I have read suggest the soils are a large problem. I assume you agree there is a problem?

There is a difference between problematic subsurface conditions and those conditions being unforeseen or not tunneled before.

That as routine as using a 57 foot boring machine that lays concrete walls behind it to tunnel under major coastal cities has become there is something not quite right about this particular project?

They are tunneling large diameter tunnels in many cities across the country. The 57 foot diameter is the largest, but 30+ foot isn't exactly unheard of.

 

[arkirk]

Mega-projects seem to have as their Genesis pre-existing mega-problems, sometimes multi-mega-problems. While they might have huge cost overruns and less than ideal over-time outcomes, by and large, once they are built, they eventually prove their worth if they function at all. There are subways all over the world in major cities and these cities could not function as they do today without them. If you want to evaluate them, you have to look at what the alternatives would be. In some cities, the cost of not having it would be less people living and working in that city. Those costs, emigration and relocation of people and businesses, are seldom considered once they do not have to be faced. They nevertheless would be there if there were no mega-project.

It is truly a monstrous task to adequately evaluate these structures because there are so many factors playing into them. I remember BART when it was just an idea being pursued for financing. As ugly as its overruns were, it is a part of bay area infrastucture that has at times provided a transportation safety net...particularly the transbay tunnel.

 

[Loren Pechtel]

And megaprojects are normally doing something new. Of course overruns are more likely.

There is a reason to expect more variability from the budget (i.e. the initial estimate would be more likely to be off), but there is no particular reason at first to think such estimates should always be biased much more to _underestimate_ the costs, consistently in time and in geography. If unbiased methods were used, we would expect such projects to be under budget just as often as they are over budget.

The estimates are always optimistic--pessimistic ones wouldn't get approved in the first place. Thus the variability will almost all be towards overrun rather than underrun.

Furthermore, in the case of an underrun the people involved will almost certainly find some way to use the money anyway so you won't get an underrun.

 

[barbos]

It applies to everything involving money transfer, you are assumed and expected to llie exaggerate.
If you are not doing that they would still think you are doing that.