Skip to main content
Analysis_

Why do most major projects fail and what can we do about it?

Analysing research on major projects

Dr Maurizio Floris and Errol Benvie review the array of causes of major project failure and consider different ways to review projects

Another day, another project over budget, over time or failing to deliver the expected outcomes. The Forrest Highway in WA and the Hunter Expressway in NSW both cost over five times the initial forecasted amounts1.

Major projects fall short across all domains, whether it be technology, infrastructure, resources or mergers and acquisitions. Suggested ‘failure’ rates for major projects typically vary between 50 and 90 percent2.

Success appears to be the exception to the rule, particularly for bigger and more complex projects. Research suggests that we have made little to no improvements over the years. What are the main causes for project failure and what can we do about it? Might we be more successful if we looked at major projects differently?

Why do major projects fail and what can we do about it?

A review of research (see Table 1) suggests an array of causes for project failure. A project that takes many years to complete may be particularly sensitive to its external context, for example, to market fluctuations of project inputs, outputs or currency. We saw this during the recent resources boom, when projects were impacted by escalating labour costs. Newly graduated mining engineers could command US$100,000+ salaries, as could a kitchen hand on a FIFO (fly-in fly-out) contract in a remote or off-shore location. While these causes are external, risk management may offer some protection, including through insurance, hedging, long-term contracts and project stage gates.

Other causes may be inherent to projects with numerous delivery partners, skill sets required, new designs or interdependent elements. The risk is exacerbated  by high political, reputational or investment stakes. All of these contribute to risks that impact the scope, timeline, costs or value of the project for different stakeholders. Breaking complex projects up into smaller projects may reduce the risk, with each smaller project delivering part of the requirements. Further, more agile teams can respond quickly to changes and incorporate emerging understanding to shape the project.

Interestingly, research suggests that project failure is not often attributed to technical issues. Technical issues may typically be prevented by leaders who ask critical questions early enough, or by addressing overconfidence that technology will work or that ground conditions are adequate. Overconfidence or delusion has plagued the success of many projects and is a major root cause of project failure. It applies across all aspects of a project (including tasks, scope and capability) and accumulates as underestimations of costs or timelines take a toll. Remedies for delusion include comparing a project to similar projects before approving the project, reviews by peers or external experts, identifying a broader outcome range and changing estimates to compensate for bias.

Deception occurs when important information is knowingly misrepresented or withheld. Prematurely announcing a major project that has not yet been adequately scoped can contribute to its failure.  This is most likely when there is a conflict of interest between planners, project estimators, contractors, financiers and politicians who might benefit from project approval even if the government as a whole or the community would not. Remedies to this problem include transparency and alignment of interests. Another solution is for customers, financiers and contractors to share the pain of recontracting if required.

A lack of leadership potentially describes the most important root cause of project failure. Arguably, many of the causes of project failure mentioned earlier could have been prevented by more effective leadership that asks the right questions about external context, verifies technical challenges and capabilities upfront. Effective leadership addresses deception courageously and guides the project and its stakeholders through the politics that unavoidably come with major projects.

According to specific recommendations in the project literature2, this requires influencing and aligning stakeholders, building an effective delivery team, non-adversarial communication and genuine dialogue, and effective project governance. Further, many of these leadership challenges can only be resolved systematically by developing a culture that nurtures its project talent, values organisational learning and understands the difference between managing and leading of projects.

Are we looking at it the wrong way?

Many of these causes and remedies are not new. So why have projects continued to fall short? One argument is that much of the research into project overruns simplistically focuses on a single cause or remedy. Instead, project failure must be addressed through systems-thinking that considers the interactions between multiple factors, interventions and sense-making from the perspective of different stakeholder positions.

A second argument is that defining project success or failure through a framework of cost, time and specification is too narrow. What may end up being delivered is likely to be quite a different project to what was initially envisioned. If the final project delivers sufficient value then should it not be considered successful?

Of course, questions remain including how we define value (for example social, economic, environmental and reputational), and who is the beneficiary? It is likely that a project has both stakeholders who win and others who loose.

A third argument may relate to how remedies are implemented. Harvard Professor Heifetz highlights that organisations often address the technical issues but fail to address the ‘adaptive’ part of a challenge. This means that ingrained beliefs, values, perspectives and patterns of behavior of individuals or organisations are not addressed; rather, they are merely treated superficially. It is far easier to implement a superficial fix but more often than not this falls short of the intended outcome.

Lastly, it is conceivable that we have actually improved our ability to deliver projects but that this has lead to undertaking more ambitious projects instead of lowering perceived failure rates.

We have a long way to go before we are likely to see project success as the rule and not the exception.

Table 1: Causes and remedies for project failure – what does the research tell us

Category

Causes

Remedies

External context

  • Market conditions
  • Material/labour cost
  • Demand uncertainty
  • Currency fluctuations
  • Unforseen events
  • Sensitivity analysis
  • Long term contracts
  • Hedging
  • Insure against selected events
  • Avoid projects with significant external risk

Project charac-teristics

  • Project complexity
  • Scope clarity or changes
  • Non-standard design
  • Length of implementation phase
  • Inefficient procurement practices, contract types/ ambiguity, or poor documentation
  • Rework
  • Simplify or modularize the project
  • Flexible development models
  • Design reviews, checks, verifications
  • Shift from competitive tendering processes
  • Organizational learning
  • Understand contract terms and claim causes
  • Integrated knowledge management systems
  • Measure and prevent rework

Technology

  • Technological uncertainty, or technical difficulties
  • Non-standard technology
  • New technology
  • Unexpected geological features
  • Ground conditions
  • Front-end loading (i.e. detailed design/ planning during the earliest stages of a project)
  • Pilot projects
  • Adequate sampling
  • Technical skill acquisition across team
  • Future proofing

Psychological (cognitive) bias

  • Delusion
  • Planning fallacy (over optimism)
  • Anchoring and adjustment
  • Hubris
  • Sensitivity analysis for overly narrow ranges
  • Comprehensive, rigorous decision process
  • Reference Class Forecasting and bias uplift
  • Good learning environment
  • Independent Peer or Expert Review
  • Broader range for sensitivity analysis, and probable outcome range (not single point)

Political and economic factors

  • Deception
  • Misaligned interests
  • Corruption or foul play
  • Promoter “postpone” appearance of cost during project construction
  • Overprice scope increases
  • Pressure from politics, superiors or other organizational pressures
  • Misrepresentation may help obtain funding
  • Premature announcement
  • Failure of professional ethics
  • Tendering method
  • Different risk profile
  • Different time horizons
  • Asymmetric accountability or information
  • Accountability, incl. criminal penalty
  • Transparency, (e.g. incentives for planners to disclose project information and assumptions; public hearings and presentations of forecasts to community)
  • Governance structures & procedures (e.g. strict audits of costs and benefits)
  • Alignment of incentives (e.g. proposing and approving institutions share financial responsibility; private financiers participate with own capital at risk; no sovereign guarantee; contractors share pain for delays and scope; incentives for planners and bidders for forecasting accuracy) 
  • Alliance contracting
  • Shared purpose statement
  • Interest-based negotiation

Leadership

  • Poor relationship management/ communication
  • Managerial incompetence
  • Overestimation of ability and control
  • Lack of learning/ feedback from previous projects
  • Cultural differences’
  • Misaligned organizational structures
  • Stakeholder alignment & collaborative engagement
  • Early non-adversarial communication and genuine dialogue
  • Select, retain and develop project leaders.
  • Courage
  • Organizational learning and feedback mechanisms
  • Flow of information/ interface management
  • Organizational structure redesign

Sources:

1 Marion Terrill, Grattan Institute (2016), Cost overruns in transport infrastructure.

2 References to key academic research can be obtained https://www.dropbox.com/s/9cbyf7zm2lxrz8i/JGC%20Examples%20of%20research.pdf?dl=0


Authors:

This article is written by Dr Maurizio Floris, Program Director, John Grill Centre for Project Leadership and Errol Benvie who is a John Grill Centre Executive Coach.