Newsletter December 2017

Welcome to the December 2017 issue of “RiskIntegral” - an occasional newsletter about the analysis and management of risk, mainly in projects, by Risk Integration Management Pty Ltd (RIMPL). In this issue, we focus on the quest for realistic assessment of project time and cost contingency, as perceived by RIMPL MD, Colin Cropley

We end the newsletter with news of our activities during 2017.

Assessing Project Costs Realistically

Introduction

Last week a familiar yet disappointing infrastructure scenario was played out in the newspaper in our city. The paper announced that the a report of the State Auditor-General showed that the cost of eliminating 50 level crossings (roads crossing railway lines) by grade separation (rail under or over the road) had blown out from an initial estimate given by the governing party during an election campaign in 2014 of $5-6 billion has blown out to $8.3 billion (as of December 2017), an increase of at least 38% on the upper limit of the original estimate. The figure includes $1.4 billion for extra scope including new railway stations, power upgrades and track duplications.

This example illustrates the finding of a major Australian transport infrastructure study by the Grattan Institute published in 2016 titled “Cost overruns in transport infrastructure”. The study of the cost outcomes of all 836 transport projects valued at $20m or more completed since 2001 found that the main driver of cost overruns was premature announcement of projects, often in the lead-up to an election and usually prior to detailed estimating having been done. Other factors were identified by the study, which made three recommendations on improving cost estimating of major infrastructure projects:

• Cost estimates should reflect predictable patterns.

• Risk assessment should be comprehensive (the study listed four estimating methods, noting that official guidance on estimating methodologies is inconsistent).

• Risk assessment should be based on actual Australian data on past projects.

Infrastructure cost overruns had totaled $28 billion and were equivalent to nearly 25% of the total of the project budgets.

The sad reality is that similarly poor performance records exist in estimating time and cost outcomes of major and complex projects across all industry sectors and across the globe.

In response to this poor performance, RIMPL has directed a significant amount of effort over the last three years to addressing the causes of such overruns and under-estimates to improve understanding and thus future accuracy of project estimates.

In what follows, it is assumed that project cost estimates consist of a Base Estimate (in which the known scope is defined and costed) plus a provision for Contingency (representing scope not yet defined plus a provision for risk events that may occur during the project). There may also be an allocation for exchange rate uncertainty where multiple currencies are involved, plus an allocation for escalation in prices charged to the project where the project is of significant duration compared with prevailing escalation rates. These latter allocations are excluded from the following discussion as they are usually (but not always) less significant in cost overruns than the drivers of contingency, which are usually scope under-estimation and inadequate provision for risk. Bias of the estimate can also be a significant factor in projects, but this discussion also excludes estimate bias, while acknowledging its existence. For the rest of the discussion, we will focus on the assessment of contingency. We note that, especially in large or complex projects, the cost consequences of delay uncertainty and risk are usually large contributors to the final cost outcomes of projects.

Gaining a Global View - 2014

RIMPL has enjoyed success in Australia and nearby, having developed and applied its Integrated Cost & Schedule Risk Analysis (IRA) methodology from 2008 onwards. We felt ready to take our methodology further and in 2014 we presented papers at three international conferences and visited the risk management teams at the head offices of four major oil & gas companies and had the opportunity to present our IRA methodology and hear about their approaches. It became clear that all the companies used separate Schedule Risk Analysis and Cost Risk Analysis to assess time and cost contingencies, passing the time contingency from an SRA to a CRA, based on assumed cost per unit time. They did this, they said, because fully integrating cost and schedule in the one analysis was too hard and they could not rely on project teams providing good enough quality project schedules. We call this methodology Serial SRA to CRA, or SRA2CRA. While more difficult, IRA methodology does not sever the schedule drivers of cost from the cost drivers of cost, nor does it rely on assumed average “burn rates” of cost per unit time to produce its results. These problems were discussed in our April 2014 newsletter article: • Integration of Project Cost & Schedule Risk Analyses. Our conclusion from this experience was that we were in a minority in practising fully integrated Cost and Schedule Risk Analysis.

RES Contingency Guideline – 2015/16

RIMPL participated in the development of a guide to methods of assessing contingency through 2015 and its review early in 2016, contributing information on our Integrated Cost & Schedule Risk Analysis (IRA) methodology. This was produced by Engineers Australia and led by the NSW Chapter of the affiliated Technical Society, the Risk Engineering Society (RES).

It focused on methods in use or known in Australia and is a valuable contribution but omitted some important international experience, particularly from AACE International.

No mention was made of the importance of empiricism (past experience, cited by the Grattan Institute as desirable for improving estimates), nor of parametric methods for assessing contingency. We also concluded that no-one else in Australia was using contingency assessment similar to our IRA methodology.

Risky Publications & Plans in 2016

During 2016 I wrote a chapter on “The Case for Truly Integrated Cost & Schedule Risk Analysis” for an academically focused book: “Handbook of Research on Leveraging Risk and Uncertainties for Effective Project Management”, which was published at the end of 2016 by IG Global.

Among the 19 contributors were:

• John K Hollmann (First editor of AACEI’s Total Cost Management Framework [10 years of effort], editor of many Recommended Practices and an honorary life member of AACEI)

• Dr David T Hulett (Author of two seminal reference books on SRA, 2009 & ICSRA, 2011)

• Dr Yuri Raydugin (Author of Project Risk Management, 2013 & editor of the book)

Leading up to the 2016 AACEI Annual Meeting, I was increasingly concerned about the low level of awareness of best practice methodologies recommended by AACEI for assessing project time and cost contingencies:

• In Australia (as represented by the Contingency Guideline) and

• Internationally (as exemplified by the widespread use of SRA2CRA)

I wrote to a number of internationally recognised risk analysis authorities including the above three, suggesting an international conference focused on alternative risk-based methodologies for assessing project time and cost contingencies. Hollmann and Hulett supported the idea, suggesting having a special track at the AACEI Annual Meeting

In June 2016 I went to the 60th AACEI Annual Meeting in Toronto. At the end of the Decision & Risk Management (DRM) presentations, the DRM Technical Committee agreed to sponsor a special Contingencies Methodologies conference track at the 2017 Orlando Annual Meeting.

Publication of “Project Risk Quantification” by John K Hollmann in mid-2016

In July 2016, John Hollmann published a book of great significance: “Project Risk Quantification”, subtitled “A Practitioner’s Guide to Realistic Cost and Schedule Risk Management”. This is having a major influence on project estimating & risk analysis. This was reviewed at length in our December 2016 issue of RiskIntegral. Some reasons for its importance are:

• It makes the reader aware of the prevalence and dominance of systemic risk in projects, especially in relation to the quality of project delivery

• It emphasises the need to make use of past performance (empiricism) in forecasting, hence the cover image of the Roman God Janus who looks both backwards and forwards and is also the god of gates (through which projects must pass to be approved for execution)

• It reveals the importance of the hybrid parametric and expected value methodology in assessing contingency, using MCS without reference to critical path schedules

This book influenced my choice of topic for the 2017 AACE DRM Methodology Conference.

AACE International 2017 Risk Conference Contingency Assessment Theme and RIMPL Paper

In August 2016, AACE International invited submissions (~20) for a special track of presentations and a panel discussion on the theme “Project Cost and Schedule Risk Quantification: Alternative Methods”. 12 accepted papers were presented at the 61st Annual Meeting of AACEI in Orlando Florida in June 2017.

Referencing AACE International’s Recommended Practice RP 40R-08 “Contingency Estimating – General Principles”, the papers were to show how the leading methods addressed the various attributes. A closing panel discussed the updating of RP40R-08 and the attributes of the various methods but did not rank or directly compare them. An update of RP40R-08 is expected to be produced following the conference.

I presented a paper titled “Modelling Realistic Outcomes using Integrated Cost and Schedule Risk Analysis”. This paper reported on an IRA model of design and construction of a tollway. It extended the IRA model to include operation of the tollway for its economic life (assumed to be 25 years). By use of probabilistic cashflows, probabilistic NPVs and IRRs can be generated covering the whole investment and operating life cycle. Uncertainties and risk events affecting revenue and operating costs can thus be added to the uncertainties and risk events covering design, procurement and construction and the whole lifecycle of project plus asset can be modelled. This then allows broad scenarios to be modelled and the resilience of the investment proposal to be tested.

It is important to keep in mind that the cause of cancellation of many Mining and O&G projects since 2012 has not been rising project costs, but dramatically falling prices and thus revenue streams for the yet to be constructed asset. Conventional CRA and SRA are unable to test such vulnerabilities.

Also included in the model and paper was the use of macros in the Monte Carlo Simulation (MCS) modelling. This was to address the objection of John Hollmann to Critical Path Method based MCS assessments of contingency that such models could not adjust to the occurrence of a risk event. In this Public Private Partnership Tollway model a macro tested, in each iteration, whether a decision had been made by the state government to build a commuter railway line parallel to the tollway. The railway decision was set by probability. If the railway were to be built, the macro stopped the construction of additional lanes to the tollway. The macro also determined when extra lanes would be built in response to increasing use of the tollway through revenue increases.

The model also included the use of parametric modelling of systemic risk in time and cost as included in Hollmann’s Project Risk Quantification book and modified by RIMPL. This addressed Hollmann’s other criticism of CPM-based modelling: that it was not empirically based and did not model systemic risk. The paper is available for download on our website

Sydney Project Controls Conference 2017

In October 2017, Engineers Australia through the NSW chapters of its technical societies Risk Engineering Society (RES) and Australian Cost Engineering Society (ACES) ran a very successful Project Controls Conference, PCC 2017. The conference included several streams covering the usual branches of project controls.

I gave a presentation reporting on the AACE International “Project Cost and Schedule Risk Quantification: Alternative Methods” conference track, to ensure that the range of methodologies covered in the Orlando conference would be made known to an Australian audience.

The presentation is also available for download on our website

RIMPL NEWS

Quantitative Risk Analyses

Since our last RIMPL Newsletter, the following activities in 2017 have kept us active in our core business:

• SRAs were performed for Dulux, focusing on commissioning and startup as the new water-based paint manufacturing facility being built by Dulux on the northern outskirts of Melbourne neared completion. The plant has started up successfully and Dulux is very happy with the quality of the pain produced.

• Expert witness services have been provided focused on the assessment of time and cost contingencies for a megaproject nearing completion.

Project Controls & Contract Administration Review Services

We have provided project planning, cost control, risk management and contract dispute resolution advisory services for infrastructure projects and also provided a further review of contract administration performance for a Victorian Municipality:

• Provision of risk management services to the Victorian Government for the High Capacity Metro Trains project through 2017 via service provider Indec Ltd.

• Provision of planning, cost control, contract and risk management services for a marine infrastructure project being managed by Argonautica Pty Ltd for a major logistics company.

• Detailed contract administration review of further community projects for a Victorian Municipality.

RIMPL Representation of John K Hollmann in Australia and New Zealand

Since the Sydney Project Controls Conference, PCC 2017, representatives from a major Australian mining company have initiated discussions with RIMPL on assessing time and cost contingencies for sustaining capital projects. Arising from our proposing the use of Hollmann’s Hybrid Parametric + Expected Value methodology, detailed discussions have occurred which are expected to continue in the new year.

John Hollmann has appointed RIMPL to represent him in Australia and New Zealand and an MoU has been signed between the respective companies. We are looking forward to making more people and organisations aware of the advantages of this methodology for assessing time and cost contingencies.

CONTACT US

18 Cottesloe Court
Doncaster East VIC 3109
AUSTRALIA
+61 412 031 161
[email protected]