Why big IT projects crash; The history of embarrassing failures offers clues to where things tend to go wrong

September 18, 2013 8:13 pm

Why big IT projects crash

By Henry Mance

There are several ways the US Air Force could have wasted $1.1bn. It could have poured tomato ketchup into 250m gallons of jet fuel or bought a sizeable stake in Bear Stearns. Instead it upgraded its IT systems. Work began in 2007 to reconfigure how the force managed its logistics, with the aim of replacing 200 dated networks with a single piece of Oracle software. By the time the project was abandoned last November, it was at least four years behind schedule and would have required an additional $1.1bn to become usable. Yet in making such mistakes, the Air Force is not flying solo. This week a UK parliamentary watchdog described a failed National Health Service patient IT programme – the cost of which has spiralled to £9.8bn – as “one of the worst and most expensive contracting fiascos in the history of the public sector”. Earlier this month the Department for Work and Pensions admitted that it had written off £34m of IT costs, incurred in an attempt to overhaul how social security benefits are paid. A week earlier Co-operative Bank said it had written off the £148m cost of a new IT system that would no longer be implemented.

“It is quite scary,” says Ralf Dreischmeier, the global head of Boston Consulting Group’s IT practice. “From my experience, 20 per cent of projects fail, and 40-50 per cent have a cost overrun, time overrun or don’t meet requirements. Only a third could be described as good projects.”

Why are companies and governments still suffering such embarrassing failures?

In a 2011 study, Bent Flyvbjerg and Alexander Budzier at Oxford university’s Saïd Business School examined 1,471 IT projects against their forecast costs and overruns. They found that the projects exceeded their budgets by an average of one-quarter.

“Over the past decade there have been no improvements even though a lot of things have been tried,” says Mr Budzier. The researchers posited that planners consistently underestimated the costs and overestimated the benefits of IT projects. They also failed to appreciate the “black swan” scenario – that is, the chance that something will go really wrong.

Citing Nicholas Nassim Taleb, author of the book The Black Swan, the researchers argue that “the high over-incidence of black swans underlines that ICT projects are a very important source of uncertainty in an organisation”. In one in six projects examined by Prof Flyvbjerg and Mr Budzier, the cost was at least triple what had been estimated. When Hershey’s, the chocolate maker, implemented a new ordering system, it ended up missing out on a whole Halloween of sweet sales, worth $100m.

What have we learnt from past mistakes?

Make realistic estimates
“People always underestimate the cost of software development. Suppliers always push their prices,” says John Fotheringham, a partner at Deloitte.

Keep it short
Longer projects tend to see a higher turnover of personnel and a greater likelihood of a change in objectives. “A reasonable recommendation would be to try to complete a project within 18 months,” says Alexander Budzier of Oxford university’s Saïd Business School. The state of South Australia recently proposed only commissioning IT projects lasting less than 90 days. Yet Mr Budzier adds that there is no statistical correlation between the value of an IT project and its success.

Everyone loves ‘agile’ 
Up to four-fifths of new IT projects are now implemented using agile methods, whereby pieces of the project are implemented quickly then improved if necessary. This can, however, require more active management.

Your project is not different
“If the project manager thinks this is a unique project, then it’s going to explode,” says Mr Budzier. “And there’s a clear reason – they don’t benchmark themselves.”

Although much criticism has been directed at civil servants, IT overruns are present in both the private and public sector.

“The private sector is just much better at hiding these things,” says Mr Budzier. He points out that large blue-chip companies continue to operate failing IT systems because they are unwilling to write down the expense.

Drawing on the work of Daniel Kahneman, the Nobel Prize-winning behavioural economist, Prof Flyvbjerg and Mr Budzier call for companies to forecast their costs better by comparing their projects to previous experience.

The problems do not end with poor forecasting. Organisations are often very ambitious in what they set out to achieve – designing long projects that require customised software. That is likely to be another mistake.

In a report published this year, the Texas state auditor’s office examined 13 IT projects, nine of which had overrun. It concluded, admittedly on a small sample, that agencies using commercial off-the-shelf technology “exceeded their budgets by a smaller amount and took less time to complete their projects” than those that did not.

There is also an unwillingness to admit to failings when they occur. In the US Air Force case, the can-do attitude of some military officials meant concerns were not raised. “Program managers fear that an honest delivery of program status will result in cancellation,” noted an independent progress report into the project.

Poor communication – particularly between business and technical experts – is a constant problem. “Corporations are organised in hierarchies, in line roles,” says Mr Dreischmeier. “Doing projects is different. Projects go across silos.”

Boston Consulting Group argues that a project must have clear objectives, an understood business case, good governance, clear deliverables and an agreement planning structure.

In recent years companies have laid the blame for IT failures at the door of their contractors. The IT industry has sought to borrow methods from the construction industry, where suppliers incur penalties for late delivery. However, IT has special challenges.

“You’ve got lots of potential excuses for things being not your problem,” says John Fotheringham, a partner at Deloitte, the professional services firm. Suppliers can blame difficulties on integration with existing IT systems. “They are very good at extracting extra money,” he says. Clients can heighten this risk by changing a project’s specifications after it has started.

Some suggest that more time should be spent on the design of IT projects. “A transport project is often planned for five years,” says Mr Budzier. “With IT there is very little upfront planning.”

Yet getting that right can also be problematic. Unlike road technology, software changes very fast.

Boston Consulting Group’s Mr Dreischmeier also warns that too much time can be spent on design. “I’ve seen a project implementing a global finance system, and you’re just doing design work for six months,” he says. “Then, when you put it together, it doesn’t work.”

An emphasis is now being placed on “agile” methods, whereby work is broken up into small chunks and the design is not considered complete before the work is started.

Those methods, which were used in the Department for Work and Pensions’ benefits overhaul, have the advantage of trial and error. Yet they also can require greater oversight.

Nonetheless, not everyone believes that such soul-searching is helpful. “If reports keep focusing on failures, we risk IT being seen as purely a cost and something that needs to be cut,” says Georgina O’Toole, an analyst at TechMarketView.

The focus on budget and cost overruns may obscure the wider benefits of a project. “You need a way to evaluate whether your overall objectives have been met,” says Larry Maccherone of Rally Software.

Some observers suggest that cloud computing, which requires less on-site hardware, will simplify future projects, rendering failure less common. Executives are also more technologically literate now, and increasingly aware of the history of failure. “There are certain changes that make me fairly optimistic that some things will be better,” says Mr Dreischmeier.

For the moment, however, that is hope over experience.

About bambooinnovator
Kee Koon Boon (“KB”) is the co-founder and director of HERO Investment Management which provides specialized fund management and investment advisory services to the ARCHEA Asia HERO Innovators Fund (www.heroinnovator.com), the only Asian SMID-cap tech-focused fund in the industry. KB is an internationally featured investor rooted in the principles of value investing for over a decade as a fund manager and analyst in the Asian capital markets who started his career at a boutique hedge fund in Singapore where he was with the firm since 2002 and was also part of the core investment committee in significantly outperforming the index in the 10-year-plus-old flagship Asian fund. He was also the portfolio manager for Asia-Pacific equities at Korea’s largest mutual fund company. Prior to setting up the H.E.R.O. Innovators Fund, KB was the Chief Investment Officer & CEO of a Singapore Registered Fund Management Company (RFMC) where he is responsible for listed Asian equity investments. KB had taught accounting at the Singapore Management University (SMU) as a faculty member and also pioneered the 15-week course on Accounting Fraud in Asia as an official module at SMU. KB remains grateful and honored to be invited by Singapore’s financial regulator Monetary Authority of Singapore (MAS) to present to their top management team about implementing a world’s first fact-based forward-looking fraud detection framework to bring about benefits for the capital markets in Singapore and for the public and investment community. KB also served the community in sharing his insights in writing articles about value investing and corporate governance in the media that include Business Times, Straits Times, Jakarta Post, Manual of Ideas, Investopedia, TedXWallStreet. He had also presented in top investment, banking and finance conferences in America, Italy, Sydney, Cape Town, HK, China. He has trained CEOs, entrepreneurs, CFOs, management executives in business strategy & business model innovation in Singapore, HK and China.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: