Sunday, August 21, 2011

Now Here Is An Article To Give Serious Pause To The Approach Being Adopted With the PCEHR.

Saturday Extra came up with a great report yesterday.

Huge IT cost blow-outs

A study at Oxford University has found that a significant number of major IT projects are 'ticking time bombs' in corporate accounts, they are twenty times more likely than normal projects to spin out of control and the survey data suggests it is almost inevitable that one or more established companies will be brought down by an IT project in the foreseeable future.

Guests

Bent Flyvbjerg
BT Professor and Founding Chair of Major Programme Management at Oxford University's Said Business School

The full page is here:

http://www.abc.net.au/rn/saturdayextra/stories/2011/3297766.htm

(Note the audio links (about 15 mins) just above the title.)

There are a couple of interesting comments that are also available.

The first one has a great first paragraph:

Mark Toomey :

20 Aug 2011 9:52:21am

“This is a long term problem and the findings of the research, while depressing, are consistent with many prior studies. The IT industry has been trying to solve the problem for 20 plus years, through frameworks and tools for management of the projects, but this has had little success. There are many well-documented Australian cases that are in the public domain such as when the Australian Customs Service introduced a new system on 12 October 2005 that effectively blocked almost all imports for more than three weeks, plus the more recent MYKI and HealthSmart in Victoria, TCard in New South Wales and Queensland Health Payroll.”

I agree we have been trying to figure out the root causes and do better with a seemingly intractable problem for a good number of years. There are some wonderful books from the 1990’s entitled things like ‘Computing Calamities’ and ‘Software Runaways’!

I also agree governance of these projects is absolutely crucial for success and for the PCEHR not to have this clearly developed and articulated by now vastly imperils any chance of success.

The source article is freely available from the Harvard Business Review.

Here are the first few paragraphs:

Why Your IT Project May Be Riskier Than You Think

by Bent Flyvbjerg and Alexander Budzier

To top managers at Levi Strauss, revamping the information technology system seemed like a good idea. The company had come a long way since its founding in the 19th century by a German-born dry-goods salesman: In 2003 it was a global corporation, with operations in more than 110 countries. But its IT network was antiquated, a balkanized mix of incompatible country-specific computer systems. So executives decided to migrate to a single SAP system and hired a team of Deloitte consultants to lead the effort. The risks seemed small: The proposed budget was less than $5 million. But very quickly all hell broke loose. One major customer, Walmart, required that the system interface with its supply chain management system, creating additional hurdles. Insufficient procedures for financial reporting and internal controls nearly forced Levi Strauss to restate quarterly and annual results. During the switchover, it was unable to fill orders and had to close its three U.S. distribution centers for a week. In the second quarter of 2008, the company took a $192.5 million charge against earnings to compensate for the botched project—and its chief information officer, David Bergen, was forced to resign.

A $5 million project that leads to an almost $200 million loss is a classic “black swan.” The term was coined by our colleague Nassim Nicholas Taleb to describe high-impact events that are rare and unpredictable but in retrospect seem not so improbable. Indeed, what happened at Levi Strauss occurs all too often, and on a much larger scale. IT projects are now so big, and they touch so many aspects of an organization, that they pose a singular new risk. Mismanaged IT projects routinely cost the jobs of top managers, as happened to EADS CEO Noël Forgeard. They have sunk whole corporations. Even cities and nations are in peril. Months of relentless IT problems at Hong Kong’s airport, including glitches in the flight information display system and the database for tracking cargo shipments, reportedly cost the economy $600 million in lost business in 1998 and 1999. The CEOs of companies undertaking significant IT projects should be acutely aware of the risks. It will be no surprise if a large, established company fails in the coming years because of an out-of-control IT project. In fact, the data suggest that one or more will.

We reached this bleak conclusion after conducting the largest global study ever of IT change initiatives. We examined 1,471 projects, comparing their budgets and estimated performance benefits with the actual costs and results. They ran the gamut from enterprise resource planning to management information and customer relationship management systems. Most, like the Levi Strauss project, incurred high expenses—the average cost was $167 million, the largest $33 billion—and many were expected to take several years. Our sample drew heavily on public agencies (92%) and U.S.-based projects (83%), but we found little difference between them and projects at the government agencies, private companies, and European organizations that made up the rest of our sample.

The full article is found here:

http://hbr.org/2011/09/why-your-it-project-may-be-riskier-than-you-think/ar/pr

It is important to note that a large proportion of the projects examined are public sector driven just like the PCEHR is.

Of considerable value I think is the second last paragraph:

“Even if their companies pass the stress test, smart managers take other steps to avoid IT black swans. They break big projects down into ones of limited size, complexity, and duration; recognize and make contingency plans to deal with unavoidable risks; and avail themselves of the best possible forecasting techniques—for example, “reference class forecasting,” a method based on the Nobel Prize–winning work of Daniel Kahneman and Amos Tversky. These techniques, which take into account the outcomes of similar projects conducted in other organizations, are now widely used in business, government, and consulting and have become mandatory for big public projects in the UK and Denmark.”

You can read more on “reference class forecasting” here:

http://en.wikipedia.org/wiki/Reference_class_forecasting

If this technique were applied to the PCEHR I guess we would think of the UK NHS Program and the Kaiser Permanente Program as possible comparators among others. On that basis we have not seen anything yet in terms of either cost or time blowouts to actually useful system delivery.

I can only assume the boffins at DoHA are fully across these sort of major project estimation and risk mitigation approaches and that we can all relax.

I sure hope so!

It also seems to me, on the basis of what is written here, that to be steaming along on a totally innovative approach without real proof on concept implementations is just pain foolhardy. The PCEHR is a large and complex project and just should not be rushed at a political whim. If we do the outcome is virtually inevitable - a big mess.

I guess we shall all see - and pretty soon now!

David.

In passing can I add my personal condolences to both the families of the 'Lake Eyre Three' and the family of Ian Carroll - and especially Geraldine Doogue (who hosts Saturday Extra) - in these difficult times. We really forget just how important these people are in our lives.

D.

4 comments:

B said...

The most common reason for IT projects failing is because they are viewed as technology projects.

If they were treated as Information Systems, there would be more emphasis on the information, not the technology; on the problem, not the solution. This lack of understanding leads to bad requirements which results in failed projects - Deep Throat said "follow the money". In the case of IT projects the equivalent is - "follow the information".

In the case of the PCeHR, there is a fundamental (and bad) decision that has unconsciously been made about the eHealth Record.

It has been assumed that the eHealth Record is a single, monolithic object. The underlying data model (from Oracle) supports this.

Unfortunately, it is highly likely that health information should be much more granular to allow for better access control and more efficient handling.

This decision will lead to complaints of "unclear requirements", "changing requirements", scope issues, privacy problems etc, etc and a project that fails to deliver the hyperbole being spouted by the uninformed, and badly advised.

Anonymous said...

Governance is self-serving motherhood designed to reassure everyone. Through its Chair and its Directors (Heads of all the Health jurisdictions) NEHTA can be assumed to have an understanding of governance and even, no doubt, able to demonstrate they have implemented a form of governance.

BUT - SO WHAT?

The point is this:

It's not Governance that's needed it is COMPETENT, HONEST, OPEN GOVERNANCE that is required. A preparedness to ask the hard questions, to dig deep, to monitor, to audit, to come out of the wood-work unannounced, to have the power to say "no, stop, fix that", to pull the plug, to start again, to identify and admit where errors of judgement and mistakes have occurred, to monitor the organization's culture, and above all to be DELEGATED with sufficient 'POWER and AUTHORITY' to be able to ACT and ensure problems get fixed and not just be a self-serving impotent committee established to make the ostriches feel comfortable.

Anonymous said...

Some bi-partisanship on an E-Health framework would of been nice.

Anonymous said...

Monday, August 22, 2011 9:29:00 AM nails it.

However, if you have a Governance Committee of the Board and management feeds it falsehoods the government committee is none the wiser. And as Board committees do not have the right to interfere in the work of management the governance committee is somewhat powerless.

For that reason complex IT projects must implement a form of governance which is more than just a steering committee or a Board Committee. The monitoring body (call it what you will) must be given some teeth to monitor delivery and exert some serious influence when things are clearly going off the rails.