Sunday, July 22, 2012

Those Undertaking The Development and Implementation of The NEHRS Were Always Kidding Themselves. Here is Some Important Evidence About What Went Wrong From The Outset.

For some time I have been mulling about the issue of IT Project complexity and what it means for large projects in general and the NEHRS is particular.
My interest was spurred by the following short quote from a presentation on top IT trends:

Gartner's Top 10 Tech Trends Through 2015

I.T. Complexity

Pointing to Glass's Law (sourced to Roger Sessions of ObjectWatch), which states that "for every 25 percent increase in functionality of a system, there is a 100 percent increase in the complexity of that system," Gartner predicts there will be an emphasis on the ability of an enterprise to get the most out of I.T. money spent.
Read about the other 9 here:
Reading this made me think of the PCEHR Con-Ops and just how many function points there were and how this must have inevitably weighed the Program down with crushing complexity.
Looking for more information of Glass’s Law led me to this really awesome presentation from Roger Sessions himself.
From his blog we have:

Friday, January 6, 2012

Web Short: The Relationship Between IT Project Size and Failure

Large IT projects have bigger budgets than small projects. They also have more formal approaches to tracking milestones and budgets. Does this make them more likely to succeed? It turns out that there is a relationship between the cost of a project and the chances that that project will succeed, but it isn't what you might think.
Watch the presentation and then leave a comment.
You can see a full list of Roger's Web Shorts here.
Would you like to be notified of future Web Shorts and White Papers by Roger Sessions? Sign up here.
And thanks to AuthorStream for hosting this presentation.
This presentation includes narration, so be sure to have your speakers on.
You can (and should I reckon) view the whole presentation here:
A few points from it:

Grim Statistics: 

An estimated 85 percent of government IT projects are late, over budget or both. - The Pew Center on the States, “Focus on Performance”, 2010
The presentation then goes on to define project success:

IT Project Success Criteria

1. Accurate determination of business needs.
2 . Accurate determination of optimal schedule.
3. Accurate determination of optimal cost.
4. Delivery of all of the above.
And finally for my purposes then goes on to point out that the evidence on what is too big for success is really alarming:
There are two recent studies cited:
(Standish) 2009 Chaos report published by the Standish Group
and
(Sauer) The Impact of Size and Volatility on IT Project Performance by Chris Sauer, Andrew Gemino , and Blaize Horner Reich, Comms of the ACM Nov 07
In broad terms what both studies concluded based on what actually happened in the real world was:

Sessions’ Law of Size IT

  • Projects less than $1M will probably succeed, regardless of how poorly managed they are. 
  • IT Projects between $1M and $10M can succeed, if they are very carefully managed.
  • IT Projects over $10M will probably not succeed, regardless of how well managed they are.
The presentation then goes on to discuss the issues of project breakdown, dependencies, timing and so on - and how this is critical.
What can be concluded from all this:
Firstly you need to get your requirements right - I would not say this was done - and then with something like the PCEHR you need very expert optimisation approaches to make the whole thing workable and manageable - if indeed you are still wanting to go ahead at scale - rather than at much greater simplicity.
For those with the stomach this article makes very interesting reading:

How to Spot a Failing Project

Often, the difference between success and failure is spotting critical early warning signs that a project is in trouble. Here are a few ways to identify the symptoms.

By Rick Cook
Tue, July 17, 2007
Link is here:
These paragraphs or two says it all.

“Looking for Warning Signs

While the Chaos surveys give reason for optimism, they leave little room for complacency. Nearly one in five IT projects still fails absolutely and more than two in five are partial failures.

What is perhaps more troubling is that the bigger the project, the worse the problems. "Seventy-three percent of projects with labor cost of less than $750,000 succeed," Jim Johnson says. "But only 3 percent of projects a with labor cost of over $10 million succeed. I would venture to say the 3 percent that succeed succeeded because they overestimated their budget, not because they were managed properly."

and this reminds you of what among the warning signs?

A "No-Bad-News" Environment

"This is a really tricky cultural thing," says Raj Kapur, executive vice president of the Center for Project Management, a software project management consultancy and education firm in San Ramon, Calif. "Everyone is allergic to bad news." As a result, it's all too easy to develop a culture where bad news is slow to percolate upward—which deprives management of vital, if unpleasant, information.

"You have to provide an environment where bad news is accepted," says Kapur. "That's critical, and it's not the job of the team members. It's the job of the leader." And by extension, the CIO.”

None of this is all that new - but it does seem to have been rather forgotten by DoHA and NEHTA. The senior managers just should have known better and been more realistic about how they went about things. They were not and so here we all are!

David.

3 comments:

Cris Kerr said...

I'm not amongst those who believe the size or cost of a project is indicative of whether it will succeed or fail.

The devil is in the detail. If the detail that should be there at the outset is missing, then everything that comes after that can be very hit and miss and the bigger the project, the bigger the miss.

A plan should contain sufficient detail to empower every person at every level involved across all related areas/projects to analyse, plan and act in accordance with the plan, and if necessary, take corrective action to get back on plan.

Anyone who develops or effectively contributes to a good, detailed plan with measurable objectives and who has the authority to action that plan and achieve those objectives is happy to put their name on it.

But there is a trend where responsibility across poorly planned projects is divested in many, where reports and analyses don't bear author names or only bear a consultancy group's name... and where it's sometimes difficult to even find the date something was produced.

Anonymous said...

I think there needs to be at least one person, in a position of authority, who has their head around the overall problem and the design and technology of the solution for something to succeed. There is no one in NEHTA or DOHA in this position. There are people with deep knowledge of narrow areas, but no overview of how it all works together. The problem is they end up filling their blanks with the latest buzz word technology. So we have things like SMD needing a whole new PKI setup rather than using the expensive one that is already in place just so they can use the latest xml based technology in Visual studio. Meanwhile the rest of the world moves on and decides to use more sensible technologies that actually work!! Another example is allowing AMT to be built for pharmacy stock control while ignoring allergy checking and decision support... wtf!

Anonymous said...

It is my experience that in many state government health departments an IT related project will be multi-year and $10-20M in value, with a portfolio comprising up to $100M-$150M. The prevalence of Prince 2 and MSP does not in any way shape or form correlate to outcomes or success. Indeed, it gives the appearance of managing risk and previous failure by focusing ever more on process and additional layers / silos and wheels within wheels of project, program and portfolio. Thus ever increasing an organisation’s reliance on contractors and consultants. (Often to the extent that there are a small number of organisational resources committed to and overseeing activities).