This appeared in the Medical Journal of Australia today.
Why e-health is so hard
Enrico Coiera
Med J Aust 2013; 198 (4): 178-179.
doi: 10.5694/mja13.10101
We need to respect the basic rules of informatics and invest in e-health expertise
Medicine holds dominion in the microcosm of molecules and genes. It is in the macrocosm of people and organisations where things seem to fall apart. Modern health care appears unsustainable in its current form,1 and information technology is increasingly seen as a major intervention that can drive “reform”.
Evidence for e-health’s potential to improve the safety and quality of care grows,2 but remains patchy.3 The long list of disappointments and failures,4,5 locally and internationally, is also hard to ignore. There is a real dissonance in the discourse between what research evidence tells us is possible and what often happens with large-scale e-health projects in practice.6
The literature repeatedly describes basic “rules of informatics” for implementation success: the need for stakeholder engagement, culture change, user training, slow and considered implementation, and user-friendly systems that fit into clinical workflow.7 The very first rule of informatics tells us to start with the clinical problem we want solved rather than the technology we want to build.8 Yet, too often, large-scale e-health projects break this most basic rule, focusing on technology rather than compelling clinical problems.5 We are often told that national e-health projects must first lay down basic technical infrastructure and that high-value clinical systems will naturally follow, in the same way that laying railway lines is a precursor to delivering transport services.9 But railways can be too expensive, over-engineered, or not take us anywhere particularly useful — unless there is a destination on which we can all agree.
Why so many projects repeatedly fail to observe these basic rules of informatics remains a mystery, but it probably reflects that there are still very few people with deep expertise in e-health.10 Despite the crucial role of the informatics workforce in e-health success, and the billions spent on e-health over the past decade by government, barely a dollar has been in direct support of informatics workforce training.
E-health is hard because it is a complex intervention in a complex system.11 Indeed, e-health projects are probably among the most complex interventions we can undertake, especially at a national scale. The rules for designing e-health at the level of clinical practice are not the same as those at large scale, and the gap is as wide as that between in-vitro and in-vivo clinical studies. This explains why success at individual sites is no guarantee of success elsewhere.
…..
Just because e-health is hard does not mean we can ignore it and do something else instead. The goal is worthy, and alternatives are thin on the ground. We do, however, need to urgently invest in the informatics workforce, as this is no game for amateurs. We must also respect the basic rules of informatics. Like the laws of physics, they exist, whether you like them or not.
---- end article.
If you have access to the MJA go here:
If not, for the full article, references and so on you can e-mail Professor Coiera. He has kindly agreed to send a .pdf of the full article to all who ask. (Be gentle).
You can e-mail him at e.coiera@unsw.edu.au
Many thanks for making this article freely available to those who don’t have access to the MJA directly.
Also, an interesting paper of a NSW Emergency Department System is found here in the same issue here:
Effect of an electronic medical record information system on emergency department performance
Murugabalaji K Mohan, Rod O Bishop and James L Mallows
and there is commentary here:
Good HIT and bad HIT
Jon D Patrick and Susan Ieraci
Both these are freely available to read at the site! Thanks MJA!
Good to see so much e-Health in the MJA!
Good to see so much e-Health in the MJA!
David.
There is mounting evidence that these monolithic systems just don't provide the benefits the vendors sell with them. They cost a bomb, and it seems they don't improve anything. But Governments keep dipping into the purse (taxpayers purse) and hand over multiple millions of dollars. Where does it end?
ReplyDeleteRe the paper on an assessment of a NSW Emergency Department System
ReplyDeleteResults: We found a reduction in performance with respect to ED KPIs after implementation of the FirstNet system. There were increases in the waiting time for all patients (median, 40 min v 78 min), and the waiting time (median, 49 min v 87 min), treatment time (median, 128 min v 147 min) and total time (median, 214 min v 280 min) for patients discharged from the ED. There were increases in the DNW rate (8.3% v 15.6%) and the proportion of ambulance offload times longer than 30 minutes (10.5% v 13.3%). All differences were statistically significant (P < 0.05).
Conclusion: Implementation of the FirstNet electronic medical record system was associated with deterioration in ED KPIs.
It is interesting that the discussion in the article covered methodological issues, but not causal. In other words - things got worse, but they don't know why.
Without knowing why things got worse, it suggests that the PCEHR could have the same negative effects. Or not. Nobody knows. The PCEHR is an evidence free initiative.
However, we do have evidence that an EMR slows things down in an ED.
It might be prudent to conduct some research into the potential impact that the PCEHR is going to have on work practices in a much wider context.
I suggest that it would have been a good idea to do have done just that before the technologists, consultants and bureaucrats trumpeted the benefits of eHealth with no hard research evidence.
But then again, prudence isn't at all obvious in government IT systems. It's all belief and vendor hype.
Just my 0.2c worth.
Bernard
ReplyDelete'It might be prudent to conduct some research into the potential impact that the PCEHR is going to have on work practices in a much wider context.'
Usually you make sense. But what exact research would you propose that (a) is posible without actually building the system as an experiment and (b) wasn't already done exactly as you recommend?
DOHA have made may deep mistakes. But I do not think this is one of them.
Bernard,if you talk to ED docs using FirstNet, you will find your reasons - perhaps the study didn't cover that, but the people on the ground really dislike the system - clunky, poor user interface, potentially clinically unsound. Your comments re PCEHR will no doubt be ignored like every other bit of advice to date, but salient points anyway.
ReplyDeleteMy comments were not intended to be a criticism of either the software or of the research paper. They were an observation on evidence, or rather the lack of it, regarding the benefits of EMRs and EHRs
ReplyDeleteI agree wholeheartedly with Enrico. These same views have, however, been repeatedly ignored for a decade. I hope this article has the desired effect.
ReplyDelete"Where does it end?"
ReplyDeleteThat is an interesting question. It ends when someone decides to good look at what has been delivered for the $billions spent.
Speculative software development like the PCEHR is something that belongs in the private sector and not from the unaccountable public purse.
It ends when the money runs out and in reality its already run out as we are in deficit. The odds of the PCEHR saving money in the health system are very low and its time the government delivered governance rather than trying to be an innovator as they have proven they can't. This is a well known feature of government IT programs and why we would throw good (borrowed) money after bad escapes me.
I think it will end if the new government actually needs to make savings and doesn't want to be saddled with a lemon, and this is likely to be the case. Short of a miracle in Ms Gillards popularity the PCEHRs days are numbered.
If the PCEHR is continued in any form it will cost the country billions over the next few years, with a low chance of return and we just don't have the $$ to allow that luxury.
It will also kill off the innovation that continues to gasp for breath under the weight of the clueless public service fat cats that are desperate to deny that anyone else has a clue. Its hard to spot an eagle when the sky is filled with turkeys.
It would be very interesting to me to have an adult conversation with "government" about the intersection between government procurement/tendering and innovation.
ReplyDeleteAs I see it:
When the thing that is being purchased by the public purse is well understood or commodity i.e. 5000 army uniforms, or cleaning services for these 5 buildings - government tendering seems to work ok.
And when there are proper markets with low barriers to entry, private consumers or businesses spending their own money we seem to get innovation i.e. tablets, google
But where we have the combination of government tendering and 'innovative' products we seem to have a disaster zone - some of which reasons have already been discussed.
Personally I find that government tenders combine a unique blend of over specification ("you must do it with this technology here") and under specification ("here is our general problem which we would like you to solve but please don't interact with any of our staff to truly understand the problems because that would subvert the tender process"). Throw in a big system mentality ("if we aren't spending $100m how can we solve anything") and the follow up ("if we're spending $100m we had better go with a well known consulting firm") - and it just doesn't seem to allow much room for innovators.
So the question is - if the existing ways of procurement aren't working - what are the alternatives? US government seems to have been running lots of 'prize' competitions i.e. $1m to the first to do task X. But I'm not sure how long losing competitors can sustain that. Maybe more 'micro' projects? Seed 10 companies with $250,000 and after 6 months further fund the most promising? That sounds awfully like government 'picking winners' which they are generally bad at. I'd be fascinated by other thoughts?
Is there anywhere in "government" where people sit around thinking about stuff like this?
I should add that one obvious mechanism is to not let governments do this purchasing.. i.e. align the payers/benefits etc so that the markets for this technology are much more like regular consumer markets such as ipads. So I'd be fascinated if anyone has thoughts on how that can happen in our current hospital/health system.
ReplyDeleteYou will find they bureaucrats consistently use the line - picking winners is not permitted we must be seen to be working to a level playing field - so they go out to tender and set up the parameters so they can then pick their favored winner.
ReplyDelete