The work reported yesterday has triggered a major reaction from all over.
From the Sydney Morning Herald we have:
Patients put at risk by software
Julie Robotham
March 7, 2011
THE computer system that runs emergency departments in NSW hospitals is compromising patients' care, according to the first systematic review of the troubled project that found it was crippled by design flaws.
The FirstNet system allows treatment details and test results to be assigned inadvertently to the wrong patient, according to the review. It is based on a technical study of the software and interviews with directors of seven Sydney emergency departments.
The system is so compromised it should be scrapped, a specialist doctors' group said yesterday.
Difficulties retrieving patient records could delay treatment, and the system - on which $115 million has been spent - automatically cancelled pathology and radiology requests if the person was transferred from the emergency department without checking whether these were still needed, according to the study by Jon Patrick, the director of the University of Sydney's health information technology research laboratory.
Sally McCarthy, the president of the Australasian College for Emergency Medicine, said Professor Patrick's findings confirmed that the system, loathed by doctors and nurses, was unsuitable for its purpose.
''When do we stop throwing good money after bad?" said Dr McCarthy, who heads the emergency department at Prince of Wales Hospital. "Anything that takes staff off the floor to spend their working time on an inefficient IT system is a detriment to patients."
The project, part of a 10-year electronic medical records plan intended to make patient histories, X-rays and test results accessible from any hospital in the state, had proceeded too fast - apparently because of contractual obligations - for clinicians' feedback to influence it, Dr McCarthy said.
Lots more here:
We also have some very interesting US reaction:
From here:
For March 7 we have:
From Aussie: “Re: Jon Patrick’s article. Mr. HIStalk, I have never seen a dissection (without anesthesia) of Cerner going to this depth. Unbelievable, although in the USA, one would be professionally dead in the HIT industry if even contemplating talking about these long known issues. Hope you will have the courage to publish something about it.” Professor Jon Patrick of the Health Information Technologies Research Laboratory of University of Sydney expands his writeup (currently in draft) about problems with the implementation of Cerner FirstNet in emergency departments in New South Wales.
You’ll love it if you sell against Cerner because everybody from doctors to software validation experts tears into FirstNet (and, by implication, Millennium in general) from every angle — usability, software and database design, and implementation methods. FirstNet competitors could create a fat anti-Cerner prospect piece just by excerpting from it.
On the other hand, I wouldn’t say it’s necessarily unbiased, it focuses on implementation of a single department application that didn’t go well for a variety of reasons (despite many successful FirstNet implementations elsewhere), it uses the unchallenged anecdotal comments of unhappy users who make it clear they liked their previous EDIS better, and it nitpicks (I wasn’t moved to find a pitchfork when I learned that the primary keys in the Millennium database aren’t named consistently).
But it is interesting when it tries to associate user-reported problems with observed technical deficiencies, such as why information known to have been entered sometimes disappears (problems with non-unique primary keys and referential integrity are mentioned – certainly the latter is a problem with many systems).
In other words, it’s not just about Cerner or some ED project in Australia. The real message is that design and support patient care software is the Wild West at this point since we’re arguably still in the first generation of systems claiming to be clinical (even though they often are really business systems masquerading as such).
Lots more of this article is on the site.
Also Scot Silverstein blog has been busy:
See
http://hcrenewal.blogspot.com/2011/03/what-to-do-about-state-of-ed-ehr-in-nsw.html
http://hcrenewal.blogspot.com/2011/03/on-emr-forensic-evaluation-from-down.html
I am sure more will follow. A big reading day.
The links to the original report is found here:
http://aushealthit.blogspot.com/2011/03/professor-jon-patrick-pops-his-head.html
Enjoy all the reading.
David
I look forward to reading the report in full as time allows.
ReplyDeleteI do wonder though, did Jon specify in his research what NSW health should have done instead of buying Cerner? ie what product should have been purchased?
If we're comparing a purportedly bad product with vaporware, then I'm not sure what the fuss is about.
While this is specifically about the Cerner product, I'm sure you could produce a similar issue list with any health software sold in Australia.
ReplyDeleteIt's no wonder that the FDA in the US is interested in becoming a quality gatekeeper for this stuff.
BTW, I deal with an EDIS implementation. I would hardly describe EDIS as a quality piece of health software.
I started reading at the end – i.e. with Jon’s ideas as to what a “better” system should look like. I got as far as the following:
ReplyDeleteAn evaluation of the complex systems model of a hospital
community provides some insight into the design frames for a
Clinical Information System such as:
1. No central control: communities/people have to be able to
create their own CIS;
2. Self-organising communities: have to be able to change
their CIS at will as their needs evolve;
3. Open System: old data and models need to be retrievable
(archived but still accessible) as much as new models
introducible;
4. Limited view: the complete CIS for the whole system has
to be able to be assembled from all the CISs of the
individual communities and yet its usability by the whole
community has to appear as an emergent property;
After that I’m afraid I stopped reading.
It seems that the heuristic axiom of Ockham’s Razor (as presented by Jon) is the design criteria by which Jon thinks NSW Health should have selected their CIS ... unbelievable!
ReplyDeleteUnfortunately Jon has not proposed any practical alternatives in his papers. His papers are full of opinions, unfounded claims and incorrect facts (not to mention the countless flaws in his research methodologies).
If Jon gets his way, as proposed in his papers, the outcome will be a ceasing of the Cerner EMR project in NSW (or a government review which wastes a lot of time and tax payer money).
Is this what Jon really wants?
Is he really recommending for hospitals revert back to their old departmental systems?
Is this what he really thinks is the best scenario for patients in NSW?
Let’s hope practical and rational thinking prevails. We need to make healthcare better. EMRs are THE ONLY known tool (there is no other wonder drug, magic government policy or genomic therapy) that can have the systemic impact on healthcare that we need to address its increasing demand and costs.
It seems that the heuristic axiom of Ockham’s Razor (as presented by Jon) is the design criteria by which Jon thinks NSW Health should have selected their CIS ... unbelievable!
ReplyDeleteUnfortunately Jon has not proposed any practical alternatives in his papers. His papers are full of opinions, unfounded claims and incorrect facts (not to mention the countless flaws in his research methodologies).
If Jon gets his way, as proposed in his papers, the outcome will be a ceasing of the Cerner EMR project in NSW (or a government review which wastes a lot of time and tax payer money).
Is this what Jon really wants?
Is he really recommending for hospitals revert back to their old departmental systems?
Is this what he really thinks is the best scenario for patients in NSW?
Let’s hope practical and rational thinking prevails. We need to make healthcare better. EMRs are THE ONLY known tool (there is no other wonder drug, magic government policy or genomic therapy) that can have the systemic impact on healthcare that we need to address its increasing demand and costs.
To add to what's already been said, some of us are old enough to remember when EDIS was introduced - not exactly to universal acclaim!Now the ED Directors are defending it... really? Jon couldn't find ANYONE who preferred Cerner? Yeah, right... and they all really want open source too don't they mate.
ReplyDeleteThe other thing that ticks me off is the ongoing and clearly deliberate misrepresentation of this as as ED project that has cost some enormous sum when we all know perfectly well ED is just one part of it. Although obviously the part with the squeaky wheels!
Bah humbug!
The object of Jon Patrick's paper is an analysis of the Cerner FirstNet implementation by NSW Health, and a sorry tale it is, not the canvassing of alternatives.
ReplyDeleteHis suggestion, however, for the "better" systems does point us in the direction of a change in the philosophical stance we should be adopting in implementing e-health in fast moving acute health settings.
The Cerner Systems, based upon the engineering design philosophy, is never going to cut the mustard in areas where the variability of human relationships, communication skills, the complexity of clinical issues can never be forced into the simple engineering model.
It is going to require a whole new paradigm in clinical information systems that will be almost the antithesis of the engineering model. The systems will have to be capable of continuous adaption to the social context it which they are situated, and be able to cope with messy coherence and ambiguity. Social media is somewhat of an archetype for this.
My latest thoughts are at http://hcrenewal.blogspot.com/2011/03/australian-ed-ehr-study-putting-lie-to.html
ReplyDelete"As usual, the consumers (doctors) had virtually no input." - from the above mentioned blog
ReplyDeleteYes, I've seen that argument to. However, this is a comment that seems to be made after the fact.
I've seen the other end of this: continual attempts to engage doctors in discussions before and during implementation fail because "I'm too busy ... I'm not a computer person ... I'm a clinician, not an IT person ...".
Excuses piled on excuses for not engaging with the project, and then complaining that they wheren't consulted.
Physicians lose their right to complain about consultation the moment they refuse to engage early in the process.
I've seen the other end of this: continual attempts to engage doctors in discussions before and during implementation fail because "I'm too busy ... I'm not a computer person ... I'm a clinician, not an IT person ...".
ReplyDeleteExcuses piled on excuses for not engaging with the project, and then complaining that they wheren't consulted.
Physicians lose their right to complain about consultation the moment they refuse to engage early in the process."
There is more than a grain of truth in this comment - speaking from personal experience!
David.
There were more than 50 design sessions held in locations across the state involving literally hundreds of clinicians. Anyone having completed even minimal research would be fully aware of this.
ReplyDeleteHowever, as I understand it, that all led to a state wide common build, which lacked the flexibility to suit individual hospitals. Hence the unhappiness as they were forced to use something that was one size fits all.
ReplyDeleteDavid.
Anonymous said:
ReplyDeleteUnfortunately Jon has not proposed any practical alternatives in his papers. His papers are full of opinions, unfounded claims and incorrect facts (not to mention the countless flaws in his research methodologies).
I have been accused of the same in my writings, yet those readers seem never to find nuggets I plant such as:
Health Informatics — Application of clinical risk management to the manufacture of health software. UK National Health Service, DSCN14 (2009), formerly ISO/TS 29321:2008(E), http://www.connectingforhealth.nhs.uk/engagement/clinical/occo/safety/dscn14.pdf
and
Health informatics — Guidance on the management of clinical risk relating to the deployment and use of health software. UK National Health Service, DSCN18 (2009), formerly ISO/TR 29322:2008(E), http://www.connectingforhealth.nhs.uk/engagement/clinical/occo/safety/dscn18.pdf
and
Medical Informatics 20/20: Quality And Electronic Health Records Through Collaboration, Open Solutions, And Innovation (GOldstein et al., the VistA pioneers), http://www.amazon.com/Medical-Informatics-Electronic-Collaboration-Innovation/dp/0763739251
and
H.I.T. or Miss: Lessons Learned from Health Information Technology Implementations, AHIMA press, https://www.ahimastore.org/ProductDetailBooks.aspx?ProductID=14181
(Disclosure, I'm quoted in 20/20 and an assoc. editor of HIT or Miss, but get no royalties of any kind for sales.)
-- SS
The problem with "one size fits all" is that -
ReplyDeleteone size does not fit all.
This is a first principle, and as such it is not open to debate.
Anonymous writes:
ReplyDelete"His papers are full of opinions, unfounded claims and incorrect facts (not to mention the countless flaws in his research methodologies)."
Really? How 'bout expounding on your above thesis, professor?
I can easily claim "you beat your wife", but without giving evidence, it doesn't mean much.
Dr Patrick - you are so misinformed. There are so many untruths in what you report. Why is this? Because you interviewed 7 people. Being Directors, what is the likelihood they are hands on with the system? Minimal I would say from experience in the EDs. Call yourself a researcher - huh! It is a totally biased report which reflects the opinions of those with an axe to grind. Why don't you talk to some nurses,clerical staff and junior doctors? There are hundreds of them working in EDs across the state. Their opinion should matter as they are the ones caring for our patients. You should be ashamed of yourself! I certainly would not hire you to do a research project if this is the biased and blatantly incorrect rubbish that you come up with. Shame on you!!
ReplyDeleteAnonymous Mar 28 1:36 PM, it seems your own axe is so large you could knock down the American Redwood National Forest with it.
ReplyDeleteI would bet you're a corporate shill or sockpuppet. I've had to deal with sockpuppets from other health IT companies, such as here: http://hcrenewal.blogspot.com/2010/01/more-on-perversity-in-hit-world.html
Directors well know what's going on with their charges, and their complaints. I know I did when I ran a clinical department, or a clinical R&D group.
Everyone Down Under should read my account of the health IT sockpuppet. They sounded - well, like you.
Has Dr Patrick ever completed (or even been involved in) an implementation of an EMR (at all or of any scale)? Can someone tell us what makes him an authority the subject?
ReplyDeleteThere are so many relevant things he hasn't considered (the benefits of integration of ED systems with the hospital EMR and the benefits to patients of having an integrated system across the entire state to name two).
Also not considered in his paper is that what NSW Health has implemented to date is just the foundation for the NSW EMR. This foundation will allow many additional applications to be deployed in the future, which will drive further value into the health system.
His view is unbalanced and his paper is clearly bias. How Sydney University tolerates papers of such poor quality and biasness (being published by their professors) is beyond me.
Anonymous said:
ReplyDelete"Let’s hope practical and rational thinking prevails. We need to make healthcare better. EMRs are THE ONLY known tool (there is no other wonder drug, magic government policy or genomic therapy) that can have the systemic impact on healthcare that we need to address its increasing demand and costs. "
I really do take exception to this statement! I have seen no evidence that EHR will have a positive systemic impact, and in fact, looking at the direction that a lot of EHR implementation is heading (including the PCEHR debacle) the impact is likely to be far from positive in outcome or cost saving for some time!
It is a far cry reading such subjective drivel compared to items such as the 8th Report of the Confidential Enquiries into Maternal Deaths in the UK (Saving Mothers’ Lives 2006 – 2008) published this week which I just read. This is real change that has been implemented positively over a long period - and it is echoed in quite a number of clinically relevant reporting initiatives.
http://www.cmace.org.uk/Publications-Press-Releases/Report-Publications/Maternal-Mortality.aspx
The Executive summary is good reading.
At the end of the day an EHR is just another tool that can be implemented well or poorly to help clinicians & patients in the management of patients' individual health problems - that is where the difference will occur.
At the end of the day, Cerner has failed to deliver. Why postulate when it's obvious?
ReplyDeleteOk, Ok.....they may have delivered according to a specific (albeit undercooked)criteria.
The issue is that they got a foot in the door in the first place at frightening cost when a better local alternative was available at a fraction of the cost.
When the bureaucrats are made to step back from "managing" health patient care and outcomes might just improve.
It is always challenging when implementing a new clinical system. Almost 20 years ago, EDIS was introduced to most EDs in NSW, and many clinicians were disappointed then and complained - EDIS had the ability to enter orders (for tests and medications), but heck! the electronic orders went nowhere, and clinicians still had to handwrite their orders on paper slips and get the orderlies to deliver these to pathology and pharmacy (same as before). The vital integration with clinical systems was not there, and EDIS became more of an administrative tool for nurses, clerks and waiting times. The difference now with systems like Cerner's Firstnet is that clinicians are now using the software, with integration - work practices are changing for clinicians - it will be a hard road (no matter what vehicle), but some day in the future when Cerner is being replaced by something else, clinicians may look back and remember it fondly...
ReplyDeletebut some day in the future when Cerner is being replaced by something else, clinicians may look back and remember it fondly...
ReplyDeleteAnd Ostriches might fly.
Patients are experiencing death and injury from these devices in the US and in the UK...and all over. The HIMSS trade group is powerfully wealthy, I am told by insiders, from trading on insider information from its member companies and start-ups. HIMSS has deceived the government leaders of innumerable countries.
ReplyDeleteIn the US, Cerner's systems cauused baby deaths at Pittsburgh's Children's Hospital, and they blamed the users and poor deployment. They also blamed the researchers stating that the data was flawed.
Then, M. DelBecarro at Seattle had allowed his name to go on what appears to be a Cerner ghost written (or sock puppet written) paper published about 6 months later, "exonerating" Cerner's system of CPOE.
What difference did it make? The babies were dead and are still dying.
Jenny S, Peds, RN
This is the second part of my response.
ReplyDeleteC7."There were more than 50 design sessions held in locations across the state involving literally hundreds of clinicians. Anyone having completed even minimal research would be fully aware of this." I have addressed the issue of manipulation of the consultation process so as to disenfranchise clinical staff in Part 2.
C8. "Being Directors, what is the likelihood they are hands on with the system? Minimal I would say from experience in the EDs" You are quite mistaken. ED Directors work on the floor attending to patients here in NSW, plus they have to manage their staff. All but one of the Directors I spoke to showed me the software working on their desk and took me through their points of discussion on the screen. The only exception was the person I spoke to by telephone, but they clearly had a detailed experience of the software.
C8"Why don't you talk to some nurses,clerical staff and junior doctors?" I work with nurses, doctors and clerical staff whenever I go to a hospital. I find no significant variation in their opinions.
C9. "I certainly would not hire you to do a research project if this is the biased and blatantly incorrect rubbish that you come up with. Shame on you!!" I would be happy not to be asked by you sir, although I would hope to give a more polite reply than yours truly.
C10. "Has Dr Patrick ever completed (or even been involved in) an implementation of an EMR (at all or of any scale)?" Yes I have completed the development of EMR systems and continue to do research as to how they might best be designed and built. See PArt 9 for proposals and discussion of a prototype.
C11. "Can someone tell us what makes him an authority the subject?" I really don't know except it is not myself, but it may well be you by complaining so vacuously about my work without actually having read it.
C12 "There are so many relevant things he hasn't considered (the benefits of integration of ED systems with the hospital EMR and the benefits to patients of having an integrated system across the entire state to name two). " yes there would be many benefits from such a system if it worked, but it doesn't work and my study exposes some of the explanations.
C13 "Also not considered in his paper is that what NSW Health has implemented to date is just the foundation for the NSW EMR. This foundation will allow many additional applications to be deployed in the future, which will drive further value into the health system." Once again if that were only true I would be happy, but a weak foundation will only collapse under the slightest vibration.
C14. "His view is unbalanced and his paper is clearly bias." I'd be grateful to know the instances in the text where these things have occurred.
I shall endeavour to answer the questions brought up in all these posts in the one answer. I will not provide answers where it is clear commentators have not read the report but refer them to an appropriate section
ReplyDeleteQ1." ie what product should have been purchased?" This is discussed in Part 1
C2. "After that I stopped reading" that's a pity as the subsequent sections say how the these USER'S criteria have influenced our designs.
C3 "It seems that the heuristic axiom of Ockham’s Razor (as presented by Jon) is the design criteria by which Jon thinks NSW Health should have selected their CIS ... unbelievable! " This is an unreasonable misrepresentation of my paper content's Ockham's razor is justified to explain a heuristic for designing software. Read the paper with care and insight and you'll understand it.
C4. "If Jon gets his way, as proposed in his papers, the outcome will be a ceasing of the Cerner EMR project in NSW" This is a false assertion I make no such statements.
C5. "EMRs are THE ONLY known tool (there is no other wonder drug, magic government policy or genomic therapy) that can have the systemic impact on healthcare that we need to address its increasing demand and costs" This is a claim of limited merit. NSW, Victoria and the UK have all had major investments in enterprise based EMRs system with little to show for it. These EMR work unsatisfactorily, or alternatively the vendors over promise. Either way in these installations they are too deficient to be used with confidence by clinicians.
C6. "Now the ED Directors are defending it... really? Jon couldn't find ANYONE who preferred Cerner? Yeah, right... and they all really want open source too don't they mate."
These are all spurious assertions about my work. I reported that 1 Director in 7 was happy with his system. But most importantly he didn't use the State BAsed Build but had control of his own build. Secondly he controlled how much of the Cerner Millennium suite he used and so didn't hit the pitfalls of the other hospitals, e.g. his departmetn still did their documentation on paper.
Phage here. Circulation on the web has the US HIT Director David Blumenthal as having been a defendant in a med mal case involving lab reports that were sent to an EMR but not seen by him or his staff.
ReplyDelete