A key objective of the PCEHR is to make clinical information that is captured electronically shareable with the patient and other relevant clinicians.
Underlying this information sharing is the unstated assumption that the information held in source systems will be ‘fit for sharing’.
All one can say is that if this recently published study is any guide we are a zillion miles from that situation.
Here is a report on the study.
Electronic records 'not fit for purpose'
- By Michael Woodhead on 19 September 2011
The assumption that electronic medical records will improve care is unrealistic because data is often missing or inaccurate, an audit of emergency department records suggests.
A review of records for almost 2600 patients seen in NSW emergency departments with conditions such as diabetes, COPD and cardiovascular disease has found that discharge summaries were missing for 12-15% of patients.
In addition, where summaries were available, only three-quarters of the diagnoses were confirmed by the discharge summary audit, the study in Emergency Medicine Australasia (online Sept 19) found.
The study examined the accuracy of the diagnoses of chronic diseases in a ‘typical’ community hospital based on the ED information system (EDIS), and the NSW Health electronic medical record (EMR).
The accuracy of electronic records was best for diabetes and worst for asthma and COPD.
The full article is here:
Here is the abstract as published yesterday.
Health reform: Is routinely collected electronic information fit for purpose?
- Siaw-Teng Liaw,
- Huei-Yang Chen,
- Della Maneze,
- Jane Taggart,
- Sarah Dennis,
- Sanjyot Vagholkar,
- Jeremy Bunker
Article first published online: 19 SEP 2011
DOI: 10.1111/j.1742-6723.2011.01486.x
© 2011 The Authors. EMA © 2011 Australasian College for Emergency Medicine and Australasian Society for Emergency Medicine
Abstract
Objective: Little has been reported about the completeness and accuracy of data in existing Australian clinical information systems. We examined the accuracy of the diagnoses of some chronic diseases in an ED information system (EDIS), a module of the NSW Health electronic medical record (EMR), and the consistency of the reports generated by the EMR.
Methods: A list of ED attendees and those admitted was generated from the EDIS, using specific (e.g. angina) and possible clinical terms (e.g. chest pain) for the selected chronic diseases. This EDIS list was validated with an audit of discharge summaries, and compared with a list generated, using similar specific and possible Systematized Nomenclature of Medicine – Clinical Terms (SNOMED-CT), from the underlying EMR database.
Results: Of the 33 115 ED attendees, 2559 had diabetes mellitus (DM), cardiovascular disease or asthma/chronic obstructive pulmonary disease; of these 2559, 876 were admitted. Discharge summaries were missing for 12–15% of patients. Only three-quarters or fewer of the diagnoses were confirmed by the discharge summary audit, best for DM and worst for cardiovascular disease. Proportion of agreement between the lists generated from the EDIS and EMR was best for DM and worst for asthma/chronic obstructive pulmonary disease. Possible reasons for this discrepancy are technical, such as use of different extraction terms or system inconsistency; or clinical, such as data entry, decision-making, professional behaviour and organizational performance.
Conclusions: Variations in information quality and consistency of the EDIS/EMR raise concerns about the ‘fitness for purpose’ of the information for care and planning, information sharing, research and quality assurance.
The two paragraph conclusion in the full article says it all:
Conclusions
The present study highlights what we already know – we are only as good as the information we have! The various purposes designated for the EMR as part of health reform are unrealistic if we cannot rely on data quality and consistency of our information systems. The current information quality and consistency of the EDIS/EMR system in a typical community hospital with an unaccredited ED needs improvement to easily support research. If it is not good enough for research, it is probably not good enough to promote and support safe and effective clinical practice.
Further systematic examination of the large repositories of routinely collected data is required to determine the fitness for purpose, which might be to share good-quality information among clinicians or to support electronic decision support systems. A logical next step is to repeat the present study with a representative sample of EDs and include a qualitative study of clinicians and information managers to understand the underlying reasons for any variations in data quality.
-----
NEHTA has recognised that ‘data quality’ is a critical issue (see page 46 of the PCEHR Concept of Operations) however it really has no idea how to actually fix the problem if the ‘management speak’ of the following 2 pages is to be read carefully. Their plan is (in summary):
4.2.1 Data quality
Ensuring a high standard of data quality is an essential requirement for the PCEHR System. High levels of data quality are required to assist healthcare providers and individuals in making safe healthcare decisions. Data quality within the PCEHR System will be ensured through a combination of validation of data loaded into the system, working with operators or source systems to improve the quality of data they are able to provide and by ongoing monitoring of data quality.
----- End Quote
The UK worked on this problem in the context of their shared EHR summary record and found a great deal of time, money and effort was required to get even close to clinical levels of reliability.
Doing a shared health record in haste and on the cheap as this plan suggests is going on, with but a brief outline plan for action, is a recipe for disaster in my view. I certainly don’t know any clinicians who would be interested in acting on the basis of the quality of information we see implied her as being planned for the PCEHR (Discharge Summaries etc.).
Much more careful planning, careful piloting and steps to ensure quality maintenance are vital - and none of this is quick or cheap!
David.
4 comments:
David, it is one thing to study a records system and conclude that information is missing, it is an entirely different matter to conclude either that one must (or even can) "fix the information first".
The problem here is that until you share the data, the "purpose" in "fit for purpose" is not "sharing", and the reason it is futile to attempt to "fix the information first" (i.e., before sharing) is because you'll be fighting against the natural dynamics of the existing system of isolated information. However, when you start sharing, you change the context and the dynamics and, if done right, create an environment where feedback loops drive an improvement on quality.
I simply disagree.
If you share material that is not accurate you put the information subjects and clinicians who use the information at risk - of all sorts of outcomes.
The way you do this properly is to only share information that there is clear accountability and responsibility for and which is used by the information provider to undertake THEIR decision making. i.e. the provider of the information has real 'skin in the game' as it were.
Nowhere in the PCEHR design can I see the feedback loops you mention.
If you don't think sharing 'junk' is dangerous you are clearly detached from clinical realities.
As far as the paper is concerned I suspect part of the cause of the major issues was due to the fact the recording clinicians were not those who used the information later in the patient stay.
Finally - as for suggesting it is futile to not aim for optimal information quality within EHR records before sharing I find the idea odd. Most experts I know suggest information quality flows from having the user need and rely on the contents and be accountable for its maintenance.
David.
"... information quality flows from having the user need and rely on the contents..." This, I believe, was my point. But you converge on optimal quality (where optimal is defined by the context, not as 100% true).
If you trust material that you know or suspect to be incomplete and/or inaccurate, then that is clearly a problem and, just a clearly I hope, I was not advocating that.
However, I am confused by your statements above:
1. You Need To Fix The Information First
2. The way you do this properly is to only share information that there is clear accountability and responsibility for...
3. ... sharing 'junk' is dangerous...
2 is a way to deal with 3, but 2 doesn't say anything about 1
Indeed, 2 is precisely about the feedback loop. and may require that some "fixing" is done, but you can't do this in isolation from the processes that constitute the sharing. My fear is that some would enact 2 simply by doing no sharing even though the evidence tells us that not sharing data (well) is harming patients.
Re the PCEHR, I made and make no claims about the proposed model containing (or not) appropriate feedback loops. Rather, I said "if done right" -- the corollary being that if the feedback loops are not there, then you have a (data quality) problem.
Michael,
The link between all three points is ownership of, need to use and accountability for what is entered in systems. My core point is that in the rush to the PCEHR these complex and difficult to address issues are being largely swept under the carpet - and that this may result in the whole thing to end very, very badly.
I suspect you might even agree with that.
David.
Post a Comment