This appeared last week:
Should you trust what e-records tell you?
22 May 2018
GPs are repeatedly told that electronic medical records will fix the problems of faulty memories, illegible hospital handover notes and patients who aren’t faithful to one GP (but don’t tell you). But do you trust what your computer says?
Researchers from Perth read the electronic patient records of almost 1000 patients and then checked with the real-life patient if they were accurate.
The results focused on whether patients had received their flu vaccination or not. These searches found that, in 84% of cases, the electronic records and the patients’ memories were the same. However, in 16% of cases, the computer said no but the patient said yes — or the other way around.
The first scenario was more common by a fair stretch (the patient said they had been vaccinated, but it wasn’t on their electronic patient record).
As the researchers from the WA Department of Health pointed out, this might be because they had the vaccination at work or a pharmacy, so it was easily missed.
They also looked at the accuracy of electronic records covering the presence of diabetes, asthma, chronic heart disease or pregnancy.
They found that concordance was highest for diabetes (96%), then heart disease (92%), pregnancy (90%) and asthma (89%).
More here:
What this article is telling you is that the source articles for the myHR have very significant omissions and errors – so just how useful is it to try and use such inaccurate records in an aggregate sense. Not very would be my feeling.
More useful would be to design a program to improve the accuracy of the records we have. I suspect that would be more useful than gathering all these erroneous records together and trying to make something of them!
BTW – does anyone have the source paper – some my like to read it I suspect and I can’t find it.
David.
IMHO. this is a very important research result.
ReplyDeleteFor those of you not familiar with things like error, accuracy and reliability analysis if you want an assessment of the data as a whole you multiply the individual rates.
As an example, taking the above accuracy rates (0.84, 0.96, 0.92, 0.90, 0.89), that gives an overall accuracy of 0.594 or 59.4% accuracy.
How do you suppose a medic would react if told that a patient’s electronic record with five data points in was probably only 60% correct, with no indication of which data were incorrect?
As another example, suppose there are 14 elements in a health record each with an accuracy rate of 95%. The total record accuracy rate would be under 50%.
And it only takes four elements to have an accuracy rate of 84% (as in the flu vaccination example above) for the accuracy rate to be less than 50%
Accuracy rates can be subject to a number of factors, including – errors in data entry, errors in taking a patient’s history, errors in tests, ageing of data.
A general rule, the more data in a health record and the older the data, the less the accuracy is likely to be.
Do you suppose ADHA will be alerting GPs and consumers to this research?
Do we know if ADHA has released any reports, evidence or research on the safety of health records?
The ADHA strategy says this:
“The interoperability of clinical data is essential to high-quality, sustainable healthcare – this means that patient data is collected in standard ways and that it can be shared in real time with them and their providers.
By the end of 2018, a public consultation on draft interoperability standards will confirm an agreed vision and roadmap for implementation of interoperability between all public and private health and care services in Australia. Base-level requirements for using digital technology when providing care in Australia will be agreed, with improvements in data quality and interoperability delivered through adoption of clinical terminologies, unique identifiers and data standards.”
Is source data quality a critical success factor in the strategy? No. Is that because of a single minded focus on technology not medical information? Quite probably. Will interoperability and standards improve data quality? No.
ADHA thinks that the magic of technology will drive data quality. I don’t support that belief; IMHO, they need to justify such an assertion.
I don’t believe that ADHA has any real understanding of the nature and complexity of medical and health data in the health care system. Their approach is built on a simplistic assumption that health data is good, more data is better and that patients and health professionals will delight in the oceans of data myhr will deliver. I’m not convinced.
I know someone who had a total knee replacement. After the operation and rehabilitation, they went to see their GP about on-going pain management.
ReplyDeleteThe discharge summary sent to the GP said that the right knee had been replaced. Unfortunately, it was the left knee - the right knee had been replaced a year earlier.
It is quite easy for the hard copy discharge summary to be edited with a black pen. Getting the original changed would not be so easy.
It is quite obvious to the patient that there was an error. Not all medical data is as easy to understand and correct.
A few observations.
A discharge summary is a distillation and analysis of all the data that was collected during the operation, post op care and then rehab care. Automating that is not easy, humans need do it, hence the scope for errors, no matter how good the documentation created during the whole process, which is what medical records tend to be.
Most clinical health record systems and the myhr in particular are mainly documentation management systems with little or no clinical process automation. No amount of data standardisation and interoperabiity is going to change that, but the creation of clinical process automation tools is where the benefit could come from, if only it could be done.
I doubt that the management at ADHA understand the true extent of the problem facing so called "digital health". Tim certainly didn't mention the real problems facing digital health at his talk last week at the NPC.
"The discharge summary sent to the GP said that the right knee had been replaced. Unfortunately, it was the left knee - the right knee had been replaced a year earlier...It is quite obvious to the patient that there was an error."
ReplyDeleteAha, perhaps from the patient's perspective it was the left knee. And from the clinician's perspective it was the right knee?
Reminds me of the old joke: the bad news it we accidentally amputated the wrong leg; the good news is that your other leg is getting better.
But seriously, by 'right' perhaps they meant 'correct'? We fixed the correct one this time!