This Is Sure To Get a Lot of Coverage. This is The Sort of Research That Often Confuses and Makes Headlines.

The following paper appeared a little while ago.

http://www.ajmc.com/supplement/managed-care/2010/AJMC_10dec_HIT/AJMC_10decHIT_Jones_SP64to71

Electronic Health Record Adoption and Quality Improvement in US Hospitals

Spencer S. Jones, PhD; John L. Adams, PhD; Eric C. Schneider, MD; Jeanne S. Ringel, PhD; and Elizabeth A. McGlynn, PhD

Published Online: December 22, 2010 - 12:00:43 AM (EST)

Objective: To estimate the relationship between quality improvement and electronic health record (EHR) adoption in US hospitals.

Study Design: National cohort study based on primary survey data about hospital EHR capability collected in 2003 and 2006 and on publicly reported hospital quality data for 2004 and 2007.

Methods: Difference-in-differences regression analysis to assess the relationship between EHR adoption and quality improvement for acute myocardial infarction, heart failure, and pneumonia care.

Results: Availability of a basic EHR was associated with a significant increase in quality improvement for heart failure (additional improvement, 2.6%; 95% confidence interval [CI], 1.0%-4.1%). However, adoption of advanced EHR capabilities was associated with significant decreases in quality improvement for acute myocardial infarction and heart failure. We observed 0.9% (95% CI, -1.7% to -0.1%) less improvement for acute myocardial infarction quality scores and 3.0% (95% CI, -5.2% to -0.8%) less improvement for heart failure quality scores among hospitals that newly adopted an advanced EHR, and 1.2% (95% CI, -2.0% to -0.3%) less improvement for acute myocardial infarction quality scores and 2.8% (95% CI, -5.4% to -0.3%) less improvement for heart failure quality scores among hospitals that upgraded their basic EHR.

Conclusions: Mixed results suggest that current practices for implementation and use of EHRs have had a limited effect on quality improvement in US hospitals. However, potential "ceiling effects" limit the ability of existing measures to assess the effect that EHRs have had on hospital quality. In addition to the development of standard criteria for EHR functionality and use, standard measures of the effect of EHRs on quality are needed.

(Am J Manag Care. 2010;16(12 Spec No.):SP64-SP71)

The paper is here:

http://www.ajmc.com/supplement/managed-care/2010/AJMC_10dec_HIT/AJMC_10decHIT_Jones_SP64to71

In passing it is worth noting this paper was one of a number on Health IT. A full list is here:

http://www.ajmc.com/supplement/managed-care/2010/AJMC_10dec_HIT

The paper caused a number of responses.

An early one is here:

Sunday, December 26, 2010

Study highlights 'lurking question' of measuring EHR effectiveness: The science in Medical Informatics is dead

The science in Medical Informatics is dead.

I'm not going to even use academic fabric softener in my assertion, e.g., "may be", "appears to be", or "is it?" (as a question) dead.

It's dead.

When HIT experts recommend changing the study goalposts when existing studies don't give results they'd like to see, rather than first and foremost critically and rigorously examining why we're seeing unexpected results, science is dead.

http://www.healthcareitnews.com/news/study-highlights-lurking-question-measuring-ehr-effectiveness

Study highlights 'lurking question' of measuring EHR effectiveness

December 22, 2010 | Molly Merrill, Associate Editor

WASHINGTON – Hospitals' use of electronic health records has had just a limited effect on improving the quality of medical care nationwide, according to a study by the nonprofit RAND Corporation.

The study, published online by the American Journal of Managed Care, is part of a growing body of evidence suggesting that new methods should be developed to measure the impact of health information technology on the quality of hospital care.

[In other words, we're not getting the results we thought and hoped we'd get with "Clinical IT 1.0", so let's alter the study methodologies and endpoints --- rather than using the results we have to identify the causes and improve the technology to see if we can do better with "Clinical IT 2.0."

Further, it's not as if there's no other data on why health IT might not work as hoped - ed.]

Most of the current knowledge about the relationship between health IT and quality comes from a few hospitals that may not be representative, such as large teaching hospitals or medical centers that were among the first to adopt electronic health records.
[This implies "other" "representative" hospitals are either not doing it right, or the technology is ill suited for them and may never work. Which is it? We really need to know before we proceed with hundreds of billions more in this "Grand Experiment" - ed.]

The RAND study is one of the first to look at a broad set of hospitals to examine the impact that adopting electronic health records has had on the quality of care.

The research included 2,021 hospitals – about half the non-federal acute care hospitals nationally. Researchers determined whether each hospital had EHRs and then examined performance across 17 measures of quality for three common illnesses – heart failure, heart attack and pneumonia. The period studied spanned from 2003 to 2007.

The number of hospitals using either a basic or advanced electronic health records rose sharply during the period, from 24 percent in 2003 to nearly 38 percent in 2006.

[How many billions of dollars diverted from patient care needs does that represent? - ed.]

Researchers found that the quality of care provided for the three illnesses generally improved among all types of hospitals studied from 2004 to 2007. The largest increase in quality was seen among patients treated for heart failure at hospitals that maintained basic electronic health records throughout the study period.

However, quality scores improved no faster at hospitals that had newly adopted a basic electronic health record than in hospitals that did not adopt the technology.

[In other words, the improvements or lack thereof had little to do with electronic vs. paper record keeping - ed.]

…..

Carl Sagan wrote that science is a candle in the dark in a demon haunted world.

It seems the demons are winning.

-- SS

Lots more here:

http://hcrenewal.blogspot.com/2010/12/science-in-medical-informatics-is-dead.html

Coverage also appeared here:

EHR upgrades no sure quality boost: RAND

By Joseph Conn

Posted: December 23, 2010 - 12:00 pm ET

Hospitals upgrading their electronic health-record systems might not see quality improvements as dramatic as expected, according to a newly released report on research funded through the RAND Corp.

The RAND researchers compared the levels of adoption of hospital EHR systems between 2003 and 2006 with their scores on selected clinical quality process measures for acute myocardial infarction, heart failure and pneumonia for those information technology-enabled hospitals and those without IT systems. The study period covered 2003 to 2007.

They concluded that availability of what the researchers defined as a basic EHR was associated with “a significant increase in quality improvement for heart failure” but conversely, adoption of an “advanced” EHR was associated with “significant decreases in quality improvement for acute myocardial infarction and heart failure,” according to a 16-page report, Electronic Health Record Adoption and Quality Improvement in U.S. Hospitals, based on research by the RAND Corp., published in the December issue of the American Journal of Managed Care.

“During the study period, the quality of care for AMI, heart failure and pneumonia was broadly improving,” the researchers said. “Heart failure quality scores improved significantly more among hospitals that maintained a basic EHR than among hospitals with no EHR.”

But, according to their research, “We did not observe a similar effect on AMI or pneumonia quality scores, nor did we find that adopting or upgrading an EHR accelerated quality improvement. Instead, our results indicate that new adoption or upgrade to an advanced EHR was associated with smaller gains in AMI and heart failure quality scores.”

Full paper here:

http://www.modernhealthcare.com/article/20101223/NEWS/312239996/

and here:

Study: The better care no guarantee with e-health

According to a recent U.S. study, hospitals that upgrade to electronic medical records may not necessarily have higher-quality medical care. The study was in the latest online issue of the American Journal of Managed Care that focused on a range of U.S. hospitals.

The 2,021 hospitals in the study represented half of the non-federal, acute-care hospitals in the U.S., researchers examining performance across 17 measures for three common illnesses such as heart failure, heart attack and pneumonia.

Spencer Jones, an information scientist at RAND, a non-profit research organization, and his co-authors reported that they had found significantly better care for patients treated for heart failure than the other two conditions.

Quality scores for heart attack improved no faster at hospitals, which used basic electronic health records compared to those that didn’t after reviewing data from 2003 to 2007.

The scientists reported numerous explanations for the mixed results, such as how adopting the technology may divert staff from other ways of improving quality, or how existing hospital quality measures may rise to a ceiling level and no higher.

Full article here:

http://shortcutgeek.com/study-the-better-care-no-guarantee-with-e-health/223825/

And last here:

EHR Effectiveness for Hospital Care Questioned

Cheryl Clark, for HealthLeaders Media , December 29, 2010

A large RAND study of nearly half the acute care hospitals in the U.S. calls into question the value of electronic medical records, saying that except for basic systems used to treat congestive heart failure patients, EHRs are not improving process of care measures for many large hospitals that have them.

"The lurking question has been whether we are examining the right measures to truly test the effectiveness of health information technology," says Spencer S. Jones, a RAND scientist and lead author for the report. "Our existing tools are probably not the ones we need going forward to adequately track the nation's investment in health information technology."

Jones and authors write that their "results should temper expectations for the pace and magnitude of the effects of the Health Information Technology for Economic and Clinical Health (HITECH) legislation. The challenges and unintended consequences of EHR adoption are well documented."

Federal aid amounting to $30 billion is stimulating a national push to adopt EHR in healthcare settings, but Jones' report says that data demonstrating that the technology improves quality comes from a few large teaching hospitals and may not be representative of hospitals at large.

The study looks at 17 process measures for three common illnesses, heart failure, heart attack, and pneumonia between 2003 and 2007. Also, the number of hospitals using basic or advanced EHR grew from 24% to nearly 38% during that period.

Those 17 process measures included whether clinicians gave aspirin to patients who arrived in the hospital with an acute myocardial infarction, an ACE inhibitor or ARB for left ventricular systolic dysfunction to patients with heart failure and an oxygenation assessment to patients admitted with pneumonia. The measures did not include any outcomes.

Quality of care at all hospitals improved overall regardless of whether they had EHR over this time, but with the exception of patients with CHF, it did not improve faster at hospitals with EHR than at hospitals without it.

"Adoption of advanced EHR capabilities was associated with significant decreases in quality improvement for acute myocardial infarction and heart failure," the authors write.

RAND used Health Information Management Systems Society survey data, which includes 90% of U.S. hospitals and includes clinical IT application implementation status, excluding hospitals that didn't disclose their software vendor. Hospital process measures were taken from Medicare's Hospital Compare.

Lots more here:

http://www.healthleadersmedia.com/print/TEC-260743/EHR-Effectiveness-for-Hospital-Care-Questioned

I suggest that people read the .pdf file which makes the tables much easier to understand and the whole paper easier to grasp.

For me, I worry about studies melding a large number of hospitals with varying systems being lumped together retrospectively to try and form conclusions based on information that is a few years out of date.

I am more comfortable assessing the outcomes when known systems are deployed in Randomised Controlled Trial like situations or similar as more trust can be placed on both the outcomes and the analysis. Not to say such work should not be done but it needs to be assessed very carefully.

Just how hard it can be to draw useful information from broad studies can be very hard. This recent paper highlights the issue.

Shahian DM et al. Variability in the measurement of hospital-wide mortality rates. N Engl J Med 2010 Dec 23; 363:2530.

The results, as reported, are interesting and do need careful analysis and ideally testing.

David.