Here is Another Study That Will Stir Some Debate! Take it From Me it Is Essentially Useless Rubbish - But I Would Say That Wouldn’t I?

The following article appeared in the Health Section of the Saturday Australian on Saturday.

Experts challenge e-health critique: researchers suggests they may not be worth the money

• Leigh Dayton, Science writers
• From: The Australian
• January 22, 2011 12:00AM

DESPITE widespread support for electronic patient records, electronic prescribing and other e-health technologies, a global review by British researchers suggests they may not be worth the money.

Specifically, a team led by physician and epidemiologist Aziz Sheikh with Edinburgh University reported this week in the journal PLoS Medicine that evidence supporting the benefits of e-health technologies is weak and inconsistent, "despite being frequently promoted by policy-makers and techno-enthusiasts".

Worse, they found some evidence that introducing e-health technologies can generate new risks, such as prescribing practitioners becoming over-reliant on the technology, resulting in errors.

This may surprise Australian taxpayers who, in the two years to June 2012 will have spent $467 million on a national e-health record system that does not yet exist.

That excludes another $218m spent during the same period on the National E-health Transition Authority, established in 1995 by the Australian, state and territory governments to develop better ways of electronically collecting and securely exchanging health information.

Sheikh and his colleagues analysed systematic reports, so-called meta-reviews, published from 1997 to 2010, identifying 53 that evaluated the impact of e-health technologies on the quality, cost and safety of healthcare delivery.

They conclude that given the lack of evidence supporting e-health, it's vital that e-health technologies should be rigorously evaluated against a comprehensive set of measures, from design to implementation and adoption.

According to Australian Medical Association vice-president Steve Hambleton, the PLoS paper shows that data being collected "isn't useful" and that evaluation of e-health technologies has "been exceptionally poor to this point".

Enrico Coiera, director of the Centre for Health Informatics at the University of NSW, agrees evidence is patchy and better methods of evaluation are needed. But he disputes the claim by Sheikh and co that there's no good evidence e-health technologies are cost-effective and improve patient outcomes. "They looked at weak literature as opposed to primary sources," says Coiera.

More commentary is found here:

http://www.theaustralian.com.au/news/health-science/experts-challenge-e-health-critique-researchers-suggests-they-may-not-be-worth-the-money/story-e6frg8y6-1225991983661

There has also been coverage here:

Little Evidence To Support Most E Health Technologies, Such As Electronic Patient Records

19 Jan 2011

Despite the wide endorsement of and support for eHealth technologies, such as electronic patient records and e-prescribing, the scientific basis of its benefits-which are repeatedly made and often uncritically accepted-remains to be firmly established.

Furthermore, even for the eHealth technologies that have proven to be successful, there is little evidence to show that such tools would continue to be successful beyond the contexts in which they were originally developed. These are the key findings of a study by Aziz Sheikh (University of Edinburgh, Edinburgh, Scotland) and colleagues, and published in this week's PLoS Medicine.

In the study, the authors systematically reviewed the published systematic review literature on eHealth technologies and evaluated the impact of these technologies on the quality and safety of health care delivery. The 53 reviews (out of 108), that the authors selected according to their criteria and critically reviewed, provided the main evidence base for assessing the impact of eHealth technologies in three categories: 1) storing, managing, and transmission of data, such as electronic patient records; 2) clinical decision support, such as e-prescribing; and 3) facilitating care from a distance, such as telehealthcare devices.

The authors found that the evidence base in support of eHealth technologies was weak and inconsistent and, importantly, that there is insubstantial evidence to support the cost-effectiveness of these technologies. They also found some evidence that introducing these new technologies may sometimes generate new risks, such as prescribing practitioners becoming over-reliant on clinical decision support for e-prescribing or overestimate its functionality, resulting in decreased practitioner performance.

More here:

http://www.medicalnewstoday.com/articles/213661.php

The article also got coverage in Time Magazine

Are Electronic Health Systems Cost Effective? Not So Much

By Alice Park Wednesday, January 19, 201

If you've visited the doctor or a hospital recently, you can't help but notice how much of your care depends on some form of electronic information exchange. From the prescription your doctor writes to the chart she consults, medicine is very definitely going digital.

But how effective is all this electronic data capture? Is it making the health care system more efficient? And what about patient care — are e-health technologies improving health outcomes for people who are sick?

Unfortunately, the answer is no, according to research published in PLoS Medicine. And this is despite the billions that governments like the U.S. have poured into such technologies — the Obama administration approved $38 billion to digitize the American health care system.

After analyzing 53 reviews of electronic technologies in health care, researchers led by Dr. Aziz Sheikh at the University of Edinburgh report that there is little or weak evidence to support the massive investment that policy makers have made in electronic systems such as electronic health records and computerizing physician orders and other decision-making. The strongest evidence in support of digitizing medical information came in electronic prescriptions, which showed a small benefit in reducing errors and streamlining ordering.

More here:

http://healthland.time.com/2011/01/19/are-electronic-health-systems-cost-effective-not-so-much/

There is a very interesting comment on the Time Article

Paul Shekelle

As the author of 3 of the reviews summarized in the original PLoS article, I am concerned that readers of this summary, and of the PLoS article, might inadvertantly come away with the impression that the evidence is that electronic health systems don't improve patient care and are not cost effective. I don't think that's what the data support. Rather, I think the published data support that the evidence is mixed, but that in some institutions the electronic health system has been transformative in the way care is delivered, with real and measurable improvements in care. So the question is - how can these results be realized by more health care organizations? At the time of our original 2005 review, there were no examples published outside of the few leaders in the field. By the time of our 2008 update, there were published a small number of successful results at other institutions. This is a fast moving field - the PLoS "review of reviews" isn't going to capture what's happened recently, and will tend to give more emphasis to older studies - in some cases, more than a decade old. Almost nobody today uses the mobile phone or computer that they used 10 years. I don't think data from 10 years ago on the value of electronic health systems is very informative for what's going today, and what is possible tomorrow and next year.

You can read about the commenters biography here:

http://www.rand.org/about/people/s/shekelle_paul_g.html

My reaction to that comment is that Dr. Shekelle MD PhD is right on the money!

That view is further confirmed by downloading the file found here.

http://www.plosmedicine.org/article/fetchSingleRepresentation.action?uri=info:doi/10.1371/journal.pmed.1000387.s002

This file lists the papers that were reviewed and what I found impressive was just how many were over a decade old. These papers were written before not only the iPad but also the iPod. Technology and our experience of what works and what doesn’t has advanced quite a way since 1990 when some of the papers seen as contributing to the evidence reported here were written - meaning the work was done in the 1980’s when all sorts of things were yet to come - e.g. the graphical user interface.

The second point is that this paper is a review of reviews. At no point did the authors use primary sources as far as I can tell. This is not the way things sensibly work. Science works by building on previous direct observation and experience - not by reading about it in summaries and then constructing conclusions from summaries papers. Meta, meta analysis is just too silly for words when attempting to draw any conclusions from the findings of studies with ages of up to 20 years and scopes and objectives that are quite different.

Last point is that we know there are examples where e-Health really works - viz Kaiser Permanente in the US, a number of Scandinavian countries and elsewhere as well.

Frankly this paper is just an ill-considered distraction to the pursuit of how to do things better and learn from the mistakes made. Just looking at summary reviews does not get you there in my view.

David.