Quote Of The Year

Timeless Quotes - Sadly The Late Paul Shetler - "Its not Your Health Record it's a Government Record Of Your Health Information"


H. L. Mencken - "For every complex problem there is an answer that is clear, simple, and wrong."

Wednesday, January 24, 2007

Evidence Based Health IT – What a Good Idea!

In what was almost a throw away line commenting on the recent discussion regarding archetypes Dr Tim Churches of NSW Health offered the following remark.

"My view is that the practice of health informatics needs, desperately, to become evidence-based, otherwise we will continue to see hundreds of millions or billions of dollars being poured into the deployment of health information systems based on what is in the sales brochure, or based on tender responses, which tend to be just more elaborate versions of the sales brochures."

Now, while I may disagree that the only current driver of Health IT deployments are sales brochures I find the broad sentiment most compelling. Tim is right in suggesting that the decisions regarding adoption and use of Health IT should be on much the same basis as we seek to consider and manage other Health interventions (drugs, treatment protocols, medical devices and so on).

This led me to consider what such Evidence Based Health Care IT may look like. What follows are my initial thoughts on how the idea could be actualised.

First it would be necessary to decide on the type of evidence that was being sought and in what domain the decisions would be assessed. The health sector has many years experience in the design and execution of clinical trials and there does not seem to me to be any intrinsic reason why such an approach would not be useful.

One can easily envisage trials comparing manual methods of practice with computer supported approaches as well as trials comparing computer systems with varying levels of sophistication in terms of user interfaces, functionality and decision support capability.

An issue which would need to be carefully considered is how, in a trial situation, the quality of the system is distinguished from the skill of the user and possibly patient related factors. We need to recognise that Health IT is much more an enabler of service delivery rather than a provider of services per se and this fact would need to be reflected in the trial design.

Second it would seem reasonable to use the typical types of trial endpoints for evaluation of clinically relevant systems – perhaps modified and controlled in different ways as appropriate. The use of endpoints, such as clinical outcomes, also has the advantage of taking a holistic view of an intervention and not seeing a Health IT intervention purely as a technical exercise – but also as something that involves the patient and the clinician as well as the technology.

Of course it is possible that in some situations there will be the need to consider intermediate outcomes rather than direct impact on morbidity or mortality when considering, for example, differing decision support systems aimed at improving specific clinician behaviours.

Third there is obviously a place for technical trials of different technical approaches to clinical problems evaluated on both technical as well as clinical criteria. In the same vein one would also want, in some circumstances, to ensure parameters such as cost, impact on workflow or efficiency and so on were evaluated as a component of an overall assessment.

Fourth as experience grows one would see, hopefully the emergence of multi centre trials of particular generic interventions and development of appropriate strategies for the conduct of systematic reviews of important interventions. Later expertise would also be encouraged to develop in cost benefit assessments of the different interventions using criteria rather more closely modelled on the Pharmaceutical Benefits Review Committee than what is done by the Therapeutics Good Administration where the focus is more on safety rather then efficacy and clinical outcomes.

Fifth it is important to consider how best the human side of Health IT deployment and use can be researched and practical trials conducted to come up with the best ways to manage system selection, clinician resistance to change, workflow alterations and so on.

Sixth one could consider the establishment of a Cochrane Collaboration like clearing house to make widely available the outcomes of both sponsored research, systematic reviews and so on – as well as Health IT guidelines to assist organisations move forward with an evidence based Health IT Agenda

Last it might make some sense to have at least some research that looks back on failures and disasters (something much beloved of IT project managers) to come up with evidence based guidelines on what tends not to work in the process of selection and deployment of apparently well considered Health IT interventions and systems.

I recognise that it could be said that what is being suggested is little more than what is classically referred to as Health Technology Assessment (HTA). I would suggest the focus on the deployments of, and the evaluation and trial of Health IT against clinical objectives using an experimental model moves a good deal beyond traditional HTA.

Overall, I can see there could be a case built for establishing an academic centre for the development, evaluation and implementation of Evidence Based Health IT.

What do others think? A good idea?



Anonymous said...

Hi David,

I think that is what the IHE process is. Take existing standards and constrain them further and then define interoperability benchmarks that are testable.

What Australia is lacking is solid backing for the process. Medical-Objects has participated in this process at the last 2 HIC "connectathons" and it is a lot of work and requires solid financial backing from NEHTA or health department(s) to devise the scenarios and provide the facilities to actually make it happen.

Until we have compliant HL7 messaging at a basic level, semantic interoperability is a pipe dream. First prove you can reliably transfer compliant data, then build on that ability to start processing that information in a rich way. Realistically the only viable option in the medium term is to use our existing HL7 V2 standards and test compliance in a rigorous way. It's the obvious way forward, and the first test everyone must pass before we climb to the heights of semantic interoperability. We have AHML testing already, its time it was used. Perhaps all the state health departments should lead by example on this?

Dr David G More MB PhD said...

Hi Andrew,

I could not agree more with your comments. I plan a new post in the next few days summarising my thoughts around issues like this and the archetype discussion.


Trevor3130 said...

From The in-between world of knowledge brokering in BMJ by Jonathan Lomas:
The ultimate aim of people engaged in health research is to get the health service's workforce, its employers, and its suppliers to have knowledge of facts (as represented by research results) and to use these facts in their practices, policies, and products. How well organised is research to achieve this aim? And how receptive and oriented are health services to this aim? The answers seem to be "not well organised" and "not very receptive." The interpersonal connections needed to bridge this know-do gap are not yet in place.1 An emerging role therefore exists for knowledge brokers, supported by knowledge brokering resources and agencies, to fill the gap.

It seems to me that the essence of Lomas' thesis, that knowledge must be shared, is buggered by the Australian research environment. Competitive tendering for funding surely leads in the opposite direction from collaboration.

Unknown said...

I was reminded during an email exchange recently that this argument resembles the "neats" vs "scruffies" divide in the Artificial Intelligence community.

I have two observations:

1. Health Informaticians can't afford to have these kinds of turf wars given the limited resources being made available to us.

2. Because of these limited resources, it would appear that David's argument (backed up by Andrew's practical suggestion) is that we can't afford to be too neat.

Both sides of the neat vs scruffy debate make valid points. Perhaps we can learn something from the history of that long running debate.

Jon Hilton