Quote Of The Year

Timeless Quotes - Sadly The Late Paul Shetler - "Its not Your Health Record it's a Government Record Of Your Health Information"


H. L. Mencken - "For every complex problem there is an answer that is clear, simple, and wrong."

Wednesday, November 15, 2006

Clinical Software Certification – What’s Practical, Necessary and What Makes Sense?

In the last few months there has been some correspondence in the Medical Journal of Australia and elsewhere on the topic of the safety of clinical software and the possible need for certification of such software.

In this commentary I want to consider this suggestion from a range of perspectives including practicality, barriers to implementation, likely impact on quality of care and so on. I would also like to point out that from my perspective, while I believe it is vital, certification of clinical software is likely to prove clinically challenging, technically complex as well as commercially contentious.

Before going any further it is important to say that I recognise the inherent difficulty and complexity of the topic and need to take a perspective that ignores the system user – recognising the unreality of pretending areas such as user skill, attention, competence and experience are not important in the overall outcome. What I seek to achieve here is to try and identify the different components of an approach that may address the issues around having the practitioner be confident that the system they are using is providing the best possible aid and support for their care delivery.

What attributes are needed for the purchaser and user to be sure this to be so?

I would suggest the following are important.

1. Data Model

The target system needs to have a sufficiently rich data model to support both the usual data capture required in a clinical EHR system as well as the detailed – and hopefully structured information required for automated decision support. This means that at the very least crucial laboratory results (e.g Haemoglobin, Creatinine etc) will be captured in atomic form.

Clearly there is also the need to address coding / terminology in appropriate areas (e.g. diagnoses, medications etc) to ensure useful values are held and are computable to support clinical decision making.

2. Functionality

The functionality of the EHR system that is required for quality care delivery has been the subject of much work over the years and is quite well understood. The requirements contained in the General Practice Computing Report in Australia and the HL7 and CCHIT specifications for the US provide quite reasonable guidance in this area.

Clearly the full scope of these requirements including clinical decision support and clinical pathway management and tracking are required.

3. Knowledge Database(s)

While vital, here we arrive at a major difficulty. This is at least one area where the quality, scope and depth of what is required is likely to prove less than straight-forward to define.

4. Interoperability

As attractive as ‘vendor lock in’ may or may not be to commercial software providers it is anathema to software customers and users. For this reason all information stored in the EHR should be exportable in a standard – and preferably standardised form to any certified competing product.

5. A Commercial Future

With use over time the value of the information held in a clinical system becomes progressively larger. The purchaser / selector of a system, which will be expected to be operational for ten years or longer, needs to be confident of the commercial viability of the software provider in terms of maintenance, updates, recurrent costs and so on.

6. Messaging and Communications Capability

Systems these days do not live in a vacuum and they need to be able to securely transmit and receive both clinical and administrative information in a seamless and standardised fashion.

7. Usability

It is important any systems should be engineered to be both easy to use and also to be designed using the principles applied to aircraft-cockpit design where the important information cannot be overlooked easily or without warning.

It should be noted that while the emphasis in this article has been on ambulatory systems all the forgoing is just as relevant to the hospital and other larger provider sectors.

The next issue is how is assurance obtained that these attributes are indeed present and are of the quality etc required.

Given the current political situation it seems to me that what is required is a CCHIT (Certification Commission for Healthcare Information Technology in the USA) like organisation, funded by, but at arms length from, Government that can work with users and all the other stakeholders to develop the requirements and criteria in each of these areas and then offer incremental and reasonably in-expensive paths to certification.

The CCHIT approach of developing roadmaps of requirements and capabilities to be delivered over a known time-frame I see as remarkably sensible and pragmatic and well worth adoption.

Clearly the Australian CCHIT would need to work collaboratively with both system developers, academia, clinicians and standards bodies to develop and then assess systems against the agreed criteria. This said it would also be important to have very professional leadership of the commission in place to ensure there is no easy route for less than high quality systems to be certified. A half baked process would be far worse than no process at all.

My guess this would be of the order of a five year project to get where we need to be with initial certifications possible in two to three years.

To not go down this path will leave individual purchasers and vendors possibly, indeed almost inevitably, liable for possible errors of omission or commission. The Government must really act to provide appropriate coverage for all those in the E-Health sector.

The other step I believe is required is that there be financial incentives for the clinical users to both install and then use the advanced systems as we know from the study reported yesterday that it is only the actual use of capable quality systems that makes a real quality and safety difference.

The business case for the Commonwealth to do this in the Ambulatory Sector I am sure would be totally compelling!



Anonymous said...


very thoughtful article.

While the UK and USA governments have taken the lead on certification issues, there has been much work done on development of implementation profiles and conformance testing and certification with Integrating the Healthcare Enterprise (IHE)

Over the past two years I have been associated with the HISA/HL7 Interoperability Demonstrations where an increasing and significant industry presence (by numbers and organisation size) has shown a committment to implementation and use of standards and public demonstration of this. While not formally associated with IHE, we have attempted to use their user and industry engagement approach.

We have to be careful about unthoughtful application of overseas models to Australia, yet there is much to learn and potential benefits in international collaboration.

Peter MacIsaac
HISA/HL7 Interoperabilty Coordinator 2005-6

Anonymous said...

I've been programming medical databases for 13 years now, and can I remind you that a lot of medical programs were originally written by clinical staff who picked up programming along the way. Any certification requirements will be anathema to local innovation in hospitals. Certification is a useful tool to distinguish vendors, but legal requirements for certified software would basically lock users into buggy and obsolete software. It would also be pointless since modern operating systems are patched frequently, other programs on the system change and you can only validate a product on a particular hardware and software configuration at a point in time.

If hospitals really want the best software, they should have a fixed price maintenance contract (so that the vendor has an interest in reducing bugs in the system to reduce his costs) with a time-and-materials clause for new features (so that the vendor can afford to design and test programs properly). Intricate certification requirements will increase a vendors costs, taking resources away from development and testing budgets. When the certification requirements are highly intricate, certification becomes a marketing exercise that keeps out new entrants to the industry.

Anonymous said...

If you like sausage, you should not look too closely how the sausage is made.

Conversly, if doctors practiced medicine the way software authors code, most doctors would be in jail.

Software in the main is created by self taught maverics, with minimal if any formal design methodology, oblivious to standards and software engineering techniques.

Im full of excreta?
Here is a test for you... ask your software vendor for a data dictionary, if you get anything else than a printout of table definition I will personally slice off my left testicle.

Software writers use seat of the pants coding, it runs, ship it.

Software Certification is a necessity not a luxury.