This appeared a few days ago.
Support for Testing in the #FHIR Specification
Posted on September 28, 2016 by Grahame Grieve
On the last day of the Baltimore meeting, a small group of us met to talk about the future of testing in the FHIR specification. In particular, the participants included both Aegis and Mitre, who develop Touchstone and Crucible respectively. Out small group made 3 decisions:
- We will work towards the adoption of a formal testing framework for FHIR interfaces
- We will simplify the existing TestScript resource
- We will draft and propose a TestReport resource
Taking these backwards:
The TestReport will contain a summary of the outcomes a test run – which tests were run, whether they passed, and references to a further information (logs, etc). It’s for helping to maintain current registries of implementations and outcomes. I’ll blog about this further once there’s a draft to look at.
I’ll comment on #2 in time, when we have a draft for people to consider.
For #1, a testing framework, we have several motivations for this:
- It’s well known that a problem with testing processes like IHE is that under the right care (e.g. configuration) and with some judicious off-road modifications, systems can pass their IHE testing – but that often doesn’t equate to conformant implementations in practice
- Many systems undergo stringent testing in-house in continuous builds, but real world integrations are extremely hard to test
- As a consequence, applications are often poorly tested prior to upgrade
- Every real clinical system I’ve worked with has had test patients so that staff can check how the system works. In Australia, custom is that the patients are called Donald Duck, Mickey Mouse, etc. Staff just have to magically ‘know’ that these are test patients, and they pollute stats etc a little
- One near exception is the Australian MyHR, for which there was no test system, and no possibility of test patients. In practice, people trying to use it had to use their own personal records as test patients (many told me that they did this, and bits of those records drifted into the media)
Lots more here:
Now the topic of the article is very important but from the PCEHR / myHR perspective, the very last paragraph quoted is was what attracted my attention.
First let me say this paragraph is hardly a mistake – it has to be a deliberate and planned revelation of just how badly managed the introduction of the PCEHR was.
Second, having had contact often with the author over the years, I have no doubt what is written is true.
So what this confirms is the hopeless incompetence we all suspected of NEHTA, and their managers in the DoH.
I wonder will the new ADHA turn out to be more professional? We can only hope.
The mess the DoH have made with the data release program must have one wonder.
David.
hi David, thanks for making a mountain out of a mole hill.
ReplyDeleteThe project control group made a choice to invest what capital it had - operating under political constraints, remember - on what features they could. Creating a test environment was not in the first round of development, but was made a priority as soon as the initial goals had been achieved. I do not see that my technical note - the lack of a working test environment, and the work around people used - justifies calling the whole project a total mess, and it wasn't my intent to attract that kind of follow up.
No doubt your post will get circulated widely, and people won't see my follow up comment. The PCEHR project, like every other project I know of - including those that I have or do run personally - was not prefect; we just aspire to do whatever we can given the constraints we work under.
update based on an offline comment: of course the project had it's own testing environment. Environments many. And still does. And the software vendors had (have) their own test environment. It was the end users / production users that didn't get a test environment right way.
ReplyDeleteI am sorry Graham, but a project like this (PCEHR), that handles peoples health information requires extensive testing and I don't think its making a mountain out of a mole hill at all. Testing of complex systems is vital as its almost never correct when the developers think it is. A $2 billion budget should have allowed testing and You can only release what you test. I might try experiments myself without testing, but they can't be released on the unsuspecting public. Even with extensive testing its rarely 100% correct but it is essential to maintain trust, and trust is already in short supply.
ReplyDeleteIts time technical people said no sometimes, like they should have with the QHealth pay roll.
Grahame,
ReplyDeleteYou appear to be missing the point. I concur with Andrew but would hasten to add that it was clearly a case of the blind leading the blind ... as far as I can see the rot started as a pipe dream by the health bureaucrats and the that the PCEHR would be highly valued and used by the GPs and other clinicians (in addition to patients), the grossly inadequate,ill-conceived procurement and implementation process (including - at best - novice contract negotiation with vendors), incompetent project management, total lack of meaningful consultation and engagement of the stakeholders (more like alienation of the clinical and informatics communities), lack of robust, transparent and accountable governance and public reporting(mostly obfuscation and cover-up), no cost-benefit or risk management strategy or planning... etc etc
The fact that there was no test environment for user acceptance testing is quite frankly dangerous for a project of this scale and clinical risk. This is just another indication of the scale of the arrogance,ignorance,ineptitude and negligence associated with this exorbitantly expensive lemon which the taxpayers continue to fund with absolutely no tangible benefits and all the risk ... I am sure that there are many (individuals and companies) that have (or continue to receive) remuneration from the government for the PCEHR/MHR project that would like to see this continue - nothing like a vested interest in keeping the gravy train rolling !!
Don't you worry Tim will fix it, it's not a technology issue just and change and communication problem. A few power points, a couple of booths at choice events and some photos shoots with innovative clinics and a few consumer stories will do the trick. I can see the lunch of the new digital health strategy involving leisurely strolls amongst the working class masses and perhaps a few snaps of baby holding.
ReplyDelete.... I am sure that there are many (individuals and companies) that have (or continue to receive) remuneration from the government for the PCEHR/MHR project that would like to see this continue - nothing like a vested interest in keeping the gravy train rolling !!
ReplyDeleteYou mean like ADHA?
October 12, 2016 9:09 AM
ReplyDeleteand ePIP.
David,
ReplyDeleteThe one key principle that seems to have eluded both DOH and the consultants and vendors working on the PCEHR / MHR is the principle of 'Duty of Care'.
Doctors, nurses and other registered health professionals have a legal duty of care.
This legal obligation is clearly not shared.
Hence, what is the point of over $1 billion of expenditure.
John Scott, surely to goodness Mr Madden, Secretary Bowles, Minister Ley should be obligated as the highest priority to make a crystal clear statement regarding this issue of 'Duty of Care'.
ReplyDeleteAnd even more importantly surely the AMA, the RACGP, the Medical Defense Organisations should also be insisting that unless and until this issue is thoroughly explained and covered to the satisfaction of the Defense Organisations health practitioners should not allow themselves to get any further involved in this project.
Stephen Duckett's investigation in Victoria into babies deaths was released yesterday. The findings are horrific revealing a total breakdown in governance at the highest levels of Victoria's health system and its hospitals. Duty of Care right now, today, should be paramount but it has been neglected on so many fronts.
John Scott - how right you are. What can be done?
October 14, 2016 8:52 PM - What can be done?
ReplyDelete"Duty of care relates to quality and safety of care" - says it all.
The Duckett Review found that the Department of Health and Human Services has failed to provide adequate oversight of quality and safety across Victoria's health services. It indicated gaps and failings in the quality and safety systems had been overlooked, and the action and leadership required from the Department to achieve change and prioritise patient safety had not occurred.
Equally well, the Federal Health Department and the ADHA also have a Duty of Care with My Health Record directly related to quality and safety of care. It is abundantly evident that as this does not exist it puts at risk every patient (consumer), health professional and hospital Board and administrators.
Why do the peak medical bodies and defense organisations remain silent on this?
...... because they won't do anything until there is a disaster - business as usual. The view is that whistle blowers and expert commentators don't need to be heeded until a disaster happens because there is no evidence it will happen until it happens. This blog is a prime example.
ReplyDeleteand it's not exactly hard to understand some of the stupidities and risks.
ReplyDeleteFor example, if you have no allergies or take no medication, you cannot put that information in your Personal Health Summary (I believe it's the same withe the Shared Health Summary)
There is a huge difference between "No Allergies" and "No allergy information has been recorded".
Whoever designed that bit of My Health Record doesn't understand the difference between Nil and Null. Data modelling 101.
Even a health minister should understand that when an emergency doctor or nurse looks at someone's health record and there are no recorded allergies that doesn't mean the patient has no allergies.
If there was a box that indicates "The Patient reports that they have no allergies" then the treating health professional knows that the treatment risks associated with allergies are low. Without that positive indication, the treating health professional has to be careful and potentially not give treatment that the patient needs.
Same for medication.
This goes to the duty of care issue where data should be as clear and unambiguous as possible.
And taking the same issue further (nothing uploaded) for all those people who have been "given" a My Health Record, does the fact that nothing has been uploaded mean that the patient has no health issues?
Is it a safe assumption that an empty record means no health problems?