Tuesday, June 16, 2015

The Opt-Out Trials May Be Much Trickier That Is Presently Believed. There Are Many Challenges I Suspect.

The latest news on the plans for the trials comes from Senate Estimates a week or so ago..
Here are the relevant bits (Skip down a page or so if you have already read it all)
Senator MOORE: What consultation has been had with the medical profession regarding the decision to shift to the opt-out system?
Mr Madden: The recommendations from the review were to increase participation in the system. The health community had said, 'If we had the majority of our patients in the system, we would be more compelled and likely to take this on and use it.' That came through in the form of submissions from the AMA, RACGP, Consumers Health Forum and others—
Senator MOORE: That was prior to 2014.
Mr Madden: That was in 2014, in the report. We did do some consultation directly with health care providers and the community between July and September 2014 just to confirm views about how that would work. The point I need to make is that opt-out, in the current budgetary decision, is to trial opt-out in at least two geographical locations to understand the issues and make sure that we have continued to maintain the consumer's or individual's confidence in the system and to understand the issues that might come with that. So we have not taken a decision to move completely to a national—
Senator MOORE: But you have made a decision to go to the trial of two opt-outs, which is a distinct change from the other process. This committee did an inquiry into the original legislation and the opt-in/opt-out model was a great point of contention at that time. So now, as a result of the review, we have gone with a trialling of opt-out.
Mr Madden: Yes.
Senator MOORE: What form are the trials going to take; has that been determined?
Mr Madden: We are looking at least at two trial sites. We are working with states and territories through the Australian Health Ministers Advisory Committee on the possible selection of sites. We need to find sites which are discernible so that people who are in the sites in the trials know that they are in the trials and people who are outside know that they are clearly not. So we will be doing consultation on the location of the trials. We will be trialling our communication processes and also working through education, communication and training for GPs and other health care providers in the trial sites. While the population and the individuals in those areas might have a registration, we want to make sure that the health care providers are engaged with that system as well. That is why it is important to work with the states, so that we have a connection through the public hospital system.
Senator McLUCAS: Are you proposing to use a PHN boundary for those trial sites?
Mr Madden: Not necessarily. The trial population that we are looking for across, again, a minimum of two and a maximum of five, would be about a million people. So it would probably be an amalgam of some PHNs and it could be based on postcodes that join a couple of PHNs together. We want to get a spread that includes lots of people or individuals and lots of GPs and specialists, allied and private, and public hospitals to get the whole connected community of health care providers for that community involved.
Senator MOORE: Have the terms of the trial been determined yet?
Mr Madden: No. Where we have got to at the moment is to describe the criteria that would pick out what those trial areas might be. We will be looking to appoint an independent person to create the evaluation criteria for that, certainly well before the trials begin.
And here:
Senator MOORE: I was just talking with you, Mr Madden, and also with NEHTA about the opt-out trials and in terms of the process you said you were going to look at two sites. Is that right?
Mr Madden: Yes. We are looking for a minimum of two sites. We do not want to go any larger than five. Again, we have criteria that would describe what would make for the best sites or not for the best sites. Did you want to know about the logistics of the opt-out?
Senator MOORE: Yes, I do, because of the process.
Mr Madden: By September we are looking to have the sites selected. In the funding for the opt-out trials or what we have called the participation trials, we have funding for education, communication and training for healthcare providers and certainly a heavy dose of communication for individuals in the areas so they are aware of what this means to them, what they get as a benefit, what their rights are and what they do if they choose to opt out. We would be looking to do that from early 2016 with the training starting around about the same time for the healthcare providers.
We then have a period for two months where we have a system available for the communities to inquire and get information about staying in the system or opting out of the system. They will have a system where they can indicate their expectation or their option to opt out of the system and after that two months we will create skeleton eHealth records for all of those people who did not choose to opt out. We will then give them six weeks or so for them to log into and take control of their records, if that is what they choose to do, because the eHealth system will still have the patient or the personally control aspects. They can still determine who can operate their eHealth record on their behalf, healthcare providers that might be allowed to or not allowed to upload records to their record and who can or cannot see particular records. They will have five weeks to take control of those and to put all of those controls in place if that is what they choose to do. Then about two weeks after that we will create the records. We are looking to target that for having records in the hands of healthcare providers and individuals after the controls have been set some time during July 2016.
Senator MOORE: So it is just over the year?
----- End extract.
So, in summary, after site selection we will have:
1. Education of providers and consumers on what is happening, what their rights will be to opt-out starting in early 2016 - with some training for providers in early 2016.
2. Two months will then be spent with consumers being able to try the system and inquire about and choose to opt-out etc.
3. Then everyone who did not opt out will have a skeleton record created within the system - presumably using data from the IHI service.
4. The opted in consumers will then have five weeks to take control of their record and set their desired access controls.
5. Two weeks after that the record will be populated with all the other information like, presumably PBS and Medicare claims data.
6. Providers and users will then have access and the system will be available for use a little over a year from now. Of course there will still be limited numbers of Shared Health Summaries and other useful clinical documents.
At present we do not seem to have information on:
1. How long the trials will actually run? - it would need to be a reasonable while - at least a year in my view.
2. Just how the trials will ensure the consumers (and providers) are actually engaged with all this - we are, remember, talking about a million people - many of whom will be children, homeless, lacking literacy or internet access, be mentally impaired or incompetent, be just who just ignore Government messages etc. (is no response treated as agreement to have a record or not?).
3. What the evaluation criteria for the Trials will be and how the trial outcomes will inform what happens next?
4. How will those who move in or out of a geographic trial area be detected and informed what is to happen to them and what they need to do?
5. How will those who reach an age of competence for independent decision making be alerted that they need to take some decisions and how will children who desire independence from parental access to their record be managed?
6. How will children who do not have control of their record and who want to suppress information (medications, referrals etc. that has been automatically loaded) from parents and others and who are old enough to know what they are doing, obtain the control they want of their record?
7. How will it be determined who controls an opt-out record if the patient is for some reason not able to take control for themselves?
8. What will happen if a disengaged consumer has information automatically loaded into the PCEHR they are concerned about being known to others and this information is then disclosed via clinician access to the record (or more likely their staff)?
9. How will control of created PCEHRs be adjusted in the event of divorce, family violence etc?
Feel free to add further concerns of your own regarding the trials. I am sure there are many others.
I have to say to have an unconsented record created and populated for you when you are a demented 90 year old in a nursing home or a fifteen year old who wants nothing to do with her abusive parents having access to her auto-created record seems pretty bizarre. Surely we need to have a positive response agreeing to opt in before record are created? If not the Government is setting itself for a ‘world of pain’ I reckon. I wonder has this been properly thought through?


Mayan said...

Recently, hackers acquired the security clearance details on most US government employees. The data includes everything from 127 page clearance forms, detailing the most intimate aspects of their lives.

If it is on a computer system, it will be hacked. It will become public, will be sold, and will be abused. This is quite apart from the potential for abuse by governments in the future.

Doctors who keep records on paper, kept in secure filing cabinets, protected by alarms will find themselves quite popular, methinks.

Anonymous said...

I often wonder whether the Prime Minister, The Treasurer, The Health Minister and the Minister for Human Services, would take heed of a well worded position statement signed by 50 or more experienced eHealth software developers and practitioners urging them to terminate this insane PCEHR profligacy.

Dr David More MB PhD FACHI said...

Hello Anon,

Is there a link to this 'well worded position statement'

We would all love to read it!


Anonymous said...

Why would these guys listen to any "experienced eHealth software developers and practitioners"? They seem to be deaf to any sort of expert advice, from the UN, the IMF & ICCC downwards.

Anonymous said...

Hello David - you have misinterpreted what I said - let me be a little clearer.

I often wonder if any good might come from preparing a well worded position statement, signed by 50 of more experienced eHealth software developers and practitioners, and then sending it to the Honorable Members mentioned above urging them to terminate this insane PCEHR profligacy.

Anonymous said...

Instead of geographical trials, perhaps a better approach might be to take a different slice of the pie? For example, focussing on health summaries and medications data for older clients in aged care homes might demonstrate some real benefits.

Anonymous said...

Might this, might that ...... I have consistently promoted a different approach for a fraction of the cost, which would have wide support by vendors and consumers, which would result in early, widespread deployment of some critical eHealth solution(s)which once embedded into the daily business processes of service providers will not easily be displaced, and I have made this known to some bureaucrats and Health Ministers and their advisers on a number of occasions but, as always, deaf ears have consistently prevailed and they have walked away to do what they have always done thereby getting the same result. THEY DO NOT GET IT.

Anonymous said...

They are lost in the enormity of what they think they can achieve - small thinkers like June 17, 2015 8:38 AM have no place in their world. In my experience many of the 'small' thinkers are wise, experienced, smart and competent and they DO get it. The problem here is the bureaucrats see them as a threat and always have - how dare these small thinkers - 'experts' - try to tell us in power with all the money to spend that we have got it all wrong', how dare they.

Bernard Robertson-Dunn said...

I went to my doctor's today. I asked him if he uses the PCEHR. He said - no, it's useless.

He said the problem is that "they, the developers" never asked the doctors what was needed.

No amount of training of doctors will get them to use the PCEHR if it doesn't do what is needed.

And no amount of "usability" will help, because "usability" isn't the issue.

But the trials will be declared a success because they are not testing usability they are testing "the implementation of opt-out to ensure public confidence in the system was maintained".

Considering the public confidence is so low, it will probably be maintained.

Grahame Grieve said...

The developers did ask the doctors what they needed. They got lots of answers, and then they picked some. What did your doctor think would be useful?

Anonymous said...

If they 'did ask' doctors for their needs, and then 'picked some' I guess they used a dart board to make the choice?

Unless they had deep experience in clinical informatics, then one answer from a clinician might very well seem as important as another. Sifting through user needs to come up with compellingly useful design is a different skill set to being an expert in the engineering mechanics of how that design is built. I fear the software engineers ticked themselves into thinking they were also interaction designers.

Bernard Robertson-Dunn said...

We didn't have time to go into what he thought would be useful.

Bernard Robertson-Dunn said...

Re what doctors and others want....

There is a huge difference between automating what currently happens, incrementally addressing existing problems/issues and completely changing the way the whole system works.

I contend that if you expect people who are expert in what they currently do to tell you a much better way of doing something (that might in fact make what they do obsolete) then you are fooling yourself.

In the case of the PCEHR all they have developed is a different sort of filing cabinet that doesn't even contain all the information a health professional needs to do their job, never mind support a new and better way of delivering health.

Anonymous said...

The developers did ask the doctors what they needed. They got lots of answers, and then they picked some."

I beg to differ. Key feedback on the original Concept of Operations by doctors and clinical peak bodies voiced their concern about incomplete records with 'blocked' documents. Also they advised that inclusion of PBS and MBS claims data was not only useless, but not safe to include in the record. They warned against not considering clinical workflow; and they specified that they needed a composite view of data in a record, with allergies, medications etc - not multiple documents that would need to be navigated and compared.

The problem was not that the doctors weren't asked, but that their advice was ignored by DOH. Developers cannot necessarily be blamed, as they were required to comply with NEHTA solution architectures and specifications, and there was no traceability back to the Concept of Operations and feedback from doctors. It would be an interesting exercise to compare what has been delivered with the original Concept of Operations and clinical feedback received.

Bernard Robertson-Dunn said...

If you look at the document map in the PCEHR High Level System Architecture document, available on the NEHTA website, you can see that the Concept of Operations is the primary source document for the system and feeds into everything else, including Business Requirements.

It also feeds into a High Level Design which in turn feeds into the PCEHR High Level System Architecture.

When I was at IBM, I was certified as an IT architect, which is accepted by the Open Group as being equivalent to their certification (which is much more rigorous than TOGAF certification) , so I claim some reasonable degree of competence in this area.

I have never seen any sort of methodology or framework that could produce the document map in the PCEHR High Level System Architecture.

Best practice starts with business requirements, develops various architectures and finally moves into Design.

The document map indicates that everything is backwards. It's almost as though they have unilaterally decided on a solution and then produced a set of documents that appears to the uninitiated as though they have followed some sort of methodology.

Anonymous said...

Was said at the time - A concept of operations document was a curious choice and said much about what the architects/software developers at NEHTA thought the problem was.

Bernard Robertson-Dunn said...

What is even more curious in that document map is that at the top it has a series of headings that belong to a Zachman style approach and down the left are the words Conceptual, Logical, Implementable, which belong to a Structured Analysis approach, although Physical is more often used than Implementable.

IMHO, not the work of an experienced architect. And I would have thought that a project supposedly trying to develop a long lasting component of a national infrastructure initiative would have needed quite a lot of brainpower and experience.

Did it get it? The evidence would suggest otherwise. Although it could be that managers (both project and public sector) were really at fault.

Grahame Grieve said...

Bernard, I believe those terms are taken from the NEHTA Interoperability Framework (https://www.nehta.gov.au/implementation-resources/ehealth-foundations/interoperability-framework). I'll see if the author wants to (and can) contribute here

Anonymous said...

Looking at this set of documents:

1. Interoperability Framework Version 2.0 — 17 August 2007
2. Concept of Operations: September 2011 Release
3, High-Level System Architecture PCEHR System Version 1.35 — 11 November 2011
4. eHealth Interoperability Framework Version 1.1 30 April 2012

They are not compatible or consistent.
A Concept of Operations is not mentioned in document 1, is referred to in document 3 and mentioned and defined in document 4.

However the definition in document 4 bears little or no relationship to the actual Concept of Operations.

The terms Conceptual/Logical/Implementable are not mentioned in docs 1 and 2 but
is defined in doc 3 and mentioned 4.

However the definition of Conceptual/Logical/Implementable is not consistent with normal architectural modelling. In doc 3, the High-Level System Architecture, Conceptual is defined as being Stakeholder centric and Implementable as Platform Specific.

Accepted architectural practice states that each of the four major architecture types Business/Application/Data/Technology can have Conceptual/Logical/Physical perspectives.

There is an interesting footnote in the High Level System Architecture on page 1, under Architectural Approach

"The NEHF (National eHealth Framework) differs from other popular frameworks such as TOGAF. TOGAF is more of a process-oriented framework for creating and managing architectural artefacts. NEHF is a specification framework used to describe system architectures. NEHF. and the SAIF framework it is based on. are strongly influenced by ISO 10746, which is an international standard reference model for open distributed processing (RM ODP). The viewpoints and levels of abstraction in the NEHF are more similar to the categories that underpin the Zachman framework. However, RM-ODP also provides a specification language that is compatible with UML."

This rather strange set of statements appear to be at odds with doc 1 which states that TOGAF is to be adopted. Doc 2 has no mention of TOGAF and doc 4, has TOGAF9 in the References section but is not actually mentioned in the rest of the document

NEHF is not mentioned in doc 1, 2, 4, but is referred to in doc 4 the High Level Architecture.

What is even more interesting is that there is no document called the National eHealth Framework available on the NEHTA website.

I really can't make sense of the architecture stuff that is publicly available; either the approach or the architecture documents themselves.

And what is really missing is a system development methodology. An architecture framework is really just a table of contents for architecture artifacts, maybe with a guide to their production. A system development methodology is the fundamental approach in which architecture is just one component.

A search of the NEHTA website does not return anything remotely like a development methodology.

Anonymous said...

Let me get this straight. Grahame, are you saying that there was only a *single* author of the NEHTA interoperabilty framework? If so, that's just head-shakingly wrong.

Anonymous said...

..... and the architect and author was .........

Grahame Grieve said...

I believe there was a single lead author, yes. Of course there was a much wider design and review team. FHIR works similarly, though there are many authors now. But you should all assume that program timelines trumped any quality processes that any managers or architects wished for.

Dr David More MB PhD FACHI said...

"But you should all assume that program timelines trumped any quality processes that any managers or architects wished for"


Hardly an excuse for the work that was not up to scratch and not properly reviewed.

The program timelines were nonsense as all involved and the serious watchers knew. Ms Roxon has a huge amount to answer for IMVHO. Another case of a politician 'not knowing what they didn't know' and rather supine technical staff.

Sadly I think it has got worse since!


Grahame Grieve said...

How can unrealistic timelines not lead to compromises in the work? Other than everyone quitting, what do you think will happen?

Dr David More MB PhD FACHI said...

It totally depends on your views on accountability and ethics. Everyone is different. I hope I would have walked out - but how can you ever know? Everyone to their own lights.


Anonymous said...

An even better alternative to walking out was the decision by some experts who were not prepared to compromise on their standards, ethics, integrity and professionalism, who would not prepared to join NEHTA because, after extensive investigation, it was apparent that they would no be able to help NEHTA due to the megalomaniacal sycophantic behaviour that was being exhibited by those in charge. There is an abundance of evidence now available to prove the wisdom of those experts decision.