This was written just moments before Minister Hunt spectacularly failed to become Deputy Liberal Leader (He got just 16 votes of 85 or so available). The problem still remains.
-----
How to cripple a large scale Government Information System with one simple decision.
Simple solutions to complex problems always create further problems. My Health Record is an example of this phenomenon. Digital Health is far more than simply automating old manual processes that document medical treatment prescribed by doctors.
The Health Minister’s decision to completely delete a person’s cancelled record is a simplistic solution to what seems to be a simple problem. What the Health Minister does not understand are the complex consequences of his decision to overturn one of the most fundamental requirements in the initial design stage of My Health Record. These early decisions are the most important in the design cycle and the hardest to modify once development and implementation proceed. An analogy is the position of a building on a block of land. It is much easier to move a building 10 cm before any construction has commenced than after it has been built and occupied.
Background
First some background on deleting data from a system as technically complex as My Health Record.
Most systems have functions to create, read, update and delete data, or CRUD. In the case of My Health Record it was decided early on that no data would ever be deleted. This decision was documented in the Concept of Operations, a document that the ADHA no longer makes available, but which is still obtainable from here [1]. It is worth noting that it has never been updated, although it does say “The Concept of Operations will be periodically updated as the development of the PCEHR System progresses”.
The Concept of Operations specified that:
“The PCEHR System will always ensure that individuals and healthcare providers are presented with the most recent version of a clinical document. If a prior version is available, individuals and healthcare providers will be given the option to access prior versions of clinical documents if they require.”
There is no evidence that this function has been implemented, there is no indication in the My Health Record itself of this capability and this function is not referred to in any of the material on the government’s website.
The decision not to delete any data has an enormous significance on the subsequent design and implementation of the system. It removes the need to develop approximately half of the normal functionality. There is no need to update the data; old documents can be replaced by new documents and the old ones kept, but hidden from normal viewers. Likewise, there is no need to delete data, just flag it as hidden.
Even if a document has been uploaded to a patent’s MyHR in error, it cannot be deleted, it is “effectively removed” as the Concept of Operations quaintly describes it.
Back-up and Restore
Apart from simplifying the functionality of the system, the no delete decision also makes back-up and restore much easier.
There are two types of back-up and restore.
The first is to protect against catastrophic failure of the system. This is a Disaster Recovery function and involves taking a regular snapshot of the whole system along with partial back-ups in‑between major backups. This is so that, in the event of a failure of the system for whatever reason, the system could be re-built either in the same premises or in alternative premises and the system restored to its state before the event.
The second is to allow deleted information to be recovered after the deletion. In Microsoft Windows this is like using the Recycle bin. If you don’t want to keep a deleted copy, you can delete the document immediately.
New Problems
By making the decision to delete records of people who wish to cancel their registration, the Minister has created multiple problems.
To immediately delete a complete record from an operational system is actually very unsafe. If the System Operator accidentally deletes a record (for whatever reason) then they can't get it back. If they do a recycle bin delete, they are not complying with the legislation. The normal process is to flag the record as deleted and/or keep a log of the deleted data, once again not complying with the legislation.
It is very unusual for a Disaster Recovery backup to be modified. Its value lies in the integrity of its data. In a system as complex as My Health Record, an individual’s health record will be spread out over multiple database tables, so it is not just a case of deleting a line in a file or a row in a spreadsheet.
Deleting data safely from a Disaster Recovery backup would probably require downloading the back-up to another version of the operational system, restoring the data, and deleting the data using system functionality that ensures the data retains its integrity. Apart from the fact that currently there is no system delete functionality, interfering with the Disaster Recovery back-up is highly risky. However, the Minister has promised that a health record would not be retained by the government. What the Minister has announced will happen; it will be costly and cumbersome – if done properly.
It should be noted that the proposed legislation only applies to complete records. What about the potential need to delete documents in a current record? The big problem here is that the people have not been asked if they want it or not.
They have not been asked if they are happy to have old documents available for the System Operator to see (nobody else can see them, or maybe they can, we don’t really know) or if they want old documents deleted by request.
This means that the Minster has created a big problem for ADHA with his changed requirements and created the possibility that he has created an even bigger, but uncertain problem regarding old documents.
The functionality required to delete individual documents could probably quite reasonably be incorporated as is commonly implemented - flag the document as deleted and then purge it after a given time. When creating a Disaster Recovery back-up do not include deleted documents.
These are just suggestions; it needs a complete requirements analysis and identification of the most efficient strategy - the sort of thing that should always precede any design and implementation activities.
One of the issues is making sure the requirements are complete. For instance, what are the requirements regarding records that have already been de-activated? Should they be deleted from the operational system? From old Disaster Recovery back-ups?
If old documents are to be deleted from current records how would that work?
What are the requirements for documents and/or records that have been downloaded to other systems connected to My Health Record but which people might want to have deleted?
Doing such things after six years of operations is challenging at best.
One approach is to completely redesign the system, something that ADHA seems to be considering as part of its re-platforming initiative.
One hopes that someone in government asks the question: What is the return on the $2b that has been expended on My Health Record that indicates that another $1-2b can be justified?
The statistic that ADHA has provided that only about 20% of the 5.9 million registrations have a Shared Health Summary, along with the unexpected negative reaction in many quarters to the opt-out initiative should give pause for thought. One can but hope.
Reference
[1] Concept of Operations http://content.webarchive.nla.gov.au/gov/wayback/20140801043103/http://www.yourhealth.gov.au/internet/yourhealth/publishing.nsf/Content/CA2578620005CE1DCA2578F800194110/$File/PCEHR-Concept-of-Operations-1-0-5.pdf
----- End article:
It is all still spot on and as for what to do with audit trails that have been backed up - let's not even think about it! Minister Hunt has left a doozy for his successor!
Thanks Bernard!
David.
4 comments:
Bernard, very well articulated. Not many people understand those concepts unfortunately.
Laughably, the explanatory memorandum to the legislation change states there is no cost involved! Accenture/ADHA must be going to do all that redesign etc for free.
Extremely well presented Bernard. A significant contribution. It is most unlikely the ADHA, the Department or any politician will comprehend the significance of your argument. Even so, getting it down on paper and making it publicly available for the few who might be able to put it to good use is a hugely important step.
I remain convinced that with the right people, with the right mindset, the job can be done, provided they have access to the right leader. In my view there are only 4 or 5 people in Australia with the right leadership skills, knowledge and insight, to lead Australia out of this unmitigated mess.
.... however, unless and until, the many peak bodies desist from their insincere and ill-informed utterances of reassurance there will be no change to the status quo.
Just about all off-the-shelf software only does "soft delete" these days. Storage is so cheap and it is simply easier to keep everything for all the reasons Bernand states. There was an article on The Conversation (https://theconversation.com/my-health-record-deleting-personal-information-from-databases-is-harder-than-it-sounds-100962) recently about exactly this topic.
However, from an architectural point of view - the current approach is that Information is a corporate asset. It is essential for doing business and like an office or stationary it needs to be managed.
Personal or confidential data - such as Health information - can be better described as a toxic asset. Like, for instance, sulphuric acid in a manufacturing plant. Absolutely essential but needs to be handled very carefully and you better have a disposal plan in place.
More topically - pharmaceuticals have to be carefully stored, tracked, used and disposed of. Health data *should* be treated the same way.
In short - deletion is a hard problem and retro-fitting it is even harder. But a well designed system would have had it built in from the start, but that doesn't mean it shouldn't be done.
Post a Comment