This appeared last week:
AI systems should be accountable, explainable, and unbiased, says EU
The European Union has published new guidelines on developing ethical AI
The European Union today published a set of guidelines on how companies and governments should develop ethical applications of artificial intelligence.
These rules aren’t like Isaac Asimov’s “Three Laws of Robotics.” They don’t offer a snappy, moral framework that will help us control murderous robots. Instead, they address the murky and diffuse problems that will affect society as we integrate AI into sectors like health care, education, and consumer technology.
So, for example, if an AI system diagnoses you with cancer sometime in the future, the EU’s guidelines would want to make sure that a number of things take place: that the software wasn’t biased by your race or gender, that it didn’t override the objections of a human doctor, and that it gave the patient the option to have their diagnosis explained to them.
So, yes, these guidelines are about stopping AI from running amuck, but on the level of admin and bureaucracy, not Asimov-style murder mysteries.
To help with this goal, the EU convened a group of 52 experts who came up with seven requirements they think future AI systems should meet. They are as follows:
- Human agency and oversight — AI should not trample on human autonomy. People should not be manipulated or coerced by AI systems, and humans should be able to intervene or oversee every decision that the software makes.
- Technical robustness and safety — AI should be secure and accurate. It shouldn’t be easily compromised by external attacks (such as adversarial examples), and it should be reasonably reliable.
- Privacy and data governance — Personal data collected by AI systems should be secure and private. It shouldn’t be accessible to just anyone, and it shouldn’t be easily stolen.
- Transparency — Data and algorithms used to create an AI system should be accessible, and the decisions made by the software should be “understood and traced by human beings.” In other words, operators should be able to explain the decisions their AI systems make.
- Diversity, non-discrimination, and fairness — Services provided by AI should be available to all, regardless of age, gender, race, or other characteristics. Similarly, systems should not be biased along these lines.
- Environmental and societal well-being — AI systems should be sustainable (i.e., they should be ecologically responsible) and “enhance positive social change”
- Accountability — AI systems should be auditable and covered by existing protections for corporate whistleblowers. Negative impacts of systems should be acknowledged and reported in advance.
You’ll notice that some of these requirements are pretty abstract and would be hard to assess in an objective sense. (Definitions of “positive social change,” for example, vary hugely from person to person and country to country.) But others are more straightforward and could be tested via government oversight. Sharing the data used to train government AI systems, for example, could be a good way to fight against biased algorithms.
Lots more here:
As high level principles these seem to me to capture what we are wanting to see from AI.
I note the CSIRO is Australia is pursuing similar work.
CSIRO promotes ethical use of AI in Australia's future guidelines
For Australia to realise the benefits of artificial intelligence, CSIRO said it's important for citizens to have trust in how AI is being designed, developed, and used by business and government.
The Commonwealth Scientific and Industrial Research Organisation (CSIRO) has highlighted a need for development of artificial intelligence (AI) in Australia to be wrapped with a sufficient framework to ensure nothing is set onto citizens without appropriate ethical consideration.
Data61, CSIRO's digital innovation arm, has published a discussion paper [PDF], Artificial Intelligence: Australia's Ethics Framework, on the key issues raised by large-scale AI, seeking answers to a handful of questions that are expected to inform the government's approach to AI ethics in Australia.
Highlighted by CSIRO are eight core principles that will guide the framework: That it generates net-benefits, does no harm, complies with regulatory and legal requirements, appropriately considers privacy, boasts fairness, is transparent and easily explained, contains provisions for contesting a decision made by a machine, and that there is an accountability trail.
"Australia's colloquial motto is a 'fair go' for all. Ensuring fairness across the many different groups in Australian society will be challenging, but this cuts right to the heart of ethical AI," CSIRO wrote.
CSIRO said that while transparency and AI is a complex issue, the ultimate goal of transparency measures are to achieve accountability, but that the inner workings of some AI technologies are not easy to explain.
"Even in these cases, it is still possible to keep the developers and users of algorithms accountable," it added. "On the other hand, AI 'black boxes' in which the inner workings of an AI are shrouded in secrecy are not acceptable when public interest is at stake."
Conceding that there is no one-size-fits all solution to the range of legal and ethical implications issues related to AI, CSIRO has identified nine tools it says can be used to assess risk and ensure compliance and oversight.
These include impact assessments, reviews, risk assessments, best practice guidelines, industry standards, collaboration, monitoring and improvement mechanisms, recourse mechanisms, and consultation.
These include impact assessments, reviews, risk assessments, best practice guidelines, industry standards, collaboration, monitoring and improvement mechanisms, recourse mechanisms, and consultation.
More here:
The good thing is that there seems to be a sense of alignment with these various initiatives, and that can only be a good thing I believe.
Worth browsing the fuller reports.
David.
21 comments:
The WHO has released a set of recommendations regarding the adoption of Digital Health
https://apps.who.int/iris/bitstream/handle/10665/311941/9789241550505-eng.pdf
The recommendations are (page xix):
1. birth notification via mobile devices
2. death notification via mobile devices
3. stock notification and commodity management via mobile devices
4. client-to-provider telemedicine
5. provider-to-provider telemedicine
6. targeted client communication via mobile devices
7. health worker decision support via mobile devices
8. digital tracking of patients’ health status and services via mobile devices
9. digital tracking combined with: (a) decision support and(b) targeted client communication
10. provision of training and educational content to health workers via mobile devices (mobile learning-mLearning)
I have a few questions.
Suppose you are a practicing clinician. Will the above recommendations, if implemented:
1. Help you diagnose your patients’ conditions?
2. Help you recommend appropriate treatment for your patent in the knowledge that you understand the side effects the patient will suffer (not maybe, but you actually know) and have agreed with the patient that it is in their best interest? i.e. you have discussed how the treatment will work for the patient, not told them that the risk of something bad happening is lowered, possibly, maybe, maybe-not.
3. Will the recommendations help clinicians diagnose and treat rare diseases? (There are about 10,000 rare diseases which account for about 10% of health problems)
4. Change the way you practice clinical medicine?
I would be interested in the opinion of Andrew and any other clinician reading this blog re these questions.
I am just a humble engineer with a Masters and PhD in modelling biological systems. I am not a GP or practicing health worker, but I can do research, read and understand the discussions that are going on in other forums about the problems clinical medicine is grappling with. The one thing I do not see is an intersection between the clinician’s view of their world and that put forward by the technology evangelists who think they are improving the delivery of health services,
At best the technologists are proposing solutions that help heath care businesses, not do better medicine.
IMHO, the WHO is far away from transforming the practice of clinical medicine. ADHA is even further away.
Wonder if any of them bothered to speak to the ethics society? I do agree David it is good to see people thinking about ethics but this is a complex area and one where I would suggest the experts are leading rather than hobbyist with best intentions
Bernard, I agree.
Those recommendations sounds like the are primarily aimed at LMICs (lower-middle income countries) and so I think they are not particular intended to be appropriate/relevant here
Grahame,
From the guidelines:
"The Target Audience
The primary target audience for this guideline is decision-makers in ministries of health and public health practitioners, to aid them to develop a better understanding of which digital health interventions have an evidence base to address health system needs.
This guideline may also prove beneficial to organizations that invest resources into digital health systems as implementation and development partners.
This document aims to strengthen evidence-based decision-making on digital approaches by governments and partner institutions, encouraging the mainstreaming and institutionalization of effective digital interventions within supportive digital systems."
Looks like the world to me.
well, yes, you can't actually say what I said in the document.
Bernard, you have highlighted the way the system works.
The misconstrued practices often employed and underlying "evidence-based" is to give credibility to something (where often there is no supporting evidence) upon which to make the evidence-based claims.
Hence the statement that "to aid them to develop a better understanding of which digital health interventions have an evidence base to address health system needs" is intended to convince the reader that the "digital health interventions" referred to have an evidence base.
.... and so the snowball rolling down the mountain gets bigger and bigger as more people refer to 'this' highly credible source which we know 'must be' credible because it claims to be evidence-based; ie. "This document aims to strengthen evidence-based decision-making on digital approaches by governments and partner institutions, encouraging the mainstreaming and institutionalization of effective digital interventions within supportive digital systems." ... and so it takes on a life of its own!!!
In short it's all bulls--t rhetoric employed all too often to convince the gullible that what is being claimed is evidence-based. Some call it marketing.
The WHO is not the only culprit. In Australia ...... you be the judge.
I do have a passion for decision support and have actually implemented a CDS language, "GELLO" which we still use under the hood. The issue is garbage in garbage out and unless you have high quality data using it for decision support is quite risky. In reality the base level compliance and quality at the message level is vital. Once you have that Terminology is the next level.
We have neither of those things. Messaging is also built on message compliance. All the things we want to, and all the things that add value depend on good quality, reliable data and its not been addressed. This is castles built on quicksand which is something the ADHA have a lot of expertise in. It all works on power point so lets ship it? Mobile is an extension of a working system to a new device, first we need a working system.
IMHO:
1. If you believe this (Yes it's the USA but probably not very different in Australia):
https://www.healthdatamanagement.com/opinion/are-we-working-to-improve-healthcare-or-health?brief=00000157-c311-d2b6-af57-cb9929c60000
Of the social determinants of health, healthcare represents only 20%.
2. Digital health is doing little, if anything to help doctors practice more effective and efficient healthcare.
3. myhr is a very poor example of digital health.
4. There is no evidence that giving patients access to a health summary they themselves are responsible for will deliver any improvements in their health.
It should be obvious even to blind Freddie that the government is wasting billions of dollars on myhr with no prospect of getting any health benefits for the population.
Why?
Here’s a guess. It is easier to spend money on technology backed by empty promises and “potential” than to address any of the real determinants of health.
The back room deals are starting to surface. http://www.mygc.com.au/doctors-slam-reckless-qld-pharmacy-trial/
I for one will be resisting this by only allowing my GP to coordinate my care. Not because of the pharmacist but because of the obvious tactics being used here.
even less reason for GPs to use MY HR. Apart from making it easier for a patient to go to another GP, they won't want pharmacists taking business away from them.
Neither will they want their jobs made harder because some enthusiastic but ignorant pharmacist stuffs up.
I wonder who is responsible for stuff ups.
And if anyone belives that it will stop at a few drugs, they need their head exaamining.
Pharmacist and doctors go down different learning paths and are highly skilled in the respective (but different) professions. The muddling of the two and erosion of the separation of concerns seems to be driven by dark constraints and fail to see the interests of the patient and either profession being the core driver here.
A question to any pharmacist out there – why would you want to go down the path of hobbits parent to be doctor?
This seems to be a step away from a collaborative model, one where after seeing my GP and gaining a diagnosis, my GP can let the pharmacist prescribe, and this could be where the pharmacist clinically checks every script that comes out of my GP clinic and has the authority ( through a legal agreement between clinical and pharmacy) to change the prescribed product if agreed/understood by my GP beforehand (and have them notified).
I recall in a simpler time before MYHR became malignant there was discussions on moulding a role for the pharmacists as medication management specialists, a partner in a patient journey and is seen as another specialist. My GP could then refer me to he medication use specialist after all the pharmacist will (or should) be up to date with the latest treatment knowledge, products, strengths doses, warnings etc……this is the real value add for me in having a pharmacist as another specialist. It might help separating them from the image of a pharmacy which is fast becoming a retail outlet.
No I won’t stop at ‘a few’ items. Back to the AI piece. I am happy to see lots of little sector groups forming around the next buzzword bonfire. However we really need this nationally coordinated rather than lots of little statements of principle and ethical manifestos sprouting up everywhere, somewhere down the track these will need to be harmonised and by that time every little group will have become maternal towards their take on the subject.
I would suggest and prefer all the effort and resources be centred on https://consult.industry.gov.au/strategic-policy/artificial-intelligence-ethics-framework/
Let get a little more holistic in our thinking and get some principles that actually can be applied to guide all organisation embracing eve loving and powerful computing practices and technologies
@1:18 PM. You make some good points. Just for clarity ‘A question to any pharmacist out there – why would you want to go down the path of hobbits parent to be doctor?’ did you mean ‘hobbyist professing to be a doctor’?? If so might be a bit harsh but a fair question.
Where is the nationally agreed model of the role of a GP and the interactions they have with other parts of the health car system? In enterprise architecture terms it called a business model. It defines processes, data and data flows.
It is done in order to understand what a GP does and identifies the interactions with the context in which they operate.
It is done so that information systems can be built that support the GP.
It is done so that when the role of other actors outside the GP context change, the impact can be understood and, if appropriate, agreed by all stakeholders.
The fact that such a business model does not exist (not in public AFAIK) explains a lot about the ineffectiveness of myhr and the cavalier attitude to changing the role of pharmacists.
It's yet another simplistic idea with no analysis or understanding behind it.
Bernard these artefacts use to exist and were freely available. ADHA has systematically undertaken a purging of any historical information. The scorched earth policy is not restricted to public ally accessible information but also access and knowledge internally. It is as if history is being erased. This is a shame (and boarding something else) as all this national work was done in consultation with stakeholders. This is one of the few things I could scrape from the internet.
http://ict-industry-reports.com.au/wp-content/uploads/sites/4/2013/09/2010-NEHTA-Blueprint-v1_0-DRAFT.pdf
I'm not surprised ADHA is trying to hide documents like the NEHTA blue print.
I have Version 2.0, FINAL, 30th Sept 2011, which differs from the draft one referenced in that the draft did not have an architecture section, the ehealth solutions section has been increased from 37 pages to 63 pages and the final has a much larger section on the PCEHR.
Here is a copy of the final blueprint
http://www.drbrd.com/docs/myhr/NEHTA_1026_2011_NEHTABlueprint_Blueprint_v2.0.pdf
A few quotes:
"1.3 The Case for Greater Investment in EHealth
In order meet this vision every interaction between individuals and their providers must achieve maximum impact on health outcomes and scarce financial and human resources must be deployed as effectively as possible
1.3.1 Benefits Estimate
Deloitte Consulting has estimated that an investment of $2.6 billion over 10 years in EHealth in Australia will yield a benefit of $5.7 billion in net present value over 10 years.
The main source of benefit is derived from time saved related to manual information handing and a reduction in avoidable hospital days related to adverse events. Additional benefits are also seen in the areas of improved chronic disease management, reduced unnecessary repeat testing and reduction in travel.
Booz Allen also looked at how EHealth value could be better optimised and constructed an investment model. The Booz Allen model looked at benefits from adopting a range of EHealth solutions. When the EHealth investment model is used to calculate the economic value of these benefit categories, Booz Allen found that the successful rollout and adoption of core EHealth capabilities in Australia are expected to be worth an estimated AU$7.6 billion annually by 2020. Approximately two-thirds (AU$5.1 billion) of the quantifiable EHealth benefits in Australia can be attributed to two benefit categories: reduced errors and enhanced adherence to best practices.
2.3.1 Outcomes
The introduction of new EHealth capabilities should provide a stepwise improvement in health system performance in one or more of the following areas8:
* Effectiveness – ensuring that care, intervention or action provided is relevant to the patient‘s needs, achieves the desired outcome and is based on standards."
Some observations:
The section on benefits clearly shows that the aim of eHealth is to cut costs, not improve a patent's health or make healthcare more effective or efficient.
The only place "patient‘s needs" is mentioned is in the sentence on effectiveness. There is no analysis of what they are or how doctors and the system meet these needs.
That's what I meant by a business model, which the blueprint isn't. The blueprint is a technology driven approach to computerising how healthcare is currently practiced. New versions of old solutions.
The bit that is missing is a section on clinical medicine. The eHealth solution section assumes that clinical medicine will be practiced exactly the same way as now. Hardly a transformation.
A search on the words "problem evidence based medicine" would have (and still does) throw up a lot of concerns about clinical medicine. Does the blueprint recognise this? No. Does it suggest new and innovative ways to help doctors cure their patients? No. Does it break the first rule of automation - do not computerise existing manual processes? Yes.
Nowhere in the blueprint is there any discussion of the costs to the healthcare system of implementing these technology solutions. Do a search for "problems electronic health records".
It's why doctors are complaining about the amount of time they are spending feeding data into computers to the detriment of their patient.
It's like going out looking at cars and deciding that a Tesla has all the benefits you could wish for, but not realising that you can't afford it. Dumb, if not outright stupid.
This NEHTA document confirms that ADHA is lost in the wilderness.
Bernard you need to knock on the Door of AHMAC or COAG for that sort of thing. NEHTA was set up to define frameworks and specifications, working with stakeholders and where required take these specification through various standards processes. Their work is technical because that was their role, until PCEHR came along the was well defined development and maintenance life cycles, tooling reflected the state of available tools back then. Implementation of NEHTA work also reflected technologies back then.
@anon April 20, 2019 1:56 PM
That sort of agrees with section 1.5 in both the draft and final versions of the blueprint:
"1.5 The Role and Strategic Focus of NEHTA
The National EHealth Transition Authority (NEHTA) will deliver key components of the National EHealth Strategy, endorsed by Australian Health Ministers in late 2008.
NEHTA‘s Strategic Plan articulates how NEHTA will support the National EHealth Strategy within its current mandate and sets a clear vision for EHealth in Australia:
NEHTA‘s work program will provide national infrastructure and accelerated adoption supporting this strategic direction and a future national approach to ensuring electronic health records are available for all Australians."
I say, "sort of" because the intent was always to ensure "electronic health records are available for all Australians.", which is an application not infrastructure.
The change between the draft (13 August 2010) and the final (30th Sept 2011) versions reflects the decisions made after the August 2010 election.
Whatever the reasons, a lot of the intended infrastructure has never been delivered, the PCEHR has been delivered in a far different form from intended and no effort has been made to advance clinical medicine, just reduce costs, and even that has not happened.
The predicted benefits from Deloitte/Booz and Company are notable for their failure to eventuate. That failure could be because NEHTA/ADHA/Health have done a bad job or because they were bad predictions or both.
My money is on both. Nobody has delivered such massive savings by implementing better health record management systems and the cost/benefit/risk of myhr just doesn't add up.
And I'd like to make one thing clear. A lot of technology initiatives in health care are very worth while and can be implemented to deliver significant benefits at minimum risk and acceptable costs. Most of these are outside what is considered Digital Health and do not include those that impose a significant extra cost on health service providers. I include controlled sharing of existing health record systems, for which interoperability needs to be improved but that and FHIR on their own are not sufficient.
The biggest danger of Digital Health is the focus on gathering more and more data (part of which is impacting doctors) and the problem of just too much unmanaged, irrelevant data (which is also impacting doctors)
The final blueprint says:
"Successful implementation of the capabilities described in this document will mean that healthcare will be significantly enhanced as consumers and providers will be able to access the right information, at the right time and place."
I'd add this:
"Healthcare will be significantly enhanced when the right technology is implemented in the right way, for the right cost and the right risk."
I see no evidence that the proponents of Digital Health fully understand the ramifications of that statement, especially the part about cost.
That's because most of the cost of technology comes from its implementation, not its acquisition. To fully understand that, it is necessary to understand clinical medicine and patient needs. The blueprint does neither, and all the signs are that ADHA doesn't either.
Ethics is knowing the difference between what you have a right to do and what is right to do.”
Like so much in today’s healthcare change advocacy it sound simple when you say it fast, say it in Greek or Latin ADHA will buy you drinks.
Post a Comment