Late last week I was pointed to the new evaluation report of the SA Care Planning Project. The document was entitled Final Evaluation Report HealthConnect SA Trial of Care Planning and Communication System. 30 November 2007. Authors were Dr Svetla Gadzhanova, Assoc Professor Elizabeth Kalucy and Professor Richard Reed of the Flinders University Department of General Practice.
The document appears to have been finalised late last year and published early in February, 2008. It is not clear why a report dated 30 November, 2007 took so long to be released.
The Executive Summary is freely available on the web and it is possible to e-mail a request for the full 56 page report.
All the links are found from this page:
http://www.healthconnectsa.org.au/Default.aspx?tabid=55
The statement positioning those involved makes interesting reading!
“The HealthConnect SA program is funded by the Australian Government and is being delivered through the South Australian Department of Health. HealthConnect is coordinated nationally by the Australian Department of Health and Ageing (DoHA) and is supported by the National e-Health Transition Authority (NEHTA), which is responsible for developing national health information management systems and information and communication technology standards and specifications.”
I wonder did NEHTA know it was “responsible for developing national health information management systems”!
The Executive Summary makes for fascinating reading.
Executive Summary
In November 2007 HealthConnect SA completed a twelve month trial of an online care planning and communication system (CPCS) for the management of chronic conditions in primary health care. One rural and two urban Divisions of General Practice in South Australia took part in the trial. The Divisions recruited, trained and supported 27 general practices, 73 general practitioners (GPs), 224 allied health professionals (AHPs), 153 patients, and other health providers in their catchment areas, in the use of the Ozdocsonline system.
The trial was evaluated prospectively in consultation with HealthConnect SA. Monthly Progress Reports and an Interim Evaluation Report were produced to maximise the value of the findings for the purpose of planning a state-wide care planning and communication system. The evaluation used data from many sources to capture the experiences and perspectives of all those involved, including patients and AHPs.
Evaluation aim 1: to test if online communication systems are an effective and acceptable method of communication within the primary health care team.
The trial demonstrated that in principle an online communication system is effective and acceptable. Overall, GPs appreciated the benefits of prompt and more efficient communication with AHPs, from which some patients benefited substantially. AHPs found the system enhanced the team approach and contributed to patients receiving the right care at the right time from the right providers. The small number of patients interviewed also found the system acceptable, and would recommend it to others. Few pharmacists and no specialists used the system.
However, GPs did not consider the benefits were sufficient to overcome specific issues of the system being trialled. The processes of completing Team Care Arrangements (TCAs) and General Practice Management Plans (GPMPs) were not quicker or easier, especially when many of the AHPs the GPs normally worked with were not registered on the system. For many GPs software limitations reduced system efficiency and increased the time taken to use it, which created substantial barriers to uptake. The software could not easily be modified to meet GPs needs, especially integration with current clinical software.
Evaluation aim 2: to test if the CPCS increases the rate in which GPs and other health care providers participate in care planning for patients with chronic conditions.
Between January and September 2007 participating GPs developed 183 new care plans using the system. Use of the system by GPs and AHPs peaked in May 2007. Some GPs used the system for most of their care plans, others reverted to previous methods, and not all users registered on the system actually used it. As the evaluators did not have access to Medicare data specifically for the GPs participating in the trial, it was not possible to determine if their rate of care planning increased; however, Medicare data for all GPs in the three Divisions showed fewer GPMPs and TCAs in the second quarter of 2007 than in the corresponding quarter of 2006.
Despite the initial assumption that GPs were already actively participating in care planning, Division liaison staff found that some GPs were not familiar with the GPMP and TCA Medicare item numbers, which they perceived as time consuming and of unproven benefit for patients. As workforce shortages increased demands on GPs, they did not see that completing GPMPs and TCAs was a high clinical priority.
Evaluation aim 3: to determine the change management techniques needed to successfully implement an online CPCS in primary health care.
Conducting the trial in three different Divisions provided an opportunity to observe some of the factors which influenced uptake, reinforcing the idea that what works for one Division may not work for another. The factors include:
• At provider level, a well tested system consistent with clinical priorities, with evidence of unambiguous benefits in terms of patient care, health providers’ work processes and/or remuneration.
• At Division level, active support for the system by senior management, clinical and IT champions within the target group
• A receptive climate for change both at practice and Division level. The high uptake of the system in the rural Division illustrated what is possible when the system is consistent with the relationships and strategic plans of the Division, in contrast to lower uptake in the two urban Divisions where the Division and practices faced competing priorities due to other initiatives and pressures.
• Flexible training and support tailored to the varying needs of users.
• Responsive support to resolve technical and other problems.
• Change management processes which address the beliefs and obstacles identified by the
target group, and overcome their resistance to unfamiliar and unproven systems.
• Appropriate incentives for trial participation at Division and at provider level. Two Divisions achieved higher participation rate by reimbursing GPs for training time or completing evaluation forms.
Findings from this evaluation suggest that successful uptake of an online CPCS at practice levelis more likely to happen if:
• GPs and practices are familiar with, and are creating, GPMPs and TCAs, and believe they are advantageous and worth the effort in both time and compensation.
• The system is compatible with existing chronic disease management processes and clinical software, facilitates secure sharing of care planning information, and is beneficial in terms of costs, time and health outcomes for patients with chronic conditions.
• There is a critical mass of AHPs and specialists with well-organized profiles on the system, with whom the GPs can share care plans.
• There is a secure broadband connection at each provider site;
• There is capacity for an extended role for practice nurses in the care planning process.
For implementation of the state-wide care planning system, the evaluation recommends the following strategies, which were outlined in the Interim Evaluation Report:
Organisational structure
• Ensure the system is consistent with the Divisions current goals and activities;
• Obtain strong ongoing support from senior clinical, IT and management staff;
• Involve Division staff with strong relationships with general practices in the promotion of the system;
• Allow time for Division staff to understand existing processes to ensure effective promotion of the new system;
• Provide Division staff with sufficient resources, time for preparation, and adequate training before the implementation of the new system.
Implementation activities
• In chronic disease management, promote e-Health widely to GPs, AHPs, specialists,
pharmacists and patients;
• If needed, educate health providers in GPMPs and TCAs.
Resources
• Provide, in electronic and printed format, information brochures and FAQs sheets suitable for all participants including patients, as well as relevant case studies demonstrating the expected outcomes;
• Ensure there is a secure broadband connection at each provider site;
• Train the GP team and AHPs at the same time to avoid time lags in getting the team onboard;
• Tailor training according to participants’ specific characteristics (prior knowledge, IT experience);
• Anticipate that additional training sessions might be required, and that time is needed to absorb training before usage.
Support structure
• Provide ongoing and timely support to users of the system, using diverse forms such as communication materials and ongoing technical support from system vendor and other organisations involved in implementation.
Participants
• Focus on specific training for practice nurses as contributors in the care planning process.
System functionality
• Implement a secure, reliable, easy to use IT care planning system which integrates well with existing chronic disease management processes and software;
• Choose a system that facilitates sharing of care plans within and between practices;
• Provide evidence of benefits in terms of costs, time and health outcomes for patients with chronic conditions.
• For information consistency, ensure a smooth transition between the trial system and the state-wide system.
---- End Executive Summary.
It is only when one looks closely at this evaluation report does one appreciate what total rubbish it, and the project were.
The statistics tell the story!
First only 183 care plans were developed in the nine months of the trial’s operation – less than one a day by my reckoning – the system was hardly likely to be overloaded!
Second the evaluation hardly provides comprehensive information on the trial as so few of those involved responded to evaluation survey requests. From the full report we see the following description of the data on which the evaluation was based. Remember 73 GPs, 200+ AHPs and 150+ patients were involved.
GP baseline survey: 8 responses (5 ACEDGP, 3 YPDGP).
GP endpoint survey: 9 responses (3 ACEDGP, 3 YPDGP, 3 SDGP)
Case studies: 3 interviews with a GP from each division
Patient interviews: 5 interviews out of 9 patients invited (3 patients from ACEDGP and 2 patients from SDGP)
AHP survey: 5 responses out of 9 AHPs invited
Third the system was hardly used!
“Use of the system peaked in May 2007 (Figure 1). Between January and September 2007 participating GPs developed 183 new care plans using the system, and made a substantial number of new planner entries and progress notes on these care plans. AHPs contributed 189 progress notes. Patients and pharmacists made negligible use of the system, and specialists none at all.”
So what we have here is a one year trial of a system essentially no one used, which was not integrated into the usual clinical workflows, and which on all the evidence available was a total and complete failure.
Worse it seems there was an interim Evaluation Report that said some change was needed that was not actioned.
Horrifyingly it seems there is still a tender on foot for procurement of what is now called the e-Health Care Planning System. The name change apparently reflects the need to identify this care planning system as an e-Health initiative! The tender closed June 12, 2007.
Much more on this farce of a tender is found at:
http://aushealthit.blogspot.com/2007/05/sa-healthconnect-opens-appalling-e.html
Amusingly the evaluation and contract was meant to be done by the end of August 2007. Seems it was done in secret or it has been canned.
One can only hope this sad chapter in Australian e-Health has been put down quietly and that those involved have been re-deployed to do something useful.
I suggest you e-mail and get the full report for yourself..it is a true collectors piece. Pity no one thought to evaluate for improved clinical outcomes and then clearly state that the system made no appreciable differences to these was thus essentially a total failure – as it truly was! This is what you get when you ask the wrong evaluation questions.
I wonder how much money was wasted on this?
David.
