Someone Has Bothered To Ask The Hard Questions On The COVIDSafe App. Interesting Lessons.

This article appeared last week.

Why, exactly, did the COVIDSafe app flop so badly?

A NSW Health-led study found a few reasons

29th April 2022

By Antony Scholefield

COVIDSafe app … It feels like we’ve not heard that name for some time now.

It was only two years ago that the Australian Government launched its automated contact tracing smartphone app, to a lot of fanfare and promises about re-opening pubs.

It was meant to use Bluetooth to identify when two people with the app spent more than 15 minutes within 1.5m of each other.

Consensus quickly arrived that it was a dud.

But exactly what went wrong has been hard to pin down.

Did not enough people download the app? Was it a technical failure with iPhones not being able to communicate with Android? Or was Australia’s manual contact tracing good enough on its own?

A study by NSW Health and the NHMRC suggests all of the above. 

It looked at the contact tracing records for 619 patients and found that only 22% had the app, compared with 44% of the Australian adult population.

It was possible that the same people most likely to download the app were the least likely to end up at a COVID-19 transmission site, they said.

Lots more here:

https://www.ausdoc.com.au/practice/why-exactly-did-covidsafe-app-flop-so-badly

I thought this paper was worth following up. Here is the Abstract.

Effectiveness evaluation of digital contact tracing for COVID-19 in New South Wales, Australia

·         Florian Vogt, PhD

·         Bridget Haire, PhD

·         Linda Selvey, PhD

·         Anthea L Katelaris, MD

·         John Kaldor, PhD

Open Access Published: February 04, 2022DOI:https://doi.org/10.1016/S2468-2667(22)00010-X

Summary

Background

Digital proximity tracing apps were rolled out early in the COVID-19 pandemic in many countries to complement conventional contact tracing. Empirical evidence about their benefits for pandemic response remains scarce. We evaluated the effectiveness and usefulness of COVIDSafe, Australia's national smartphone-based proximity tracing app for COVID-19.

Methods

In this prospective study, done in New South Wales (NSW), Australia, we included all individuals in the state who were older than 12 years with confirmed, locally acquired SARS-CoV-2 infection between May 4 and Nov 4, 2020. We used data from the NSW Notifiable Conditions Information Management System, the national COVIDSafe database, and information from case interviews, including information on app usage, the number of app-suggested contacts, and the number of app-suggested contacts determined by public health staff to be actual close contacts. We calculated the positive predictive value and sensitivity of COVIDSafe, its additional contact yield, and the number of averted public exposure events. Semi-structured interviews with public health staff were done to assess the app's perceived usefulness.

Findings

There were 619 confirmed COVID-19 cases with more than 25 300 close contacts identified by conventional contact tracing during the study period. COVIDSafe was used by 137 (22%) cases and detected 205 contacts, 79 (39%) of whom met the close contact definition. Its positive predictive value was therefore 39%. 35 (15%) of the 236 close contacts who could have been expected to have been using the app during the study period were identified by the app, making its estimated sensitivity 15%. 79 (0·3%) of the estimated 25 300 contacts in NSW were app-suggested and met the close contact definition. The app detected 17 (<0·1%) additional close contacts who were not identified by conventional contact tracing. COVIDSafe generated a substantial additional perceived workload for public health staff and was not considered useful.

Interpretation

The low uptake of the app among cases probably led to a reduced sensitivity estimate in our study, given that only contacts who were using the app could be detected. COVIDSafe was not sufficiently effective to make a meaningful contribution to the COVID-19 response in Australia's most populous state over a 6 month period. We provide an empirical evaluation of this digital contact tracing app that questions the potential benefits of digital contact tracing apps to the public health response to COVID-19. Effectiveness evaluations should be integrated into future implementations of proximity contact tracing systems to justify their investment.

Funding

New South Wales Ministry of Health (Australia); National Health and Medical Research Council (Australia)

Here is the link:

 https://www.thelancet.com/journals/lanpub/article/PIIS2468-2667(22)00010-X/fulltext

The full article is available below this abstract.

Reading all this it is clear that the app simply did not make enough difference to be worthwhile!

There are real learnings from the project on how to do this better and make a next app a success.

The people who implemented the app gave the project a good try but fell short on not managing the needed adoption.

Well worth a read.

David.