Quote Of The Year

Timeless Quotes - Sadly The Late Paul Shetler - "Its not Your Health Record it's a Government Record Of Your Health Information"

or

H. L. Mencken - "For every complex problem there is an answer that is clear, simple, and wrong."

Sunday, February 25, 2018

Talk About An Appalling IT Stuff-Up - I Really Feel For The Victims!

This story broke in the middle of last week.

Trainee doctors forced to resit exam after 'technical fault'

By Allie Coyne on Feb 20, 2018 6:22AM

First computer-based test failed, results voided.

Trainee doctors who sat a basic exam yesterday have had their results voided after a "technical fault" locked some students out of the second half of the test.
It was the first time the Royal Australasian College of Physicians (RACP)'s divisional written exam had been delivered online, managed by computer-based testing solutions provider Pearson Vue.
The medical college apologised to students for yesterday's technical error and said it had decided that all students would resit the exam - this time via pen and paper - "in order to be fair".
"We understand that some trainees have booked leave or holidays following today’s computer based test, and we are working as quickly as possible to reschedule a new exam," the RACP said in a statement.

"It is likely to be more than 24 hours before we can notify candidates of a new exam date."
The college said it had "explored all options" with Pearson Vue after the "unknown technical fault" locked a "significant" number of students out of the computer-based test and unable to complete the second part of the exam, but ultimately decided to call it off.
"The RACP is very disappointed that there has been a problem with today’s [exam], and we apologise to all trainee candidates for the distress caused," it said.
More here:
https://www.itnews.com.au/news/trainee-doctors-forced-to-resit-exam-after-technical-fault-485491
Harrowing reporting on the impact of the issue is found here:

Burnt-out doctors deeply distressed by botched high-stakes exam

Kate Aubusson, Jenny Noyes
Published: February 21 2018 - 9:32AM
Up to 1200 doctors are deeply distressed by the Royal Australian College of Physicians “appalling” handling of an IT meltdown that wiped out a crucial, high-stakes examination on Monday.
Senior physicians have raised serious concerns for the mental health and wellbeing of the registrars now forced to resit the test after months of gruelling study regimen, hospital workloads and personal sacrifice.
A technical fault abruptly shut down the basic training exam. The test cost each candidate $1800 to sit and is a requirement for doctors aspiring to specialise as physicians or paediatricians.
The college is facing mounting criticism from its membership, with calls for its president and others responsible for the incident to resign.
Several doctors who sat the botched exam spoke to Fairfax Media on condition of anonymity over concerns that speaking publicly could impact on their future careers.
As crowds of confused registrars poured out of exam centres across the country, many were crying inconsolably, visibly distressed and angry.
“It was just awful … complete chaos” one registrar said.
“There were a number of candidates sitting on kerbs crying ... no one [was] advising of what happens next.”
Several registrars described chaotic scenes long before the technical meltdown.
Lots more here:

Also included in a lot of subsequent coverage was commentary from some one who had sat the exam the year before:

Computer glitch in hardest exam of your life is unforgivable

Sanj Mudaliar
Published: February 21 2018 - 12:06PM
It’s hard to explain to someone outside the medical field what this examination means to those who sit it. The anguish that it can cause and the amount of time that candidates spend preparing.
After roughly 10 years of training and a lifetime of exams, the Royal Australian College of Physicians examination is the last written one you have to sit - and pass - on the road to becoming a fully fledged physician or paediatrician. It assesses your knowledge across all the medical specialties. It is the final hurdle, the finish line, what many see - correctly or incorrectly - as the point where they can stop striving for a life and start actually living one. Holidays, weddings, even births, have all been put on hold, planned around this last brutal test.
Which is why the 1200 or so junior doctors who had to abandon their attempts at the exam on Monday because of a computer glitch are so distressed.
The written exam consists of two papers -  a basic sciences and a clinical paper - and is held only once a year, on the same day nationwide. It takes 6.5 hours to complete, with a one-hour break, and covers two years' worth of study.  The written exam is like the hardest, broadest university-level final exam you have ever sat.  This year, for the first time, it was held in small venues across the country, with eight to 15 candidates per room, instead of in one central venue in each capital city.
Lots of annoyed rage found here:

Now in another life, in another time, I did a similar exam - (the Part 1 for the Anaesthetics Fellowship - which I passed on my first attempt much to my relieved amazement.) and I can say had my 4 hour exam suffered a similar fate there might have been enraged homicide ensue.

That these systems were not tested to within an inch of their life is simply unbelievable and unforgivable. My view is that the CEO of the College should resign - as simple as that - given the centrality of conducting these exams to the College’s role.

It is possible to ensure such important systems work and this should have been 100% ensured. It is really hard to understand how such a mission-critical system was not fully bullet-proof?

I feel for all the victims can I say!

David.

5 comments:

Anonymous said...

They will of course reimburse all the fee to sit the exam and provide free of charge the rescheduled exam.

Anonymous said...

Where lies the root cause? First the RACP failed to manage risk, to select a competent technology supplier. Second, IT vendor, in this instance demonstrated what we are repeatedly witnessing another failed critical IT system. This is on par with the QLD Health payroll system albeit fewer people impacted.

Anonymous said...

... and will they reimburse the cost of my airfares which the travel insurance doesn't cover and the airfare conditions do not permit changing dates.

Bernard Robertson-Dunn said...

Who knew IT could be so complicated?

Company behind botched medical exam has track record of failure

"The multinational education-testing company behind a bungled exam that left 1200 trainee doctors distraught and in limbo on Monday has a well-documented history of stuff-ups.

Fairfax Media can also reveal registrars warned the Royal Australian College of Physicians their exam could be compromised months before they sat the botched text.
Trainee doctors were left distraught by the exam glitch.

Pearson - which was involved in marking the paper-based NAPLAN tests - has been heavily criticised over its poor track record in the US.

A Washington Post article published in 2016 cited a total of 65 examples of stuff-ups, from test-score delivery delays to programming and printing errors, misgradings and security issues.
Related Articles

Computer glitch in hardest exam of your life is unforgivable

The company was also taken to task in 2015 by John Oliver in a segment on his satirical news program Last Week Tonight that tore down the state of standardised testing in the US.

Monday's RACP Basic Training Test, the culmination of 12 months to two years of intensive study for registrars preparing to specialise in either paediatrics or adult medicine, was cut short following an "unknown technical fault", forcing them to sit through the high-stress ordeal again next month.

It was the first time the exam was computer-based. Registrars and senior physicians are outraged the RACP went ahead with the change despite Pearson's poor record and a list of concerns raised months ago about the software and proposed examination process.

... lots more

http://www.smh.com.au/national/company-behind-botched-medical-exam-has-track-record-of-failure-20180221-p4z15a.html

Anonymous said...

unknown technical fault - that is a very concerning statement. Systems and software should be surveying themselves and generating something as simple as a file through that error logs and codes should quickly determine the issue. Look forward to reading some honesty.