AI Scribes in Australian Healthcare: The Alarming Rise of ‘Hallucinated’ Medical Records and the Risk to Patient Safety
- Australians’ medical records are being compromised by AI scribes that are ‘hallucinating’ false symptoms and conditions, putting patients’ health and livelihoods at risk.
- A recent report by Digital Rights Watch (DRW) has revealed widespread errors in AI-transcribed medical records, including a case where a scribe recorded the wrong breast in a diagnosis of breast cancer.
- Experts warn that patients may be unaware of mistakes on their medical record, which could have serious consequences for their health, career, or insurance claims.
- The Therapeutic Goods Administration (TGA) has acknowledged that some AI scribes are potentially in breach of the law by operating without regulation or oversight.
When Rebecca Green consented to having her urologist appointment transcribed by artificial intelligence (AI), she had no idea that it would lead to a serious but false claim being made about her medical history.
The AI scribe incorrectly stated that she had been taking psychedelic mushrooms, which could have had huge consequences for her workers’ compensation case.
The incident has left Ms Green shaken and reevaluating her trust in AI transcription. “It felt very impersonal to me.
I wasn’t getting the level of care and familiarity from the doctor that I would be hoping for in such a vulnerable situation,” she said. The urologist later apologized and corrected the error, but the experience has raised serious concerns about the use of AI in healthcare.
The use of AI scribes in Australian healthcare is becoming increasingly prevalent, with the Royal Australian College of GPs (RACGP) estimating that 40% of GPs are regularly using them.
While AI can help reduce the administrative burden on doctors and allow them to focus more on their patients, it also poses significant risks. Research has shown that AI scribes can ‘hallucinate’ false symptoms or conditions, which can lead to incorrect clinical decision-making.
Analysis: What This Means for Australia
The use of AI in healthcare raises serious concerns about patient safety and the accuracy of medical records. If left unregulated, AI scribes could compromise the integrity of Australia’s healthcare system.
The TGA’s failure to regulate AI scribes that claim to be solely for transcription and summarizing is a significant oversight, and the regulator’s acknowledgment that some scribes are potentially in breach of the law is alarming.
Security analysts warn that the lack of regulation and oversight of AI scribes poses a significant risk to patient safety.
“If AI scribes are not properly regulated, it could lead to a situation where patients’ medical records are compromised, and their health and livelihoods are put at risk,” said one expert.
Law enforcement insiders also warn that the use of AI scribes could have serious consequences for the justice system.
“If AI scribes are not accurate, it could lead to incorrect diagnoses and treatments, which could have serious consequences for patients and their families,” said a law enforcement source.
Experts recommend that patients ask questions about the use of AI scribes in their medical appointments and ensure that they understand how their data will be used and protected.
They also recommend that doctors and healthcare providers take steps to ensure that AI scribes are used safely and accurately.
Dr Sean Stevens, a Perth GP who has been using AI scribes for three years, believes that the technology is becoming more accurate, but acknowledges that doctors still need to check every transcript.
“Three years ago, I would have to correct multiple things on each transcription. These days it’s like one correction every third or fourth transcription,” he said.
The DRW is calling on the newly established AI Safety Institute to test AI scribes and publish a list of approved tools.
This would be similar to New Zealand’s approach, where the national AI advisory group reviewed tools and last year gave approval for clinicians to use two AI tools: Heidi Health and iMed.
The error on her medical record has left Rebecca Green reconsidering if she would give consent to AI transcription again. “I would only give consent for AI if I had an established relationship with the doctor,” she said.
As the use of AI in healthcare continues to grow, it is essential that patients, doctors, and regulators take steps to ensure that AI scribes are used safely and accurately to protect patient safety and the integrity of Australia’s healthcare system.
