In CAI Currents, The American Psychoanalyst
People across the cultural spectrum, from writers and artists to educators to psychoanalysts and psychotherapists, love to contrast the value of the uniquely human to what chatbots can’t do. But might there be places where the chatbot wins, like a physician’s often arduous task of keeping accurate, comprehensive notes. Freeing up a doctor’s time to make room for more quality time with patients seems a boon and a relief. These AI scribes seem to be all promise and no peril.
AI scribes are proliferating at an exponential rate, even becoming word of the day at the popular tech podcast Hardfork. No surprise. There’s value in their ability to synthesize huge amounts of material with near perfectly accuracy, and to do so in seconds. No surprise so many find them magical, even seductive. After all, physicians don’t want to think of themselves as highly trained and very expensive clerks.
But there’s trouble in this promised paradise. So argues Helen Ouyang, a physician and an associate professor of emergency medicine writing in The New York Times Magazine. Doctors do something more than record and summarize a clinical intervention, something an AI doesn’t do.
Physicians process the surface content while also applying intuition and experience to glean what might be going on. They study how symptoms may not make sense and how conclusions may need to be reevaluated. In writing a note, they have the opportunity to think about what may be missing. The AI’s note comes across as all-inclusive, confident and complete.
The presence of AI in the room can tend to sway the doctor to just go along for the ride, abandoning their own judgment. Cognitive off-loading is an ever-present AI-risk. Plus, there is an inevitable reduction in patent-doctor intimacy so that the doctor feels less connected to the patient. The machine has weighed in, so move on, next!
Plus, there’s an even larger problem: Leaning on AI may erode the doctor’s autonomous decision making that struggling through uncertainty helps. De-skilling, mis-skilling, and never-skilling are also ever-present AI-risks. While the promise of AI scribes is clear, the peril is less so, but no less real. Both deserve attention.