- Education & Catastrophe
- Posts
- AI More Empathetic And Accurate Than Human Doctors
Image credit: Towards Conversational Diagnostic AI by Google Research and Google DeepMind
Couple of weeks ago I wrote that jobs of the future will be about the heart. Perhaps no profession is more closely associated with the heart than medicine (ha-ha-ha). In a research paper published by Google researchers in January, a Large Language Model (LLM) based AI system optimised for diagnostic dialogue in the medical field outperformed doctors on 28 out of 32 evaluation axes related to conversation quality, diagnoses, and management plans.
To put it simply, a chatbot was able to diagnose more accurately and show greater empathy to patients than human doctors.
Is this a cause for concern, or should we be looking forward to more effective and affordable health care?
Let me just say upfront that I don’t think doctors will be entirely replaced by machines in the foreseeable future. Figure AI, an AI company building general purpose humanoids may have just raised $675m from Jeff Bezos, Microsoft, Nvidia, Intel etc, but replacing doctors and surgeons is not part of their masterplan. At least not what they have shared publicly.
We believe humanoids will revolutionize a variety of industries, from corporate labor roles (3+ billion humans), to assisting individuals in the home (2+ billion), to caring for the elderly (~1 billion), and to building new worlds on other planets. However, our first applications will be in industries such as manufacturing, shipping and logistics, warehousing, and retail, where labor shortages are the most severe.
In early development, the tasks humanoids complete will be structured and repetitive, but over time, and with advancements in robot learning and software, humanoids will expand in capability and be able to tackle more complex job functions. We will not place humanoids in military or defense applications, nor any roles that require inflicting harm on humans.
Our focus is on providing resources for jobs that humans don’t want to perform.
Going back to the research paper "Towards Conversational Diagnostic AI", the AI system Google researchers call AMIE (Articulate Medical Intelligence Explorer) demonstrated greater diagnostic accuracy and outperformed primary care physicians (PCPs) on various evaluation axes according to specialist physicians and patient actors.
Image credit: Towards Conversational Diagnostic AI by Google Research and Google DeepMind
Top-k is simply a sampling method that picks from a shortlist of top responses. DDx refers to differential diagnosis evaluated based on several criteria to assess the appropriateness, comprehensiveness, and accuracy compared to the answer key.
Image credit: Towards Conversational Diagnostic AI by Google Research and Google DeepMind
When you think about it, what is empathy? Empathy is about active listening, it’s about being incredibly patient. And there’s nothing that is more patient than a robot that has no sense of time and is stateless.
The researchers who wrote the paper did caveat that the study has important limitations, most notably a text-chat interface, which although enabling potentially large-scale interaction between patients and LLMs specialized for diagnostic dialogue, was unfamiliar to PCPs for remote consultation.
The study highlighted the potential of AI systems in improving healthcare outcomes through conversational diagnostic capabilities. The research emphasised the importance of optimising AI systems for clinical history-taking and diagnostic dialogue to bridge the gap in healthcare expertise and access. AMIE's development involved utilising real-world datasets, a simulated self-play learning environment, and a chain-of-reasoning strategy during inference to enhance its diagnostic conversation capabilities. In other words, AMIE could continuously learn and improve its conversational and diagnostic capabilities, progressively refine its responses based on the current conversation context, and provide informed and grounded replies during diagnostic dialogues.
Again, I want to emphasise that it is erroneous to suggest that human doctors are going to be replaced by AI based on the preliminary findings of one research paper. The study offers hope to the millions of people around the world who do not have easy access to primary care physicians. If AI systems can consistently deliver accurate diagnoses and manage patients’ concerns, healthcare will become more affordable and accessible, perhaps making universal healthcare a reality. Human doctors will continue to play a critical role in healthcare, from providing pediatric care to performing surgeries.
AI for healthcare has never been about replacing doctors, but doctors who use AI might end up replacing those physicians who don’t.
As much as we can look forward to the promise of AI in healthcare, Elon Musk’s lawsuit against OpenAI this week alleging that OpenAI and its chief executive Sam Altman have compromised the startup’s original mission of building artificial intelligence systems for the benefit of humanity is a reminder that AI containment needs to remain top of mind. Musk’s lawsuit alleges that GPT4 breached the threshold for artificial general intelligence (AGI), at which computers function at or above the level of human intelligence.
Mr Musk has long recognised that AGI poses a grave threat to humanity - perhaps the greatest existential threat we face today.
Wanna help your child develop active listening, patience, empathy and all these other human traits? Get them in nature!
With spring coming, Saturday Kids Unplugged Karuizawa 2024 is only three months away. Two dates to choose from
10 June - 14 June
17 June - 21 June
Extremely limited number of spots for 4 to 6-year-olds, so please sign up early.