©
Shutterstock
A team led by the University of Oxford and the Institute of Psychiatry, Psychology and Neuroscience (IoPPN) at King’s College London say their research shows the potential for AI Large Language Models (LLMs) to help identify people at high risk of psychosis, as well as making assessments faster and more accessible for patients. The findings have been published in the journal npj Digital Medicine.
It comes as Wellcome fund a new £2.4 million project to develop a voice-based system called ARIADNE, powered by generative artificial intelligence, designed to identify people at risk of developing psychosis. It will be able to hold a natural conversation, ask patients the right questions, and produce clear, tailored reports for patients, clinicians, and researchers.
Psychosis accounts for around 10% of the global health burden for young people, with onset usually around the ages of 20-35. Early detection of psychosis results in patients receiving treatment earlier, spending fewer days in hospital and having better outcomes overall.
However, people at a high risk of psychosis often face long waiting lists for assessment, with almost 9 out of 10 people not identified in time to prevent onset. Meanwhile many of those referred for assessment do not actually meet the criteria for being high risk, meaning a lot of time is spent on unnecessary interviews.
The study team evaluated 11 LLMs on 678 clinical interview transcripts from 373 participants to assess their ability to identify individuals at clinical high risk for psychosis, estimate symptom severity and frequency, and generate clinical summaries.
They found that overall the LLMs were able to identify the key data and symptoms and classify patients correctly, comparing well with assessments done by human researchers. Even smaller models worked well and were cost effective.
Senior author Dr Dominic Oliver, from the Department of Psychiatry at the University of Oxford, said:
Early detection is critical for preventing psychosis, but current approaches rely heavily on specialist interpretation of clinical interviews. But these interviews are long, require highly trained clinicians, and are only done in specialist services, with patients often finding the subsequent reports confusing and difficult to understand.
Patients tell us that getting help earlier can make a significant difference in their lives. Our findings show that AI can identify those at risk and presents us with an opportunity to reach more people more quickly, making assessments faster, more accessible and more empowering for individuals and their families."
He added: “People may worry that this means they won’t see a doctor, but that’s not what we’re suggesting. In the framework we have designed with patients and young people, clinicians will still see everyone and be the ones making decisions. AI will simply help collect information in advance, so these highly trained doctors and psychiatrists can focus their time on supporting patients and delivering care. Through using AI, we can navigate a bottleneck in these services, allowing patients to access the specialist support from clinicians they need more quickly.
“We are now developing a bespoke system that is accurate, safe, ethical, cost effective and acceptable to patients. By working closely with young people, patients and clinicians throughout its development, we hope to create a tool that complements clinical expertise, expands access to specialist care and improves the experience of assessment for patients.”
The LLMs also worked well across different groups, with minimal demographic disparities across ages, genders, ethnicities and first languages. Most errors involved the LLMs over-pathologising everyday experiences, which the researchers say highlight the need for careful human oversight too.
Lead author Dr Taiyu Zhu, Lecturer in Large Language Models for Healthcare in the Department of Biostatistics and Health Informatics at IoPPN, King’s College London, said: “AI tools, particularly large language models, are developing rapidly and are becoming increasingly capable of understanding and organising complex clinical information. This creates exciting opportunities to support healthcare professionals with tasks that are time-consuming and difficult to scale, while helping to make care more efficient, accessible and personalised.
“The next challenge is to translate these advances into tools that are safe, reliable and genuinely useful in clinical practice. Achieving this will require close collaboration with patients, healthcare professionals and researchers, alongside careful evaluation of how these tools are introduced and used.”
The new ARIADNE project involves several phases including developing the technology and infrastructure for the AI screening tool, working with patients and young people to address safety and ethical concerns, assessing the accuracy and feasibility of the tool, and mapping how it can be scaled up and delivered in clinical practice.
Naomi Herz, Head of Digital Health Technologies at Wellcome, said:
GenAI could potentially help revolutionise mental health treatment and diagnosis which is why we’re funding a range of exciting fundamental research on AI, including the ARIADNE project. Through investing in science, we can better understand the opportunities of AI for mental health, as well as potential risks. This will help us identify potential tools for earlier intervention in anxiety, depression and psychosis."