Mental healthcare AI needs evidence beyond a reassuring conversation

Trials of therapeutic chatbots are producing encouraging signals, alongside evidence of important limitations. The next challenge is to establish which services help, for whom and with what continuing support.

A chatbot can be available when a clinic is closed and respond without apparent impatience. Those qualities help explain its appeal as a source of emotional support. They do not establish that the conversation provides effective treatment, recognises deterioration or connects somebody with appropriate human care when their needs change.

Mental healthcare AI is therefore facing a more demanding test than whether people enjoy using it. Researchers are examining symptom change, the quality of therapeutic interactions and risks that develop across a conversation. For European services considering adoption, the most useful evidence connects those questions with the practical arrangements around the software.

An early trial supports further clinical investigation

The Therabot trial, published in NEJM AI in 2025, randomised 210 adults to a specialist generative chatbot or a waiting-list control. Participants had clinically significant symptoms of depression or anxiety, or were at high risk for feeding and eating disorders. The intervention involved four weeks of prompted use, followed by four weeks of continued access without prompts.

The researchers reported greater reductions in symptom measures than in the control group at four and eight weeks. Their summary also records 15 staff interventions concerning safety and 13 corrections to inappropriate responses. These details matter together: the observed outcomes arose within a study that included human monitoring and intervention.

The trial did not compare Therabot directly with a course of professional psychotherapy. A waiting-list comparison can establish an initial signal while leaving questions about specific treatment effects, alternative support and longer-term durability unresolved. It also cannot establish that an unrelated general-purpose chatbot produces the same benefit.

For future commissioning, the complete intervention should be the unit of comparison. That includes software, onboarding, staff availability, escalation arrangements and follow-up. Removing one of those elements could change both the cost and the clinical proposition.

Therapeutic competence and recovery are different endpoints

A March 2026 Nature Medicine study examined an architecture that adds specialised therapeutic reasoning to language models. The work was funded by Limbic, with authors declaring employment or consultancy relationships and shareholdings. In a randomised, double-blind evaluation, 227 participants generated conversation transcripts assessed by 22 expert clinicians. The augmented models received stronger ratings on specified cognitive behavioural therapy competencies than the comparison agents, including human clinicians.

The paper also analysed deployment data involving 8,920 users and reported an association between greater activation of the architecture and symptom improvement. These are distinct forms of evidence. Ratings of transcripts assess the interactions presented to reviewers; observational associations do not establish that the architecture caused the subsequent clinical change.

The commercially attractive headline would be that AI outperformed therapists. The more useful reading asks what was actually compared, how performance was scored and whether the people receiving the intervention improved. A system could produce a technically appropriate response without being suitable for every person or every stage of care.

This distinction suggests a research sequence rather than a single verdict. Structured assessment can help refine an intervention, while comparative clinical studies establish what happens when it is offered through a defined service. Neither stage should be asked to answer all the questions on its own.

Risk can build across several exchanges

The SIM-VAIL study, published in August 2026, examined a different problem: how chatbot behaviour changes over a conversation. Researchers generated 810 conversations across nine target models and 30 simulated user profiles, evaluating responses through clinically grounded risk dimensions. Concerning behaviour varied with the simulated vulnerability and conversational intent and could accumulate across turns.

The study identified situations in which apparently supportive responses reinforced an unhelpful psychological pattern. Its contribution is an auditing framework and evidence from adversarial simulations. It does not measure the rate of harm among real users or show that every supportive response is dangerous.

For developers, the implication is that testing isolated answers may miss something important. An acceptable first reply could be followed by repeated reassurance, increasing dependence or a failure to revisit an earlier assumption. Evaluation should examine the trajectory of the interaction, including whether the system changes its response when new information arrives.

Service teams could use that approach to test how a tool handles a person whose needs move beyond its intended scope. The relevant outcome would include whether the boundary is recognised and whether the next step is understandable and usable. Displaying a general warning somewhere in an interface does not establish that this handover works.

Digital therapy is already a supported clinical service

NICE’s work on digitally enabled therapies offers a useful European reference point, although these products should not all be described as generative AI. Its assessment of therapies for adults with anxiety describes psychological interventions delivered digitally with regular practitioner or therapist support. Its depression assessment discusses initial clinical assessment, matching treatment to needs and preferences, and ongoing monitoring of safety and progress.

That service model makes the surrounding work visible. Somebody assesses suitability, reviews progress and responds when the current approach is insufficient. A conversational interface may change how a person engages with exercises or records their experience, but it does not automatically remove those responsibilities.

A useful comparison would examine what additional work an AI intervention creates as well as what it saves. Staff may need to review unusual exchanges, clarify misleading advice or contact a participant who stops responding. Those tasks belong in the resource estimate, rather than being treated as exceptional costs outside the product.

The service also needs to understand discontinuation. A person might stop using the tool because they feel better, because it is unhelpful or because their condition has worsened. Engagement data alone cannot distinguish those possibilities. Follow-up should be designed to learn about outcomes without assuming that more messages always mean better care.

Intended purpose determines the regulatory question

The MHRA’s guidance on digital mental health technologies asks manufacturers to consider intended purpose and functionality when determining whether a product is software as a medical device. Some technologies fall within that framework and others do not. A tool presented for general wellbeing and one intended to treat a condition cannot simply be assigned the same regulatory status because both use a chatbot interface.

For purchasers, a precise description of the product is therefore more useful than a broad claim that it is an AI mental health platform. What is it intended to do? Which users and conditions were evaluated? What version was tested? Which functions are available in the proposed deployment? These questions connect the evidence with the actual service being offered.

Regulatory status also answers a different question from comparative clinical effectiveness. It should not be presented as proof that a product works better than existing therapy or that an encouraging result transfers to a different patient group. Evidence for an adult study, for example, does not establish suitability for children.

Build a service that patients can understand and question

WHO’s guidance on large multimodal models calls for attention to the people affected by their development and deployment, alongside transparency, accountability and continuing assessment. It is international guidance rather than a product authorisation or a substitute for national requirements. For mental healthcare, participation can help identify expectations that a technical evaluation overlooks.

Patients may reasonably want to know whether a person reads their messages, when anybody will respond and what happens if they stop using the service. The answers need to match the operating arrangements. A system that feels continuously present should not leave users to infer that a clinical team is continuously monitoring it.

Research should also examine whose preferences the service accommodates. Some people may value private digital contact; others may find a conversational system intrusive, confusing or unsuitable. Meaningful choice requires understanding those experiences and the availability of other forms of support, rather than measuring satisfaction only among people who remain active users.

The next advance in mental healthcare AI should be an intervention whose benefits, limits and support arrangements can be explained together. Encouraging symptom changes deserve further investigation. So do uncomfortable findings about interaction and safety. Keeping both in view would help European services develop evidence for a dependable form of care, rather than mistake a reassuring exchange for a complete treatment.