RicardoRei

Hi, HN, I'm Ricardo, Head of AI Research at Sword Health.

We've been working on AI mental health support for a while now, and one of the biggest challenges we kept running into is how poorly general-purpose safety classifiers work in this context. They're built to flag harmful content broadly, so when someone in a therapy conversation says "I feel like I'm drowning," the system can't tell if that's a metaphor or a genuine crisis signal. That leads to two problems: either the system over-escalates on benign therapeutic content and breaks rapport, or it misses subtle signals that actually require intervention.

MindGuard is our "first" attempt to solve this. We developed it in close collaboration with licensed clinical psychologists, who helped us build a risk taxonomy that reflects how clinicians actually reason about urgency - distinguishing between safe therapeutic content, self-harm risk, and harm to others. We trained lightweight classifiers (4B and 8B) that achieve 2–26× fewer false positives than general-purpose models like Llama Guard, while still maintaining high recall on the signals that matter.

We're also open-sourcing the models, the evaluation dataset (annotated by clinical experts), and the risk taxonomy.

Happy to answer any questions.

Again, these things don't depend on each other.

LLMs have room for improvement (we show that their scores are medium-low on several dimensions).

Maybe the average human also has lots of room for improvement. One thing does not necessarily depend on the other.

the same way we can say that LLMs still have room for improvement on a specific task (lets say mathematics) but the average human is also bad at mathematics...

We don't do any claims about human therapists. Just that LLMs have room for improvement on several dimensions if we want them to be good at therapy. Showing this is the first step to improve them

Cite the ArXiv paper for now.

yep yep. Its something we have to study and its likely we can improve the LLM as a Judge further.

Same thing for the patient LLM. We can probably fine-tune an LLM to do a better job at simulating patients.

Those two components of our framework have space for improvement

Exactly. We don't do claims about humans. But there is room for improvement on current LLMs... For researchers to be able to improve LLMs we first need to know how to evaluate them. We can only improve what we can measure so we studied how to measure them :)

The results are not meaningless but they are not comparing humans against LLMs. The goal is to have something that can be used to test LLMs on a realistic mental health support.

The main points of our methodology are: 1) prove that is possible to simulate patients with an LLM. Which we did. 2) prove that an LLM as a Judge can effectively score conversations according to several dimensions that are similar to how clinicians are also evaluated. Which we also did and we show that the average correlation with human evaluators is medium-high.

Given 1) and 2) we can then benchmark LLMs and as you see, there is plenty of room for improvement. We did not claim anything regarding human performance... its likely that human performance also needs to improve :) thats another study

This is a good point. We have not tested the clinicians but I believe they would not score each other perfectly as we observed some disagreement also between the scores which also reflects different opinions between clinicians

The prompts are kept the same for all models. Otherwise comparison would not be super fair. In any case you can check all the prompts in our github repo.

Hi HN - I’m the Head of AI Research at Sword Health and one of the authors of this benchmark (posting from my personal account).

We built MindEval because existing benchmarks don’t capture real therapy dynamics or common clinical failure modes. The framework simulates multi-turn patient–clinician interactions and scores the full conversation using evaluation criteria designed with licensed clinical psychologists.

We validated both patient realism and the automated judge against human clinicians, then benchmarked 12 frontier models (including GPT-5, Claude 4.5, and Gemini 2.5). Across all models, average clinical performance stayed below 4 on a 1–6 scale. Performance degraded further in severe symptom scenarios and in longer conversations (40 turns vs 20). We also found that larger or reasoning-heavy models did not reliably outperform smaller ones in therapeutic quality.

We open-sourced all prompts, code, scoring logic, and human validation data because we believe clinical AI evaluation shouldn’t be proprietary.

Happy to answer technical questions on methodology, validation, known limitations, or the failure modes we observed.