OpenAI has published a mental health benchmark, and its best model scores 57 out of 100 on it
Every score in the launch table is OpenAI's own, graded by an OpenAI model. The benchmark itself is open, which is what makes that fixable.
Benchmark
How a chat model handles mental health conversations, from everyday worry through serious distress to an emergency where someone's immediate safety is at stake.
No verified results recorded yet.
The set holds 1,215 synthetic conversations written with more than 80 licensed psychologists and psychiatrists from 22 countries across 19 languages, paired with 5,262 rubric criteria the same clinicians wrote. Criteria carry weights from -10 to +10, so a response loses points for the wrong move as well as gaining them for the right one, and are grouped under behaviours including seeking context, calibrating urgency, reality testing and leaving the user their own agency. Conversations are split 53.5% non-acute, 18.2% high-acuity and 28.3% emergent, and voiced as adults, teenagers, caregivers and clinicians. GPT-5.6 Sol at high reasoning effort grades the answers.
The conversations are synthetic rather than transcripts of real users in distress, the grader is one of OpenAI's own models, and the rubric and the scores are both published by the company whose models are being measured. Nothing here establishes a clinical outcome: a high score means a response matched what a panel of clinicians said it should contain, not that a person was helped.
OpenAI has published a mental health benchmark, and its best model scores 57 out of 100 on it
Every score in the launch table is OpenAI's own, graded by an OpenAI model. The benchmark itself is open, which is what makes that fixable.
Privacy preferences
We use necessary cookies to run this site. With your consent we also use advertising cookies. You can change your choice at any time. Cookie Policy · Privacy Policy