Benchmark

MentalHealthBench

How a chat model handles mental health conversations, from everyday worry through serious distress to an emergency where someone's immediate safety is at stake.

Maintainer
OpenAI
Score
%
Last checked
September 23, 2026

Leaderboard

No verified results recorded yet.

Methodology

The set holds 1,215 synthetic conversations written with more than 80 licensed psychologists and psychiatrists from 22 countries across 19 languages, paired with 5,262 rubric criteria the same clinicians wrote. Criteria carry weights from -10 to +10, so a response loses points for the wrong move as well as gaining them for the right one, and are grouped under behaviours including seeking context, calibrating urgency, reality testing and leaving the user their own agency. Conversations are split 53.5% non-acute, 18.2% high-acuity and 28.3% emergent, and voiced as adults, teenagers, caregivers and clinicians. GPT-5.6 Sol at high reasoning effort grades the answers.

Important limitations

The conversations are synthetic rather than transcripts of real users in distress, the grader is one of OpenAI's own models, and the rubric and the scores are both published by the company whose models are being measured. Nothing here establishes a clinical outcome: a high score means a response matched what a panel of clinicians said it should contain, not that a person was helped.

Related news