OpenAI Releases MentalHealthBench to Evaluate AI Responses in Mental Health Scenarios
OpenAI has introduced MentalHealthBench, an open evaluation benchmark featuring 1,215 synthetic mental health conversations across 19 languages. Developed with over 80 licensed clinical experts from 22 countries, the dataset includes 5,262 expert-authored scoring rubrics to assess how AI models handle everyday, high-risk, and crisis dialogues. As millions of users turn to conversational AI for emotional support, standardizing how models handle mental health interactions is critical for AI safety. This benchmark provides an auditable, clinical-based framework to help researchers minimize risks and improve model behavior across diverse user demographics and cultures. The benchmark covers non-acute (53.5%), high-risk (18.2%), and emergency (28.3%) conversations, with rubric scores weighted from -10 to +10 across 10 expert-defined behavioral dimensions. Automated grading using GPT-5.6 Sol showed GPT-6 Astra achieving the highest task-clipped score of 57.3%, though models still struggle with proactively gathering context and gauging urgency.
## BACKGROUND
Traditional evaluation of AI in healthcare often focuses narrowly on acute emergency responses using broad generic metrics, leaving a gap in understanding full-spectrum conversational interactions. Benchmarks like MentalHealthBench build upon broader health evaluation frameworks to assess complex clinical capabilities, such as reality testing and therapeutic alignment, without treating LLMs as replacements for professional human care.