MentalHealthBench tests more than crisis refusal

What's this about?

The benchmark covers non-acute, high-acuity and emergency conversations involving adults, teens, caregivers and clinicians across multiple languages.

What is a rubric-based benchmark?

Experts write situation-specific behaviours to reward or penalise, such as asking for missing context or preserving agency. A model grader then checks responses against that list.

Why does this matter?

1. More than 80 clinicians across 22 countries authored the criteria.
2. Synthetic conversations protect privacy but may miss real conversational messiness.
3. Using GPT-5.6 Sol as grader creates possible self-family and calibration bias.