Benchmark
GPQA Diamond
Multiple-choice accuracy on graduate-level biology, physics and chemistry questions written by domain PhDs to be hard even for skilled non-experts with web access.
- Maintainer
- New York University and collaborators
- Score
- %
- Last checked
- September 2, 2026
Leaderboard
No verified results recorded yet.
Methodology
Questions were authored by PhD-level experts and validated so that other experts in the same field answer them correctly while non-experts, given unrestricted web search and substantial time, largely do not. Diamond is the highest-quality subset of that set.
Important limitations
Multiple-choice format allows partial credit from elimination, so accuracy overstates genuine understanding. The question set is public, which makes training contamination a live risk for any model trained after its release.