Benchmark
Multiple-choice accuracy on graduate-level biology, physics and chemistry questions written by domain PhDs to be hard even for skilled non-experts with web access.
No verified results recorded yet.
Questions were authored by PhD-level experts and validated so that other experts in the same field answer them correctly while non-experts, given unrestricted web search and substantial time, largely do not. Diamond is the highest-quality subset of that set.
Multiple-choice format allows partial credit from elimination, so accuracy overstates genuine understanding. The question set is public, which makes training contamination a live risk for any model trained after its release.
Reflection says its new open model matches GLM 5.2 on a quarter of the inference compute
The Nvidia-backed lab raised about $4.7bn to put a Western model back on the open-weight frontier. Its first release sits a point below GLM 5.2 on every test it published — and the pitch is the inference bill, not the score.
The company published the detection rates alongside the announcement, and they fall sharply on short passages, on mathematics, and on anything that has been edited.
Aleph Alpha has opened a 78-billion-parameter German model that uses 3.5 billion at a time
The Heidelberg lab is selling jurisdiction rather than frontier capability. Every benchmark score it published is its own, and the rivals it chose to measure against all date from the spring.
Artificial Analysis doubles the secret half of its index to stop labs gaming it
Version 4.2 adds an agentic knowledge-work evaluation and a 4,592-page document-reasoning set, and drops a benchmark the maintainer says models have exhausted.
Privacy preferences
We use necessary cookies to run this site. With your consent we also use advertising cookies. You can change your choice at any time. Cookie Policy · Privacy Policy
