Nine evaluations, four equal quarters
Artificial Analysis rebuilt its Intelligence Index on 27 October. Version 5 runs nine evaluations in four categories weighted at 25% each: knowledge work, coding, science and general ability.
Knowledge work splits into AA-Briefcase at 15%, AutomationBench-AA at 5% and GDP.pdf at 5%. Coding is FrontierCode 2.0 at 15% and Terminal-Bench 4.0 at 10%. Science is Terminal-Bench-Science at 15% and the hard subset of CritPt v2 at 10%. The general quarter is AA-Omniscience v1.1 at 15%, itself split 10% accuracy and 5% non-hallucination, plus HLE-Diamond at 10%.
Two evaluations are new: FrontierCode 2.0, a private coding benchmark from Cognition, and Terminal-Bench-Science, 70 agentic scientific research tasks. Three are upgrades — HLE-Diamond replaces Humanity’s Last Exam with a refined 1,000-question version, AA-Omniscience v1.1 strips out saturated and public questions and revises ambiguous ones, and CritPt v2 contributes a private 40-question hard subset. Three were dropped: GDPval-AA v2.1, SciCode and AA-LCR v1.1. Artificial Analysis gives no reasons for the removals.
Sixty per cent of the score is now unpublished
The structural change is that 60% of the index now rests on private test sets or held-out answers. Artificial Analysis names FrontierCode 2.0 and the CritPt v2 hard subset as private outright, and says AA-Omniscience keeps a private set with a separate public subset released on Hugging Face. It does not itemise which evaluations make up the rest of the 60%.
The reason to do this is contamination: a benchmark whose questions are on the public internet eventually ends up in training data, and a score on it stops measuring what it was built to measure. The cost is that a score nobody outside the evaluator can reproduce has to be taken on trust.
The open-weights gap went from 3 points to 12
The effect on the leaderboard is lopsided. Under v4.3 the best open-weights model sat 3 points behind the leader. Under v5 that gap is 12.
Moonshot’s Kimi K3.1 Pro is the best open-weights entry at 46 — and its weights have not actually been released yet. Among models whose weights are out, DeepSeek V4.1 Flash 1017 at maximum reasoning and GLM-5.3 at maximum reasoning lead on 32, which is 26 points behind the top of the table. Anthropic’s own small model, Claude Haiku 5.5 at maximum reasoning, scores 36, above both.
The top of the table, for one day
At publication the index read Claude Opus 5.5 at maximum reasoning on 58, GPT-6 Astra at maximum on 57 and GPT-6.1 Sol at maximum on 55, with Anthropic and OpenAI holding all five top places and Gemini 4 Argon at high reasoning next on 50.
That lasted a day. Google released Gemini 4 Carbon on 28 October and Artificial Analysis scored it 59, first on the new index.
What to watch
The number worth tracking is the gap, not the leader. A 12-point spread measured mostly on tests the open-weights community cannot run itself is a different kind of claim from a 3-point spread on public ones, and the first open-weights release scored under v5 with published weights will say whether the gap is a property of the models or of the measurement.
