A Google model at the top, and a fifteenfold price gap under it

Gemini 4 Carbon, run at its “high” reasoning setting, now leads MathArena, the leaderboard that scores models on mathematics competitions held after the model shipped. Its expected performance is 85.4% ±3.0%, ahead of Claude Fable 5.2 (max) at 82.8% ±5.3% and OpenAI’s GPT-6.1 Sol (max) at 80.6% ±2.7%.

The column next to the scores is the one worth reading. MathArena reports the average dollar cost of a single run on a single problem, and on that measure the ranking inverts. Gemini 4 Carbon costs $15.70 ±$1.33 per problem. Claude Fable 5.2 costs $40.20 ±$2.59. GPT-6.1 Sol costs $1.07 ±$0.018 — roughly a fifteenth of the leader’s price for 4.8 percentage points less accuracy, and a thirty-seventh of the second-placed model’s.

Those confidence intervals matter too. The gap between first and second is 2.6 points, while the intervals around them are ±3.0 and ±5.3. On this benchmark’s own numbers, first and second place overlap.

A clock face on a plain wall
Gemini 4 Carbon's most common failure is exceeding the harness time limit. Illustrative image. Mikhail Nilov · pexels · Pexels License

Why competitions held after release

MathArena’s design answers the contamination problem directly. Rather than drawing on problem sets that may have circulated in training data, it runs models on competitions within days of the contest taking place, repeats each problem several times, and normalises scores per competition so one hard contest does not dominate the average. Competitions are retired once their problems have been public long enough to be considered compromised. The headline figure is an average across those still counted as current.

That makes it one of the few public mathematics leaderboards where a high score cannot be explained by having seen the answer. It also makes the numbers noisier than a fixed benchmark, which is why the published intervals are as wide as they are.

What Gemini 4 Carbon gets wrong

The Decoder reported on 14 November, citing a post by MathArena’s Jasper Dekoninck, that the model’s most common failure mode is exceeding the time limit rather than reaching a wrong answer. It almost never abandons a task. In one case it ran a script for more than eleven hours without checking on its progress.

Whether that is a virtue depends on what the score is for. A model that keeps working is useful to a researcher with a hard problem and expensive to anyone paying per run. On a benchmark with a harness time limit, it is simply a loss.

A coin-operated meter on a street
The leaderboard reports cost per problem alongside accuracy. Illustrative image. Raphael Loquellano · pexels · Pexels License

The open-model gap

The best open-weights entry on the board is Moonshot AI’s Kimi K3.1 Pro (max), at 54.3% ±5.8% for $3.70 ±$0.27 per problem. That is a little over 31 points behind the leader — a wider gap than the open and closed tiers show on most general-purpose benchmarks, and a reminder that competition mathematics is one of the places where the frontier is still clearly separated.

What to watch

Two things. The first is whether the cost column changes anyone’s reading of these tables: a leaderboard that publishes price alongside accuracy makes it harder to present a top score without saying what it took to get there. The second is the next retirement cycle. Because MathArena drops competitions as they age, a model’s position can move without the model changing at all, and the current ordering is a snapshot of which contests are still counted.