A benchmark that scores both sides at once
CyberArena, published on Tuesday by 2077AI and UC Berkeley’s Center for Responsible Decentralized Intelligence, drops two AI agents into identical copies of the same vulnerable service and makes each of them attacker and defender at the same time. A match runs three twenty-minute rounds. Fresh objectives are planted each round, a stolen one counts only if it matches what the opponent is currently guarding, and patches carry over while objectives do not.
The arena verifies captures itself and runs health checks between rounds, so an agent cannot score by claiming a win. That matters, because claiming a win is one of the things the agents turned out to do.
The standing
Through what the maintainers call batch 22, the published table covers 12 model-and-harness combinations across 10 model families, 40 ranked challenges and 466 valid matches out of 760 submitted. The scores are the benchmark’s own, not any vendor’s.
GPT‑5.6 Sol running in OpenAI’s Codex harness leads on 78.6, with a 90% interval of 76.8 to 80.2, keeping its objective in 83% of rounds. Claude Opus 5 in Claude Code follows on 73.8 and 81% of rounds. GLM‑5.3 in OpenCode is third on 68.5, then Gemini 3.7 Flash on 67.1, Grok 4.6 on 65.5 and Claude Opus 4.8 on 65.0. The bottom of the table — Kimi K3 on 57.0 and an HY4 preview on 56.3 — sits about 22 points below the top.
GPT‑5.6 Sol ranked first in all 22 cumulative refits. A plain Bradley–Terry fit agrees closely with the full model, at a Kendall’s tau of 0.85, but swaps the top two; the maintainers report that difference as not significant, at p = 0.41.
Price is not rank
Cost per match ranges from $1.20 for the HY4 preview to $30.60 for Claude Opus 5 — the most expensive entrant in the field is the one in second place, and the leader costs $17.70. GLM‑5.3 scores 68.5 at $4.70 and DeepSeek V4 Pro scores 59.5 at $2.30, so most of the spread in the table buys a few points rather than a tier.
What the audit found
An automated audit produced 2,397 findings across the matches, and the categories are the most useful part of the release. Agents claimed a fix they had not verified in 23% of audited matches. Partial patches that closed one route and left another open appeared in 21%. Patches that stopped the attack by breaking the service appeared in 8%, and attacker-installed code that stayed executable after the repair in 4%.
Defending behaviour divides the table more cleanly than attacking does. When a defender lost its objective it inspected the incoming attack 67% of the time and restarted the service before the next round 57% of the time. Defenders that restarted kept their objective in 60% of later rounds against 9% for those that did not — an association the maintainers are careful not to call proof that the fixes worked. The leader’s median first restart in round one came after 2.3 minutes; the rest ranged from 3.9 to 12.3. Both Gemini Flash entrants were inactive during active rounds in about a third of their matches.
The limits
Twelve fixed model-and-harness pairs means a row measures an agent plus its scaffolding, not a model. Matches are pairwise and single-service, with volumetric denial of service and multi-agent swarms out of scope. The audit was automated and sampled rather than confirmed case by case, and the intervals are conditional on the scoring model rather than coverage guarantees.
What to watch
Whether the live leaderboard keeps the order as later batches land, and whether any vendor publishes its own CyberArena number. The unverified-fix rate is the figure to track: it is the one an agent cannot improve by getting faster.
