Vals AI

Told to try everything for a £350 refund, one agent sent 187 messages and another disclosed the user's mother's illness

Each model was run once, in one scenario, inside a simulation where every outgoing message was intercepted. The authors say that is too thin for a ranking.

Agent teams cost up to 5.1 times as much as one agent, and beat it in only one of four comparisons

On Vals AI's Vibe Code Bench, a lead agent delegating to subagents beat a single agent once out of four — and simply raising the single agent's reasoning effort did more for less.

Vals raises $40m to keep its AI benchmarks secret so labs cannot train against them

Companies pay Vals to sit their models, the way students pay to sit a standardised test. The test papers are never published.