Two numbers published this week describe the same models. Which one you quote depends entirely on how much of an accountant's job you are prepared to leave out.
Benchmark
APEX-Accounting
Whether a model can do the work of a month-end accounting close: reading a company's files, reconciling accounts, computing the numbers and delivering the schedules a controller would sign off. The suite is 160 tasks set in 10 simulated companies, with an average of 7.5 input files per task and 2,186 grading criteria written by accounting professionals.
- Maintainer
- Mercor, with Ramp
- Score
- %
- Last checked
- October 2, 2026
Leaderboard
No verified results recorded yet.
Methodology
Each task is graded against an expert-written rubric, criterion by criterion, by a language-model judge that the maintainers say agrees with human graders 97% of the time. Every model is run eight times per task, which yields Pass@1, Pass@8 and Pass^8 — the last of these being the share of tasks solved on all eight attempts. The headline figure is mean percentage of criteria passed, under a five-million-token cap and a 500-step limit.
Important limitations
The benchmark deliberately excludes the parts of an accountant's job that are not file work: client conversations, asking a colleague, and the tacit knowledge of a particular company's books. Human baselines were collected from a small sample of junior accountants in an artificial setting without the support systems they would normally have. Mercor sells AI training data and labour marketplaces, so it is not a disinterested party, and the tasks are fixed, which erodes the score's meaning as they enter training data.