A research deal, not a product launch
OpenAI said on 6 October that it is partnering directly with a small number of software companies to turn their hardest customer workflows into training and evaluation tasks for its agents. The first is Ironclad, a contract lifecycle platform. The point, OpenAI said, is to train models “to understand a company’s business rules, execute multi-step workflows, and verify that their work meets the original requirements”.
This is a different kind of deal from an API partnership. Ironclad supplied hosted instances of its own product for the models to practise in, and its employees — plus OpenAI staff who use Ironclad internally — helped identify 11 tasks across legal, commercial and procurement work. The examples given include setting up non-disclosure agreements, creating procurement approval processes, and updating a reusable legal clause so it reflects whichever jurisdiction a requester picks. OpenAI estimates each would take an experienced user about 30 to 40 minutes.
The numbers, and what they are measuring
Each task was scored against 8 to 50 criteria depending on complexity. OpenAI said researchers then built synthetic training tasks around representative workflows and used reinforcement learning to improve the models through practice.
Comparing GPT-6 Astra at Max reasoning with GPT-5.6 Sol at High reasoning — the settings where each scored highest — OpenAI reported a mean rubric score of 55.0% for Astra against 41.6% for Sol, and an estimated average time per attempt of 19.2 minutes against 37.0. An unreleased internal model used in Astra’s development scored 63.7%.
The footnotes matter. OpenAI states the results cover those 11 tasks and not all Ironclad workflows, and that the times are “simulated estimates based on assumed model processing and generation speeds, not measured customer time savings”. The synthetic training tasks were built from contracts in the SEC’s EDGAR database after filtering for personal information; OpenAI says it did not use customer data, its own internal contracts, or non-public Ironclad customer contracts.
Why this is worth noticing
The headline score is less interesting than the arrangement. Frontier labs have been running out of hard, verifiable tasks that look like real work, and public benchmarks keep getting contaminated or saturated. Buying access to a vendor’s live product, its domain experts and its definition of success is one answer to that — and it hands the lab a private evaluation set that nobody else can run.
Ironclad’s chief technology officer, Sunita Verma, framed the limit rather than the win: “Agents need to understand the full contracting lifecycle, including how business workflows connect while preserving the controls teams rely on.” OpenAI’s own text goes further, saying the failure mode — an agent losing track of a business rule halfway through — “underscores why human oversight still matters” and why a full contracting platform “remains essential”.
OpenAI is inviting more software companies to apply, asking for a concrete failing task, evidence of where agents fail on it, and people who know the work. The thing to watch is how many vendors decide that handing a frontier lab their hardest workflows is a good trade.
