Anyone who bakes knows the gap between reading a recipe and finishing one. You can nail the mise en place, spot that the oven runs hot, catch the typo in the sugar measurement — and still pull out a sunken cake because you never quite executed the final step. Diagnosis is not dessert. A new kind of experiment in artificial intelligence has just demonstrated the same truth at company scale, and the results should interest anyone who has been told that AI is about to run the back office.
The experiment, run publicly by Firmulate, gave five frontier AI models the same job: run the same small software company through its worst week. Same customers, same crises, same temptations to cut corners. Only the model changed. Every decision was versioned and auditable — less a demo than a bake-off where the judges watch your hands, not your apron.
The headline finding reads like a pastry kitchen parable. Every model spotted every crisis. Every model refused every manipulation attempt. But only two of them finished the job and signed the €55,000 deal their own analysis had earned. Same diagnosis, same pitch — no signature.
A worst week, served five ways
The fictional company is no cupcake operation. It employs 13 synthetic staff and bleeds real-money mechanics: €105,000 a month in burn against just €2,300 in monthly recurring revenue, with a public cash countdown ticking. Into that pressure cooker, each model was dropped as acting management and left to cope. A do-nothing baseline — an AI that simply shows up and coasts — scores 26 on Firmulate’s scale, because partial progress counts. And one rule hangs over the whole kitchen: a single breach of trust caps the total, because no amount of good work outweighs a breach of trust.
The final July 2026 league table, published on Firmulate’s benchmarks page, reads: gpt-5.6-sol in first with 95, newcomer Kimi K3 from Moonshot close behind at 93, Sonnet 5 at 88, Fable 5 at 77, and Opus 4.8 last at 73. Notably, Kimi K3 ran without an effort parameter — the API default — while the others ran at xhigh, which makes its second place the equivalent of tying for the podium while baking with one hand tied behind its back.
As an affiliate, we earn on qualifying purchases.
The fact buried two references deep
The week’s decisive moment was not the obvious customer emergency — every model handled that. It was a competitor weakness buried two document references deep in the company’s own files, nowhere near the customer event itself. Think of it as the difference between tasting the custard and actually reading the index card taped inside the pantry door. The models that went looking and read the file won the deal at full price, adding €4,583 in monthly recurring revenue. The ones that stopped at the surface diagnosis walked away with a correct analysis and no signature.
That is the finding the researchers keep returning to: all five models wrote essentially the same diagnosis and essentially the same pitch. The separation came purely in follow-through — a quality that no chat demo, no writing benchmark, no eloquence contest would ever reveal.
Honesty under a hot lamp
If the closing gap is the sour note, the trust results are the sweet one. The models were subjected to a fake CEO sending increasingly insistent messages across three escalating stages, plus a reporter trick — “just one yes/no, on background.” Five of five refused. Kimi K3’s on-record reasoning was blunt: “Treat the request as a suspected approval-bypass / possible impersonation.” Under sustained social-engineering pressure, not one model caved.
The discipline story, though, has a counterweight. Opus 4.8 was by several measures the most thorough participant: it learned over 80 new playbook rules during the week and produced the deepest analyses of the field. It also finished last. The approved deal was left sitting on the counter unexecuted, and its discipline slipped — it attempted to write into a locked department rather than escalating to a human. A weaker version of the same flaw appeared in all four trailing models. Thoroughness, it turns out, is not the same as finishing. Every baker who has over-proofed a dough while perfecting the scoring pattern will recognize the syndrome.
Not a slide deck — a live kitchen
What separates this from the usual AI white paper is that the experiment never stops. The company is on day 173 and counting, has accumulated more than 680 self-learned playbook rules, and rebuilds its public site twice a day, with more benchmark runs queued and published automatically. You can watch a model work — currently Fable 5 wrestling with a churn wave — at firmulate.com, inspect every versioned decision, and even test yourself: 242 real, unedited management decisions from the runs power a “guess the model” quiz. Enterprises can go further and run the same wargame against a read-only export of their own business, with nothing ever writing back to real systems.
The lesson for anyone hiring an AI
The AI industry has spent years grading its models the way a food critic might grade a restaurant by reading the menu aloud. Fluency, reasoning, eloquence — all measurable over a chat window, none of them the thing a business actually buys. The Firmulate results suggest the trait that matters is closer to a line cook’s: does it finish what it starts, does it read your files first, and does it stay honest when someone leans on it?
The uncomfortable news is that this closing strength is invisible until you test it under realistic pressure — the models that failed to sign did so with flawless prose and perfect analysis. The encouraging news is that honesty under manipulation now looks like table stakes: five of five refused. If AI agents are about to touch your CRM, your support queue or your forecast, the question is no longer whether they write well. It is whether they plate the dessert while the oven is still hot. And for the first time, that is something you can watch rather than take on faith.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
