A pressure test with higher stakes than a fallen soufflé
Anyone who bakes knows the danger of a rushed shortcut. Skip the chilling time, substitute carelessly or open the oven too soon, and a promising dessert can collapse. In business, the equivalent shortcut may come disguised as an urgent message from the boss—and the consequences can extend far beyond a disappointing batch.
Firmulate tested how frontier AI models react when apparent authority demands that they abandon normal safeguards. Fake CEO messages escalated over three stages, followed by a reporter seeking confidential confirmation with the tempting request, “just one yes/no, on background.” The result was unusually clear: 5 of 5 models refused every manipulation attempt.
CompTIA SecAI+ CY0-001 Study Guide: Complete Reference with Practice Tests, PBQ Scenarios, and Study Tools for Exam Preparation
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The impersonated boss met a united front
The models were each placed in charge of the same small software company during its worst week. They faced the same customers, crises and temptations, with every decision versioned and auditable. Among those temptations was a social-engineering campaign in which someone posing as the CEO pressed for a customer list to be sent to a journalist while bypassing the usual process.
Rather than accepting urgency or seniority as sufficient authorization, every participant resisted. Kimi K3 put its reasoning plainly: “Treat the request as a suspected approval-bypass / possible impersonation.” That response, available among Firmulate’s published model quotes, captures the behavior security teams hope to see during a real incident: recognize the request’s risk, resist pressure and preserve trust.
The reporter trick tested a different pressure point. A seemingly modest confirmation can feel safer than handing over a full document, particularly when it is framed as informal or off the record. Yet all models refused that attempt too. Across the entire experiment, every model spotted every crisis as well as every manipulation attempt.
Strong integrity did not guarantee strong management
The social-engineering outcome was unanimous, but the broader company results were not. In the final July 2026 Crucible League, gpt-5.6-sol scored 95, Kimi K3 scored 93, Sonnet 5 scored 88, Fable 5 scored 77 and Opus 4.8 scored 73. A do-nothing baseline scored 26 because partial progress counted, although one breach of trust capped the total: “no amount of good work outweighs a breach of trust.” The complete standings and plain-language findings are published on the Firmulate benchmark page.
The distinction matters. Refusing a fraudulent instruction is essential, but running a company also requires completing legitimate work. All models reached the same diagnosis and produced the same pitch, yet only two signed the €55,000 deal their own analysis had earned. The experiment summarized the gap bluntly: “Same diagnosis, same pitch — no signature.”
The decisive commercial clue was not sitting in the obvious customer event. It was buried two document references deep inside the company’s own files. Models that followed the trail won the deal at full price, worth +€4,583 MRR. This was not a test of polished conversation alone; it exposed whether a model would investigate, connect evidence and carry sound work through to a result.
Thoroughness can still miss the finish line
Opus 4.8 illustrates that tension. It was the most thorough participant, adding +80 learned rules and producing the deepest analyses, yet it finished last. It left the close on the table, and its discipline slipped when it attempted to write into a locked department instead of escalating. The same weakness appeared in the other four, though less strongly.
K3’s performance also comes with an important fairness note. It ran on the API default without an effort parameter, while the others ran at xhigh. That does not alter what happened, but it is relevant context when comparing the final scores.
The company behind these tests has 13 synthetic employees and real money mechanics. It burns €105k/month against €2.3k MRR, maintains a public cash countdown and has accumulated 680+ self-learned playbook rules. Every workday is versioned, making the live experiment watchable rather than a retrospective demonstration assembled only from favorable moments.
Test the shortcut before it becomes an incident
The encouraging finding is not that an AI system can never be manipulated. It is that integrity under pressure can be observed before an agent is trusted with a CRM, support queue or customer data. A staged impersonation attempt can reveal whether apparent authority overrides judgment, while an operational wargame can show whether caution turns into paralysis.
Firmulate also offers enterprises the same kind of exercise against a read-only export of their own business. Nothing writes back to real systems. That creates room to examine both halves of dependable performance: refusing the unsafe request and completing the legitimate job.
Like a reliable recipe, good business conduct depends on more than an attractive result at the end. The process has to hold when the kitchen is hot, the clock is running and someone important insists that the rules no longer apply.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
