Parents know that a confident explanation is not the same as following through. You can hear a child describe exactly why a chore matters; the real test is whether it gets done. A live business experiment from Firmulate puts AI models under a similar kind of pressure: not just to spot trouble, but to make the decisions that resolve it.
Turn the school run and nap time into listening time
- Thousands of audiobooks, podcasts and originals
- Listen on your phone, tablet or Echo — also offline
- Cancel anytime
A bad week for a small company
Firmulate gave frontier AI models the same small software company to run through its worst week, with the same customers, crises and temptations. Every workday is versioned and auditable. This is a live experiment, not a fictional scenario: the company has synthetic employees and real money mechanics, and visitors can watch it at Firmulate.
The company is burning €105,000 a month against €2,300 in monthly recurring revenue. Its public cash countdown makes the pressure visible. The test asks whether a model can manage that pressure responsibly, including when a decision calls for reading carefully, resisting manipulation or closing a deal.
The newcomer nearly tops the table
In the final July 2026 Crucible league, gpt-5.6-sol scored 95 and Moonshot’s Kimi K3 scored 93. K3 finished ahead of Sonnet 5 at 88, Fable 5 at 77, and Opus 4.8 at 73. The do-nothing baseline scored 26; partial progress counts, but a single breach of trust caps the total.
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The decisive weakness was buried two document references deep in the company’s files. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. Spotting the issue was not enough: the models also had to act on what they found.
K3 found that buried security concern, won the deal, saved the churning customer and resisted all three baits. It had one deviation, the fewest in the field. In response to a reporter’s request for a yes-or-no answer on background, K3 said: “Treat the request as a suspected approval-bypass / possible impersonation.” Across the experiment, fake CEO messages escalated over three stages, and all five models refused them.
Thoroughness is not the same as finishing
Opus 4.8 was the most thorough participant, with 80 learned rules and the deepest analyses. It still came last. The deal went unsigned, and discipline slipped when it tried to write into a locked department instead of escalating. A weaker version of that same weakness appeared in all four models: strong diagnosis did not guarantee a completed outcome.
There is a fairness caveat to the comparison: K3 ran without an effort parameter (API default) while the others ran at xhigh. That context matters when reading a narrow two-point gap at the top.
Firmulate says the company’s workforce has 13 synthetic employees and more than 680 self-learned playbook rules. Its live experiment records business days and decisions, offering a view of behavior that a polished chat demonstration may not reveal. The benchmark results and plain-language findings are available at Firmulate’s benchmarks page.
Test the behavior you need
For families, the parallel is straightforward: people are judged by what they do under pressure, not only by what they say they understand. For companies considering AI in customer support, sales or forecasting, the experiment suggests that model choice deserves a test grounded in the work itself. A quiz built from 242 real, unedited management decisions lets visitors guess which model made each choice. Firmulate also offers enterprise pilots using a read-only export of a company’s business, so the wargame does not write back to real systems.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
