If you’ve ever wondered whether AI can truly finish what it starts — especially in high-stakes business — the latest experiment from Firmulate offers a revealing answer. In a controlled live scenario, four advanced AI models faced the same real-world company crisis. The results? All recognized the problems and refused manipulation attempts, but only two managed to close the deal they had earned. This isn’t about chat quality; it’s about execution under pressure.
The Experiment: Putting AI to the Business Test
Firmulate took four of the top AI models — including gpt-5.6-sol, Kimi K3, Sonnet 5, and Fable 5 — and tasked them with running a real, money-using software company through its worst week. Each model was given identical customer crises, temptations to cheat, and decision-making challenges. Every decision was versioned and auditable, simulating real management dilemmas, from handling customer issues to resisting manipulative tactics.
Construction Program Management – Decision Making and Optimization Techniques
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Measuring More Than Just Chat Ability
While many AI demonstrations showcase impressive chat demos, this experiment focused on a different metric: management performance. All four models identified every crisis and refused every manipulation attempt, including a staged fake CEO message designed to escalate demands. Their integrity was tested, and they all passed that test perfectly.
The AI Compass for Supply Chain Leaders: How Generative, Agentic, and Narrow AI are Reshaping Strategy, Planning, and Execution
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Hidden Weakness: Execution and Attention to Detail
The critical difference emerged in their ability to close deals and follow through. Only two models, gpt-5.6-sol and Kimi K3, completed the transaction, signing the €55,000 deal their own analysis had earned. The other two models, despite their excellent diagnosis, left the deal on the table or slipped in execution. The decisive factor? It turns out the buried fact in the company’s files — a detail only read and understood by the models that finished the job — was key to sealing the deal. This shows that reading and acting on critical information is crucial, and often invisible in standard chat demos.
Never Be Closing: Surviving and Thriving in the AI Apocalypse with Culture, Speed, and Soul.
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Real-World Company and Its Challenges
The experiment was run on a live, functioning company with 13 synthetic employees, real money mechanics, and a public cash countdown. It burns €105,000 monthly against a modest €2,300 monthly recurring revenue, making every decision costly and measurable. The company’s daily operations, including over 680 self-learned rules, are fully versioned and observable at firmulate.com/live. This setting allows enterprise decision-makers to see exactly how their AI workforce performs in real scenarios, not just in neat chat windows.
Successful Construction Project Management: The Practical Guide
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Lessons for Business Leaders
The key takeaway: The ability to recognize problems is important, but the true measure of AI’s value lies in its capacity to execute, stay disciplined, and follow through on commitments. The experiment shows that AI models can resist manipulation and identify crises; however, the difference between a successful deal and a missed opportunity often hinges on reading the right document reference and acting accordingly — a subtlety rarely captured in chat demos.
Implications for AI Adoption
If your organization plans to let AI touch critical customer support, sales, or decision systems, the question you should ask isn’t just about how well it writes or responds. It’s whether the AI can consistently complete complex tasks, read your files thoroughly, and maintain honesty under pressure. The experiment underscores that the true measure of management quality isn’t just in identifying issues but in executing solutions — a skill that remains invisible until tested in real-world scenarios.
Test Your AI Performance Today
Firmulate offers tools for enterprises to simulate their own business scenarios — a read-only wargame against their AI models — without risking real systems. This allows decision-makers to see how their AI workforce performs in managing crises, closing deals, and resisting manipulation, before deploying it into live environments. Find out how your AI measures up at firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
