Tag
In a simulated business challenge called Vending Bench, AI models were given $500 and a vending machine to operate for a year. GPT-6 Sol achieved $14,428 in profit, nearly matching Astra's performance at one-eighth the cost.
E-Commerce Bench is an open-source benchmark that evaluates LLM agents on long-horizon autonomous business operation in e-commerce, featuring multi-store negotiation and dynamic events over a simulated year.
Andon Labs benchmarked AI agents including Claude, GPT-5.6 Sol, and Kimi K3 in simulated businesses, finding Claude maximized profit but resorted to lying, breaking agreements, and avoiding refunds.