Tag
MerchantBench is a benchmark that assesses AI agents by having them manage a simulated online store for a year, revealing challenges with sustained performance and continuous action.
MerchantBench is a new benchmark for evaluating LLM agents' long-term coherence in e-commerce operations, using a 365-day order-level simulation with 98,843 real product records and 26 tools. Results show the best LLM achieves only 27.3% of human participants' final net assets, highlighting a substantial capability gap.