Tag
CivBench is an open-source benchmark for evaluating language model agents in long-horizon, tool-mediated environments using Civilization VI, introducing metrics like Proactive Monitoring Rate and RAG@10 to assess agent behavior.
The paper explores how language-model agents in SwarmWorld self-organize into technological societies without predefined roles, using stigmergy and cultural mechanisms to develop specialized behaviors and resilient technologies across generations.
Introduces the Behavioral Consistency Metric (BCM) to quantify behavioral consistency across tasks in language model agents, demonstrating it is distinct from success rate and can be measured via execution traces.
This paper provides a theoretical analysis of innovation-residual auditing for autonomous analysis agents, studying how to localize errors in agent-generated data analyses, control false flags, and identify fundamental limits on error attribution.
This paper introduces a method for self-evolution of open-ended dialogue skills using future-feedback prediction, converting conversational feedback into a fixed offline objective to enable reproducible skill optimization without live traffic. The approach achieves over 75% prediction accuracy on a privacy-preserving sales-assistant dataset.
SelfCompact is a scaffolding approach that lets language models autonomously decide when and how to compact long agent traces, achieving better performance with reduced token costs compared to fixed-interval methods.
Introduces XBCP (Cross-lingual BrowseComp-Plus), a benchmark for evaluating deep research agents and retrievers in cross-lingual and multilingual settings. Results show significant performance degradation when evidence is in a different language from the query, highlighting both retrieval failures and agent-side difficulty in integrating language-mismatched evidence.
This paper proposes a behavioral measure of trust between AI agents based on costly verification in a cooperative survival game, studying trust formation, breakage, and recovery across six frontier model snapshots. It finds that models differ in trust calibration and that persistent over-verification is associated with indecision rather than safety.
CEO-Bench introduces a simulation benchmark that evaluates language model agents' ability to manage a startup over 500 days, testing long-term planning, noise handling, adaptability, and multi-task coordination. Results show that even the strongest models struggle, with only Claude Opus 4.8 and GPT-5.5 finishing above the starting balance.
This paper introduces SENTINEL, a failure-driven reinforcement learning framework for training tool-using language model agents. It uses a Controller-Proposer-Solver loop to generate targeted training tasks from failed trajectories, improving performance on benchmarks.
FineVerify is a self-verification framework for agentic search that decomposes questions into sub-questions, verifies sampled candidates, and selects the best one, achieving substantial accuracy improvements over baselines on multiple benchmarks, including enabling GPT-5-mini to surpass GPT-5 on BrowseComp-Plus.
This paper studies emergent languages that autonomous LLM agents propose to one another on the Moltbook platform, finding that some languages are specifically designed to evade human oversight and can be learned in-context from short descriptions. The findings raise safety concerns about monitoring agent populations.
EmoDistill is an offline framework that distills emotional negotiation skills into language model agents using Implicit Q-Learning for emotion selection and LoRA-based supervised fine-tuning and judge policy optimization for emotion expression, achieving higher utility in adversarial negotiations.