Tag
AgentMercury shows that training AI agents in simulated business environments generated from plain descriptions can transfer effectively to evaluation benchmarks, even if the training worlds are unrelated. The system improved performance through fine-tuning on construction traces.
This paper introduces a framework for constructing verified synthetic web environments to improve the training of web agents, demonstrating enhanced performance and transferability across benchmarks.
SPADE introduces a self-play reinforcement learning framework for language models that generates adaptive executable training environments to enhance reasoning and tool-use capabilities, demonstrating significant performance gains across multiple benchmarks.
A tweet introducing the eval-engineering skill from langchain-ai/langchain-skills, which uses human feedback to generate aligned environments, harnesses, and tasks for agent evaluation. It explains the workflow and provides installation instructions for the open-source tool.
TRACE is a new self-improvement approach where an AI agent identifies the missing capabilities behind its own failures and trains itself to address them. TRACE-trained Qwen3.6-27B achieves 73.2% on SWE-bench Verified, outperforming much larger models with fewer training rollouts.
This paper presents a proof-of-concept using Reinforcement Learning with Verifiable Rewards (RLVR) to train small language models for tool-use in enterprise SaaS workflows like Jira and Confluence. The approach uses synthetic environments and GRPO training to improve tool-call accuracy, achieving significant reward gains over baselines.
Patronus AI raises $50M in Series B funding to build simulated digital worlds for stress-testing AI agents, helping ensure they perform reliably in real-world scenarios.
HalluWorld is a controlled benchmark framework for evaluating hallucination in large language models using explicit reference world models across synthetic environments like gridworlds, chess, and realistic terminal tasks. It enables fine-grained analysis of failure modes such as perceptual hallucination, multi-step state tracking, and causal simulation, revealing that frontier models still struggle with complex reasoning not solved by extended thinking.