MIMESIS: Learning User Simulators as Training Environments for Interactive Agents

Hugging Face Daily Papers Papers
user-simulation reinforcement-learning interactive-agents language-models self-distillation multi-turn

Summary

本文提出MIMESIS，一个基于真实用户对话与显式推理监督训练的专用用户模拟器（9B模型），在行为保真度上超越前沿模型；研究还将该模拟器冻结后用于多轮强化学习训练智能体，并提出Coached On-Policy Self-Distillation（CSD）方法，在九个未见用户模拟器上实现更强泛化性能。

Training and evaluating interactive language agents typically requires rich user interactions, yet collecting human feedback is expensive and difficult to scale. Simulated users offer a scalable alternative, but they must both resemble real user behavior and provide useful learning experiences for agents. In contrast, most agent-training frameworks rely on off-the-shelf assistant LLMs, whose helpfulness can make them overly cooperative, explicit, and behaviorally homogeneous compared with real users. We introduce MIMESIS, a purpose-built user simulator trained on human conversations with explicit reasoning supervision and 13 realistic behavioral patterns derived from real user interactions. Empirically, our 9B model achieves a SOUL-Index of 65.7, surpassing the strongest frontier model. Compared with Claude-Opus-5, the strongest baseline on RealUserSim and SimulatorArena, MIMESIS improves behavioral fidelity by 13.4 points and reduces Turing distance by 3.6 points, respectively. We then freeze the simulator and train an agent by interacting with the frozen simulator using multi-turn reinforcement learning. Across eight environments, training with MIMESIS yields better agent performance than training with GPT-5.5 under all nine unseen user simulators, demonstrating stronger generalization to new user simulators. Moreover, we propose Coached On-Policy Self-Distillation (CSD), which leverages simulator-generated private reasoning traces and subsequent utterances as feedback on how well the agent addresses user needs. A coach converts this information into concise coaching notes that describe how the agent can better anticipate user needs and adapt its behavior over the course of an interaction. CSD turns this feedback into dense, token-level supervision beyond sparse task rewards, yielding further gains across all nine evaluation user models.
Original Article
View Cached Full Text

Cached at: 10/08/26, 07:59 PM

Paper page - MIMESIS: Learning User Simulators as Training Environments for Interactive Agents

Source: https://huggingface.co/papers/2610.09484

Abstract

Trainingandevaluatinginteractivelanguageagentstypicallyrequiresrichuserinteractions,yetcollectinghumanfeedbackisexpensiveanddifficulttoscale.Simulatedusersofferascalablealternative,buttheymustbothresemblerealuserbehaviorandprovideusefullearningexperiencesforagents.Incontrast,mostagent-trainingframeworksrelyonoff-the-shelfassistantLLMs,whosehelpfulnesscanmakethemoverlycooperative,explicit,andbehaviorallyhomogeneouscomparedwithrealusers.WeintroduceMIMESIS,apurpose-builtusersimulatortrainedonhumanconversationswithexplicitreasoningsupervisionand13realisticbehavioralpatternsderivedfromrealuserinteractions.Empirically,our9BmodelachievesaSOUL-Indexof65.7,surpassingthestrongestfrontiermodel.ComparedwithClaude-Opus-5,thestrongestbaselineonRealUserSimandSimulatorArena,MIMESISimprovesbehavioralfidelityby13.4pointsandreducesTuringdistanceby3.6points,respectively.Wethenfreezethesimulatorandtrainanagentbyinteractingwiththefrozensimulatorusingmulti-turnreinforcementlearning.Acrosseightenvironments,trainingwithMIMESISyieldsbetteragentperformancethantrainingwithGPT-5.5underallnineunseenusersimulators,demonstratingstrongergeneralizationtonewusersimulators.Moreover,weproposeCoachedOn-PolicySelf-Distillation(CSD),whichleveragessimulator-generatedprivatereasoningtracesandsubsequentutterancesasfeedbackonhowwelltheagentaddressesuserneeds.Acoachconvertsthisinformationintoconcisecoachingnotesthatdescribehowtheagentcanbetteranticipateuserneedsandadaptitsbehavioroverthecourseofaninteraction.CSDturnsthisfeedbackintodense,token-levelsupervisionbeyondsparsetaskrewards,yieldingfurthergainsacrossallnineevaluationusermodels.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2610\.09484

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper3

#### phanviethoang1512/MIMESIS-9B Text Generation• 9B• Updated22 minutes ago • 411 #### phanviethoang1512/MIMESIS-4B Text Generation• 5B• Updated22 minutes ago • 427 #### phanviethoang1512/Qwen3-8B-UserRL-Agent Text Generation• 8B• Updated21 minutes ago

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.09484 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.09484 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

MADS: Multi-Agent Dialog Simulation for Robust Conversational Agents

arXiv cs.CL

提出了 MADS（多智能体对话模拟）合成数据生成管线，用于生成含对抗性、失败场景的对话轨迹，并构建了覆盖 6 个领域、55 个角色和 1095 段对话的 MADS-Bench 基准，以评估任务导向对话智能体在非合作用户行为下的鲁棒性。

Continually Post-training Small Models via Synthetic Self-distillation

Hugging Face Daily Papers

该论文提出Synthetic Self-Distillation (SynthSelf)，通过借助同家族稍大的对齐模型进行合成自蒸馏，实现小型模型的持续后训练，在保持原有分布的同时兼顾新能力获取与知识保留，并在多个基准上超越现有方法。

Cowork-Gym: Synthesizing Interactive Work Environments at Scale

Hugging Face Daily Papers

Cowork-Gym 通过 build-execute-revise 循环与有状态用户模拟器大规模合成协作式 agent 环境（64.9K 个、14 个应用场景），并用于后训练 9B/27B/35B-A3B Qwen 模型，在六个 agentic 基准上分别提升 8.7、6.4、4.1 分，其中 27B 版本在 PawBench 和 ClawEval 上取得最高平均分。

RealUserSim: Bridging the Reality Gap in Agent Benchmarking via Grounded User Simulation

arXiv cs.AI

The paper introduces RealUserSim, a framework that grounds LLM-based user simulation in real human behavioral data from 14,000+ authentic conversations to bridge the reality gap in agent benchmarking. It shows that grounded simulation raises behavioral match rates from 24.2% to 45.3% and reveals failure mechanisms invisible to cooperative simulators.