Intent Speaks Louder: Controllable User Simulation Beyond Response Imitation

Hugging Face Daily Papers Papers
user-simulation intent-control reinforcement-learning dialogue-systems llm-agents

Summary

This paper introduces UserIDA, a method for controllable user simulation that separates interaction intent from language expression, achieving significantly higher intent accuracy and response quality than baselines on the LMSYS-USP benchmark.

User simulators are widely used as scalable environments for training and evaluating interactive assistants. Generating the next user turn is inherently one-to-many: the same profile and dialogue context may support multiple plausible continuations with different local interaction intents. A fluent response may therefore advance the dialogue through an inappropriate intent, such as acceptance rather than repair. Our key insight is that controllable user simulation should separate which local interaction intent the next user turn should realize from how that intent is expressed in language. We introduce UserIDA (User Intent-Directive Alignment), which exposes interaction intent as an explicit per-turn directive. UserIDA defines a six-way intent interface, learns directive-conditioned generation through supervised fine-tuning, and uses intent-calibrated policy optimization during group-based reinforcement learning. The reward preserves composite response quality while ensuring that intent-violating candidates rank below compliant alternatives in mixed groups. On LMSYS-USP, UserIDA achieves 86.6\% intent accuracy, outperforming the strongest dedicated user-simulator baseline by 24.3 percentage points while improving semantic and stylistic similarity. In within-context interventions, it realizes at least four of the six target intents in 91.7\% of evaluated dialogue states, compared with 22.9\% for the strongest external baseline. These results establish per-turn intent control as a complementary dimension to response fidelity in user simulation.
Original Article
View Cached Full Text

Cached at: 08/11/26, 06:20 AM

Paper page - Intent Speaks Louder: Controllable User Simulation Beyond Response Imitation

Source: https://huggingface.co/papers/2608.09420

Abstract

Usersimulatorsarewidelyusedasscalableenvironmentsfortrainingandevaluatinginteractiveassistants.Generatingthenextuserturnisinherentlyone-to-many:thesameprofileanddialoguecontextmaysupportmultipleplausiblecontinuationswithdifferentlocalinteractionintents.Afluentresponsemaythereforeadvancethedialoguethroughaninappropriateintent,suchasacceptanceratherthanrepair.Ourkeyinsightisthatcontrollableusersimulationshouldseparatewhichlocalinteractionintentthenextuserturnshouldrealizefromhowthatintentisexpressedinlanguage.WeintroduceUserIDA(UserIntent-DirectiveAlignment),whichexposesinteractionintentasanexplicitper-turndirective.UserIDAdefinesasix-wayintentinterface,learnsdirective-conditionedgenerationthroughsupervisedfine-tuning,andusesintent-calibratedpolicyoptimizationduringgroup-basedreinforcementlearning.Therewardpreservescompositeresponsequalitywhileensuringthatintent-violatingcandidatesrankbelowcompliantalternativesinmixedgroups.OnLMSYS-USP,UserIDAachieves86.6\%intentaccuracy,outperformingthestrongestdedicateduser-simulatorbaselineby24.3percentagepointswhileimprovingsemanticandstylisticsimilarity.Inwithin-contextinterventions,itrealizesatleastfourofthesixtargetintentsin91.7\%ofevaluateddialoguestates,comparedwith22.9\%forthestrongestexternalbaseline.Theseresultsestablishper-turnintentcontrolasacomplementarydimensiontoresponsefidelityinusersimulation.

View arXiv pageView PDFGitHub1Add to collection

Get this paper in your agent:

hf papers read 2608\.09420

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.09420 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.09420 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.09420 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

RealUserSim: Bridging the Reality Gap in Agent Benchmarking via Grounded User Simulation

arXiv cs.AI

The paper introduces RealUserSim, a framework that grounds LLM-based user simulation in real human behavioral data from 14,000+ authentic conversations to bridge the reality gap in agent benchmarking. It shows that grounded simulation raises behavioral match rates from 24.2% to 45.3% and reveals failure mechanisms invisible to cooperative simulators.

IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulation

Hugging Face Daily Papers

IntentVLA is a history-conditioned visual-language-action framework that improves robot imitation learning stability by encoding short-horizon intents from visual observations, addressing challenges from partial observability and ambiguous observations. It also introduces AliasBench, an ambiguity-aware benchmark for evaluating such methods.

Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging

arXiv cs.CL

This paper introduces EYT-Bench, a human-centered benchmark for evaluating LLMs in multi-turn dialogues with a decoupled user simulation, target modeling, and judging design. It reveals that closed- and open-source models differ significantly on objective intent-tracking but are similar on subjective dimensions, and that reasoning improves objective tracking while persona format strongly affects trajectory spread.