τ-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions
Summary
τ-Emotion-Bench introduces a ten-domain benchmark for evaluating tool-calling LLM agents under emotion-conditioned user interactions, alongside Tracer, a scalable agentic data-generation pipeline that improves Qwen3 models and generalizes to τ-Bench and τ-Trait, revealing that frontier agents remain unreliable when users behave emotionally.
View Cached Full Text
Cached at: 10/15/26, 07:03 AM
Paper page - τ-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions
Source: https://huggingface.co/papers/2610.19560 Published on Oct 14
·
Submitted byhttps://huggingface.co/sahsaeedi
Amiron Oct 15
Abstract
Inlong-horizontool-callingtasks,LLMagentscanfailnotonlybecauseoftool-useerrors,butalsobecauseofineffectiveinteractionwithusers.Yetexistingbenchmarkslargelyassumecooperativeandbehaviorallystableusers,leavingtheeffectofemotionalvariationonagentreliabilityunderexplored.Weintroduceτ-Emotion-Bench,aten-domainbenchmarkforevaluatingtool-callingagentsundercontrolledemotion-conditioneduserinteractionswhilepreservingtheunderlyingtaskobjective.Toscalebothbenchmarkconstructionandagenttraining,weintroduceTracer,ascalableagenticdata-generationpipelinethatconstructsgrounded,policy-validtasksandcollectstrajectoriesthroughaninteractiveagent--user--environmentharness.Byallowingdissatisfieduserstoterminateinefficientinteractions,theharnessprovidesaninteraction-awaresignalthatfilterstrajectorieswithexcessiveuserdependenceandfavorseffectivetooluse.Experimentswithtwousersimulatorsshowthatfrontieragentsremainunreliable,withsubstantialgapsbetweenaveragetasksuccessandconsistenttaskcompletion.Tracer-generatedsupervisionconsistentlyimprovesQwen3modelsacrossscales,generalizestounseendomains,andtransferstoτ-Benchandτ-Trait.Behavioralanalysesfurthershowthathighertasksuccessisassociatedwithabetterbalancebetweentoolexecutionandnecessaryuserfollow-up.Overall,τ-Emotion-BenchandTracerprovideascalableframeworkforevaluatingagentreliabilityunderdiverseuserbehaviorsandimprovingtool-callingagentsthroughinteraction-awaresupervision.
View arXiv pageView PDFProject pageGitHub4Add to collection
Get this paper in your agent:
hf papers read 2610\.19560
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper6
#### tau-emotion-bench/Tracer-Low-tau-emotion-data Viewer• Updatedabout 2 hours ago • 21.4k • 2 #### tau-emotion-bench/Tracer-High-tau-emotion-data Viewer• Updatedabout 2 hours ago • 32.7k • 1
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.19560 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
EIBench: A Simulator-Based Benchmark and Turn-Credit RL for Emotion Management
EIBench introduces a simulator-based benchmark for interactive emotion management in LLMs, enabling evaluation and training via per-turn user state feedback. The authors propose CTC-GRPO, a reinforcement learning method that improves emotion management performance across multiple benchmarks.
BenchTrace: A Benchmark for Testing Reflection Ability and Controlled Evolution in LLM Agents
BenchTrace is a benchmark for evaluating the self-evolution abilities of LLM agents, focusing on reflection and controlled evolution through a dataset of 1,821 annotated episodes and two evaluation tasks: Reflection Evaluation and Evolution Evaluation. Experiments with Qwen3-32B and GPT-4.1 show both models struggle, with a main bottleneck in diagnosis and issues in generalization and forgetting.
Toward User-Conditioned Evaluation of Personal LLM Agents under Temporal Interventions
This paper argues that evaluating personal LLM agents requires replaying temporal interventions across different user-conditioned states and identifies a gap in current benchmarks. It proposes a minimal benchmark design and reporting metrics for user-conditioned adaptation.
CAREBench: Evaluating LLMs' Emotion Understanding by Assessing Cognitive Appraisal Reasoning
Introduces CAREBench, a benchmark grounded in appraisal theory to evaluate LLMs' emotion understanding through cognitive appraisal reasoning, revealing that current models struggle with reasoning and positive emotion recognition despite matching humans on some downstream tasks.
Exposing Weaknesses in Emotion Recognition in Conversations
This paper investigates weaknesses in emotion recognition in conversations (ERC) by analyzing LLM performance in zero-shot settings, revealing systematic failures due to annotation ambiguity, and proposes an LLM-as-Judge framework for more robust evaluation.