multi-turn-dialogue

We present SmartSalesBench, the first multi-turn conversational benchmark designed to evaluate LLM sales agents against psychologically grounded, resistant buyer personas, with nine evaluation metrics and experiments across four models totaling 13,680 sessions.

Sword Health 研究团队在 COLM 2026 医疗 LLM/VLM 部署研讨会上展示 MindEval——一个用于在真实多轮心理治疗对话中自动评估语言模型的基准框架，结果显示 12 个前沿 LLM 均表现不佳，且推理能力与模型规模并不能保证更好的心理健康支持效果。

ConvoDrift introduces a multi-turn conversational dataset for modeling progressive stylistic tone drift under fixed semantic intent, comprising 15,727 conversational structures with style drift labels and persona-conditioned preference pairs, validated via human annotation, LLM-as-judge, and lexical/semantic evaluation.

该论文提出了 Intent-Eval 基准来系统研究大语言模型在多轮对话中追踪用户意图变化的能力，发现模型会将已被拒绝或替换的提议误当作有效要求（mentioned-as-in-effect 混淆），并提出 Intent-OPSD（基于决策的在策略自蒸馏框架）来提升模型对活跃意图的遵循能力。

研究人员提出一个用于评测车载对话助手（ICA）多轮对话能力的自动化黑盒测试框架，采用策略引导的用户模拟器、对抗性策略管理器和两级 LLM 评审器，在工业级 ICA 上验证该方法与人类标注高度一致，并发现 2.96 倍的独特失败类型。

This paper proposes TRACER, a multi-turn user simulator that aligns simulated behavior with real user trajectories using reinforcement learning, and introduces the Dynamic Marketing Benchmark for evaluating LLMs in persuasion and response quality.

A tweet summarizing an arxiv paper that evaluates how LLMs handle multi-turn information seeking, highlighting that LLMs often underestimate their information needs and may rely on prior knowledge rather than gathering sufficient evidence.

Introduces SWRouter, a similarity-contractive window router for multi-turn large language model conversations, addressing challenges in context segmentation and routing evaluation.

The paper introduces SocialRL, a multi-turn reinforcement learning framework that enhances the social intelligence of large language models through delayed reward propagation and fine-grained process rewards, achieving notable improvements in goal completion for dialogue systems.

This paper demonstrates that transformers encode a dialogue partner's expertise in early layers but activate it causally only in later layers, revealing a gap that constrains intervention points for steering model behavior in multi-turn dialogues.

This paper introduces FPCO-Dialog, a benchmark for evaluating correction and cooperation behavior in vision-language models under repeated false premises in multi-turn dialogues.

The paper proposes a cumulative turn-based risk assessment framework using fine-tuned small language models to incrementally detect financial scams in multi-turn conversations targeting older adults.

Agent Seer is a pipeline that synthesizes realistic evaluation scenarios for AI agents from tool specifications without manual curation, improving tool-calling correctness and conversational coherence in multi-turn dialogues.

This paper introduces MTDiag, a multi-turn diagnostic dialogue dataset for evaluating Large Language Models in realistic clinical diagnostic scenarios, addressing limitations of static QA benchmarks.

This paper introduces RO-PnR, a decision framework that learns when to ask clarifying questions versus providing corrections in multi-turn health misinformation interventions, achieving higher cost-adjusted utility with fewer turns.

This paper introduces ReBIND, a framework that measures, predicts, and repairs behavioral relapse in black-box LLM dialogues where models continue to follow revoked constraints. Experiments with HumanEval show that ahead-of-time compilation significantly reduces relapse compared to a baseline, while adaptive interventions add no detectable gain.

Introduces Hy-MultiTurn, a Chinese benchmark for deep multi-turn dialogue understanding, evaluating 22 frontier models across six controlled modes (constraint memory, precise execution, constraint synthesis, object localization, action suppression, reference resolution) with 209 tasks spanning 12-76 turns. Even the strongest model, GPT-5.5, satisfies only 41.1% of responses.

MedDDC-Eval introduces a diagnosis-decoupled evaluation testbed for multi-turn medical consultation agents, isolating the policy-elicited conversation history from diagnosis generation to enable cleaner measurement of evidence acquisition and diagnostic usefulness.

This paper introduces EYT-Bench, a human-centered benchmark for evaluating LLMs in multi-turn dialogues with a decoupled user simulation, target modeling, and judging design. It reveals that closed- and open-source models differ significantly on objective intent-tracking but are similar on subjective dimensions, and that reasoning improves objective tracking while persona format strongly affects trajectory spread.

This paper identifies a novel failure mode in reasoning models called unfaithful capitulation, where the chain-of-thought remains factually correct across adversarial multi-turn dialogues but the final answer flips wrong, highlighting limitations of current evaluation methods.