multi-turn

MMShopBench introduces a real-log benchmark for multimodal, multi-turn shopping agents, requiring joint inference from user images and dialogue, with an offline sandbox and training set to improve open-source model performance.

Introduces CMT-RAG, a complementary memory framework for multi-turn multi-hop conversational RAG that aligns conversational memory with retrieval using sub-question-level reasoning traces. Also presents MuMu-QA, a benchmark with cross-turn sub-question dependencies.

Introduces PALATE, a scalable benchmark for evaluating role-playing agents using person-aligned LLM-simulated users and personalized rubrics, addressing limitations of fixed-history evaluation.

This paper presents an evaluation of multi-turn multimodal diagnostic reasoning using challenging real-world clinical cases, aiming to assess AI models' ability to handle complex medical scenarios.

CallBench is a Chinese benchmark for evaluating dual-goal coordination in phone call assistants, containing 50,000 multi-turn dialogues across six scenarios, with a preset-aware evaluation protocol covering semantic understanding, safety, and dialogue rhythm.

EvoCode-Bench is a multi-turn coding benchmark with 26 tasks across 5 domains, designed to evaluate AI agents on evolving specifications and cumulative testing in a persistent workspace, revealing that single-turn scores dramatically overstate reliability.

This paper introduces a controlled multi-turn environment to systematically study long-horizon planning ability in foundation model agents across pre-training, post-training (via GRPO and on-policy distillation), and integration (via multi-teacher on-policy distillation) stages.

This paper introduces a framework to convert static tasks into dynamic multi-turn conversations to evaluate how well LLMs track evolving user intent, finding that strong static performance does not transfer to dynamic settings.

StabilityBench is a benchmark operator that transforms single-turn LLM evaluations into multi-turn interactions with user simulations and baiting modules, revealing significant performance instability in current models and motivating more realistic evaluation protocols.

This paper identifies and characterizes two failure modes in LLM-based autonomous agents—Safety Drift and Operational Hallucination—and proposes a lightweight architectural layer to intercept violations without false positives.

User reports that Gemma 4 is lazy and poor at multi-turn agentic tasks compared to other models like Qwen, DeepSeek, and GPT-OSS, despite being a good chatbot.

LAPO proposes a leave-one-turn attribution method for self-generated process rewards in multi-turn search reasoning, enabling fine-grained credit assignment without external reward models. It achieves state-of-the-art results across seven datasets.

This paper proposes ReOPD, a method for on-policy distillation of LLM agents that reuses pre-collected teacher trajectories as replayed prefixes, achieving improved efficiency and accuracy without new environment interactions.

The paper introduces ThReadMed-QA, a multi-turn medical dialogue dataset, and evaluates five LLMs on correcting patient misconceptions, finding substantial degradation over subsequent turns.

This paper proposes DC-GRPO, a turn-level credit assignment framework for multi-turn LLM jailbreak learning, achieving over 98% attack success rates across benchmarks, outperforming existing methods.

IdeaTrail is a dataset of multi-turn process trajectories for scientific ideation, synthesizing research processes from evidence gathering to proposal construction using a Generator–Advisor loop to ensure grounding.

EvoCUA-1.5 introduces an online reinforcement learning framework for multi-turn computer-use agents, achieving a 63.2% success rate on OSWorld-Verified and outperforming comparable open-weight models up to 35B parameters through step-level policy optimization and dynamic curriculum learning.

The article identifies a structural flaw in voice agents where they cannot detect when they over-promise across multiple conversation turns, and describes building a deterministic checker that flags contradictions without relying on LLM evaluation.

A second-order early warning signal for multi-turn prompt injection is introduced, based on information geometry and a statistical manifold. The method uses a meta rate derived from the second derivative of the stability parameter to predict adversarial trajectories before threshold crossing, providing proactive detection.

BayesBench evaluates how closely large language models' belief updates match Bayesian reasoning in multi-turn evidence accumulation tasks, finding that while scaling improves latent inference, models struggle to use that understanding for downstream predictions.