@rohanpaul_ai: This paper shows a better way to train multi-turn agents: score each turn separately, then use a self-teacher to focus …
Summary
The paper introduces CREST, a method to improve multi-turn agent training by assigning separate credits to each turn and using a self-teacher to focus learning without overriding rewards, showing performance gains on benchmarks.
View Cached Full Text
Cached at: 09/01/26, 01:43 PM
This paper shows a better way to train multi-turn agents:
score each turn separately, then use a self-teacher to focus learning without letting it override the reward.
Standard RL has a basic problem.
A long agent session can contain successful and failed turns, yet 1 overall reward can blur them together.
CREST fixes that by giving each turn its own verified credit, then using the same model as a teacher to put more learning weight on uncertain decisions inside that turn.
The teacher can strengthen an update, but it cannot reverse the verifier’s judgment.
On Qwen3-4B-Instruct, it reaches 52.0% average BFCL V3 accuracy versus 49.25% for the strongest RL baseline.
– arxiv. org/abs/2608.13179
Title: “Teach the Magnitude, Not the Direction: Verifier-Bounded Credit Assignment for Multi-Turn Multi-step LLM Agents”
Similar Articles
Towards Better Agents for Multi-Turn User Interaction: The Next User Turn Is More Than Context
This paper introduces Feedback-Aware Credit Assignment (Faca) to improve multi-turn tool-using language agents by using next-turn user reactions as local credit signals, showing significant performance gains on interactive benchmarks.
RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training
The paper proposes Reverse-Turn Policy Optimization (RTPO) to stabilize multi-turn agentic reinforcement learning training by organizing rollouts as reverse trees and performing turn-level updates in reverse order, showing improvements over baselines.
@rohanpaul_ai: New paper from Cambridge Univ+NVIDIA and other top labs teaches AI agents and AI judges to improve together, so neither…
A new paper from Cambridge, NVIDIA, and other labs introduces the Red Queen Gödel Machine, a method where AI agents and their evaluators co-evolve to prevent stagnation. The approach avoids fixed benchmarks by allowing judges to improve at safe handoff points, leading to better performance in coding and paper writing tasks.
@rohanpaul_ai: No amount of post-training cleanly fixes weak long-horizon foundations: noisy trajectories compound errors, sparse rewa…
The paper argues that weak long-horizon foundations in AI agents cannot be fixed by post-training alone, advocating for clean world-model and long-trajectory training, and using on-policy distillation to handle sparse rewards in longer tasks.
The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation
This paper introduces a controlled multi-turn environment to systematically study long-horizon planning ability in foundation model agents across pre-training, post-training (via GRPO and on-policy distillation), and integration (via multi-teacher on-policy distillation) stages.