CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents

Hugging Face Daily Papers Papers
critique-aware-training action-level-supervision policy-optimization llm-agents reliability tool-calling

Summary

CAST is a critique-aware training framework that improves LLM agent reliability by generating structured action-level rationales from sparse outcomes, outperforming GPT-OSS-120B on tool-calling benchmarks.

Large language model (LLM) agents are increasingly deployed in long-horizon, interactive, and stateful environments. In these settings, a single wrong action, such as refunding the wrong purchase, can cause irreversible task failure and must be intercepted before execution. Such failures may not appear in every single run, but can emerge across repeated trials, making reliability across steps and trials critical. However, ensuring agentic reliability is challenging: even frontier LLMs struggle to explain why an action may be wrong, especially in long, intertwined trajectories governed by domain-specific policies. Much recent work relies on prompt-based critique agents, while optimization-based methods lack a systematic way to produce rich verification rationales for training. We address this gap with CAST, a critique-aware training framework that converts sparse task outcomes into action-level supervision for critique learning and policy optimization. CAST analyzes agent trajectories to synthesize structured rationales explaining action validity under partial observability. The resulting critique model is used to construct critique-aware training data for optimizing the policy model. Fine-tuning Qwen3-family models on dynamic tool-calling benchmarks, CAST improves reliability across domains, outperforming GPT-OSS-120B by over 10% pass^4 on Retail tasks and yielding an additional 9% improvement on Telehealth in an out-of-domain setting. These results demonstrate that critique-aware training improves the robustness of LLM agents in realistic dynamic environments.
Original Article
View Cached Full Text

Cached at: 09/01/26, 11:52 AM

Paper page - CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents

Source: https://huggingface.co/papers/2608.30147 Published on Aug 31

·

Submitted byhttps://huggingface.co/sahsaeedi

Amiron Sep 1

Abstract

CAST improves LLM agent reliability by generating structured action-level rationales from sparse outcomes to train critique and policy models.

Large language model (LLM) agents are increasingly deployed in long-horizon, interactive, and stateful environments. In these settings, a single wrong action, such as refunding the wrong purchase, can cause irreversible task failure and must be intercepted before execution. Such failures may not appear in every single run, but can emerge across repeated trials, making reliability across steps and trials critical. However, ensuring agentic reliability is challenging: even frontier LLMs struggle to explain why an action may be wrong, especially in long, intertwined trajectories governed by domain-specific policies. Much recent work relies on prompt-based critique agents, while optimization-based methods lack a systematic way to produce rich verification rationales for training. We address this gap with CAST, acritique-aware trainingframework that converts sparse task outcomes intoaction-level supervisionfor critique learning andpolicy optimization. CAST analyzes agent trajectories to synthesizestructured rationalesexplaining action validity underpartial observability. The resultingcritique modelis used to constructcritique-aware trainingdata for optimizing the policy model. Fine-tuningQwen3-family models ondynamic tool-callingbenchmarks, CAST improves reliability across domains, outperforming GPT-OSS-120B by over 10% pass^4 on Retail tasks and yielding an additional 9% improvement on Telehealth in an out-of-domain setting. These results demonstrate thatcritique-aware trainingimproves the robustness of LLM agents in realistic dynamic environments.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.30147 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.30147 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.30147 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

CAST: Game Solvers as Turn-Level Teachers for LLM Agents

arXiv cs.CL

CAST is a method that uses game solver state values to provide turn-level credit assignment signals for training LLM agents via reinforcement learning, outperforming baselines across multiple games in both in-domain and unseen-difficulty evaluations.