SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents

Hugging Face Daily Papers Papers
ai-agents skill-selection reinforcement-learning credit-assignment policy-training benchmark-improvement academic-research

Summary

SkillGate addresses selector credit starvation in AI agent skill selection by using separate credit channels for execution and skill-naming tokens, improving success rates and reducing misleading skill exposure in long-horizon tasks.

Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.
Original Article
View Cached Full Text

Cached at: 08/20/26, 12:03 PM

Paper page - SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents

Source: https://huggingface.co/papers/2608.18852

Abstract

SkillGate fixes selector credit starvation in agent skill selection by separating outcome credit for execution tokens from local advantage for skill-naming tokens, improving success rates and reducing misleading skill exposure.

Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy,outcome-rewarded RLover the candidate slate, cannot teach it, for a structural reason we identify and nameselector credit starvation: under a broadcast,sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run’s own training artifacts confirms all three properties, each worsening monotonically with horizon.SkillGateremoves the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separateaction-local advantagereaching exactly the skill-naming tokens, positive only when a trajectory’s single read is the correct one. On fiveagentic benchmarksunder a 16-candidate slate,SkillGatelifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2608\.18852

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### simonlqy/SkillGate-9B Text Generation• 10B• Updatedabout 9 hours ago • 214

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.18852 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.18852 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

SKILL.state: Scalable Long-Horizon Agent Skills

arXiv cs.AI

SKILL.state introduces a runtime architecture for LLM-based agents that uses mutable execution state instead of growing conversation history, enhancing accuracy and reducing token usage in long-horizon tasks.

SkillGen: Verified Inference-Time Agent Skill Synthesis

arXiv cs.LG

This article introduces SkillGen, a multi-agent framework that synthesizes and verifies reusable inference-time skills for LLM agents by contrasting successful and failed trajectories. The method ensures skills are auditable and empirically verified for their net positive impact on agent performance.