SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents

Hugging Face Daily Papers Papers
ai-agents skill-selection reinforcement-learning credit-assignment policy-training benchmark-improvement academic-research

Summary

SkillGate addresses selector credit starvation in AI agent skill selection by using separate credit channels for execution and skill-naming tokens, improving success rates and reducing misleading skill exposure in long-horizon tasks.

Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.
Original Article
View Cached Full Text

Cached at: 08/20/26, 12:03 PM

Paper page - SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents

Source: https://huggingface.co/papers/2608.18852

Abstract

SkillGate fixes selector credit starvation in agent skill selection by separating outcome credit for execution tokens from local advantage for skill-naming tokens, improving success rates and reducing misleading skill exposure.

Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy,outcome-rewarded RLover the candidate slate, cannot teach it, for a structural reason we identify and nameselector credit starvation: under a broadcast,sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run’s own training artifacts confirms all three properties, each worsening monotonically with horizon.SkillGateremoves the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separateaction-local advantagereaching exactly the skill-naming tokens, positive only when a trajectory’s single read is the correct one. On fiveagentic benchmarksunder a 16-candidate slate,SkillGatelifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2608\.18852

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### simonlqy/SkillGate-9B Text Generation• 10B• Updatedabout 9 hours ago • 214

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.18852 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.18852 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

SkillGen: Verified Inference-Time Agent Skill Synthesis

arXiv cs.LG

This article introduces SkillGen, a multi-agent framework that synthesizes and verifies reusable inference-time skills for LLM agents by contrasting successful and failed trajectories. The method ensures skills are auditable and empirically verified for their net positive impact on agent performance.

SkillCenter: A Large-Scale Source-Grounded Skill Library for Autonomous AI Agents

arXiv cs.AI

Introduces SkillCenter, the largest open skill library for AI agents with 216,938 structured, source-grounded skills across 24 domains, built using an automated pipeline with an LLM-based quality gate called SkillGate. The library combines peer-reviewed and community skills, emphasizing traceability and offline searchability.