SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Use
Summary
SkillCoach introduces a self-evolving rubric framework that evaluates and enhances LLM agent skill-use by analyzing skill selection, following, composition, and reflection, providing process-level supervision beyond outcome-only metrics.
View Cached Full Text
Cached at: 07/03/26, 03:52 AM
Paper page - SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Use
Source: https://huggingface.co/papers/2607.01874
Abstract
SkillCoach is a self-evolving rubric framework that evaluates and improves agentic skill-use by analyzing skill selection, following, composition, and reflection processes, providing better supervision than outcome-only metrics.
Skills are becoming a reusable operational layer for LLM agents, encoding SOPs, domain rules, tool workflows, scripts, and validation routines. In realistic skill repositories, overlapping skills make reliableskill-usedifficult. Final verifier success is too coarse for both evaluation and training, since an agent may pass through trial and error while selecting distractor skills, skipping required steps, composing workflows incorrectly or omitting final checks. We introduce SkillCoach, a self-evolving rubric framework for evaluating and enhancingagentic skill-use. SkillCoach derives skill-groundedprocess rubricsfrom real rollouts and evaluates trajectories along four dimensions:skill selection,skill following,skill composition, andskill-grounded reflection. It keeps the external verifier as a separate outcome signal, allowing process quality to be distinguished from accidental task success. The evolved rubrics further serve as process supervision for selecting high-qualitytraining trajectories. Experiments show that evolved rubrics substantially improve evaluation quality, expose failures hidden by final accuracy, and provide stronger supervision signals thanoutcome-only filteringfor enhancingagentic skill-use.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2607\.01874
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.01874 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.01874 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.01874 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
SkillMaster: Toward Autonomous Skill Mastery in LLM Agents
This paper introduces SkillMaster, a training framework that enables LLM agents to autonomously create, refine, and select skills through trajectory-informed review and counterfactual utility evaluation.
SkillAdaptor: Self-Adapting Skills for LLM Agents from Trajectories
SkillAdaptor is a training-free step-level skill adaptation framework with explicit failure attribution for LLM agents, improving performance on WebShop, PinchBench, and Claw-Eval.
SkillCAT: Contrastive Assessment and Topology-Aware Skill Self-Evolution for LLM Agents
SkillCAT is a training-free framework for LLM agent skill self-evolution that addresses limitations of single-trace bias, unverified merging, and full corpus loading via three stages: Contrastive Causal Extraction, Assessment-Augmented Evolution, and Topology-Aware Task Execution, achieving up to 40.40% improvement on benchmarks.
Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents
This paper proposes a method for co-evolving evaluation metrics and skills in self-improving LLM agent systems, demonstrating that metrics can be evolved and that a co-evolution approach recovers most of the performance of a ground-truth-driven oracle across code generation, text-to-SQL, and report generation tasks.
SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks
SkillLearnBench introduces the first benchmark for evaluating continual skill learning in LLM agents across 20 real-world tasks, revealing that no method dominates and scaling LLMs does not guarantee better skills.