SkillOS: Learning Skill Curation for Self-Evolving Agents

Hugging Face Daily Papers Papers
reinforcement-learning llm-agents skill-curation self-evolving-agents research

Summary

This paper introduces SkillOS, a reinforcement learning framework that enables LLM agents to learn long-term skill curation policies for self-evolution, improving performance and generalization across tasks.

LLM-based agents are increasingly deployed to handle streaming tasks, yet they often remain one-off problem solvers that fail to learn from past interactions. Reusable skills distilled from experience provide a natural substrate for self-evolution, where high-quality skill curation serves as the key bottleneck. Existing approaches either rely on manual skill curation, prescribe heuristic skill operations, or train for short-horizon skill operations. However, they still struggle to learn complex long-term curation policies from indirect and delayed feedback. To tackle this challenge, we propose SkillOS, an experience-driven RL training recipe for learning skill curation in self-evolving agents. SkillOS pairs a frozen agent executor that retrieves and applies skills with a trainable skill curator that updates an external SkillRepo from accumulated experience. To provide learning signals for curation, we design composite rewards and train on grouped task streams based on skill-relevant task dependencies, where earlier trajectories update the SkillRepo, and later related tasks evaluate these updates. Across multi-turn agentic tasks and single-turn reasoning tasks, SkillOS consistently outperforms memory-free and strong memory-based baselines in both effectiveness and efficiency, with the learned skill curator generalizing across different executor backbones and task domains. Further analyses show that the learned curator produces more targeted skill use, while the skills in SkillRepo evolve into more richly structured Markdown files that encode higher-level meta-skills over time.
Original Article
View Cached Full Text

Cached at: 05/08/26, 07:26 AM

Paper page - SkillOS: Learning Skill Curation for Self-Evolving Agents

Source: https://huggingface.co/papers/2605.06614 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

SkillOS enables self-evolving LLM agents to learn complex long-term skill curation policies through reinforcement learning, improving performance across diverse tasks while generalizing across different executor architectures.

LLM-based agentsare increasingly deployed to handle streaming tasks, yet they often remain one-off problem solvers that fail to learn from past interactions. Reusable skills distilled from experience provide a natural substrate for self-evolution, where high-qualityskill curationserves as the key bottleneck. Existing approaches either rely on manualskill curation, prescribe heuristic skill operations, or train for short-horizon skill operations. However, they still struggle to learn complex long-term curation policies from indirect and delayed feedback. To tackle this challenge, we propose SkillOS, an experience-driven RL training recipe for learningskill curationinself-evolving agents. SkillOS pairs a frozenagent executorthat retrieves and applies skills with a trainable skill curator that updates an external SkillRepo from accumulated experience. To provide learning signals for curation, we designcomposite rewardsand train on groupedtask streamsbased on skill-relevant task dependencies, where earlier trajectories update the SkillRepo, and later related tasks evaluate these updates. Across multi-turn agentic tasks and single-turn reasoning tasks, SkillOS consistently outperforms memory-free and strong memory-based baselines in both effectiveness and efficiency, with the learned skill curator generalizing across different executor backbones and task domains. Further analyses show that the learned curator produces more targeted skill use, while the skills in SkillRepo evolve into more richly structured Markdown files that encode higher-levelmeta-skillsover time.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2605\.06614

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.06614 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.06614 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.06614 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

OpenSkill: Open-World Self-Evolution for LLM Agents

Hugging Face Daily Papers

OpenSkill is a framework for LLM agents to self-evolve skills and verification signals from open-world resources without target-task supervision, achieving high performance across benchmarks.

SkillForge: Co-Evolving Skills and Agents via Dynamic Skill Lifecycles

Hugging Face Daily Papers

SkillForge proposes an agentic reinforcement learning method that co-evolves an LLM agent and its skill library through a fitness-driven lifecycle of trial, active, stable, and retired states. It reports up to 7.8% relative improvement over strong baselines on interactive agent benchmarks and releases SkillFurnace, a 5k+ record dataset for studying skill quality and lifecycle management.

Self-Supervised Skill Optimization

arXiv cs.CL

This paper introduces Self-Supervised Skill Optimization (SSO), a framework that learns and optimizes reusable agent skills from unlabeled task instances using LLM-judged pairwise comparisons, without requiring ground-truth labels or rewards. SSO outperforms existing ground-truth-free prompt optimizers and approaches ground-truth-based methods on closed-ended benchmarks.