Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills

Hugging Face Daily Papers Papers
self-evolution co-evolution reinforcement-learning skill-learning llm-training tool-use reasoning

Summary

The paper introduces Skill Self-Play (Skill-SP), a co-evolutionary framework that uses a proposer, solver, and skill controller to bridge structured verification and open-ended exploration, improving LLM performance on tool-use and reasoning benchmarks.

LLM training is shifting from manual design and annotation to interaction-driven self-evolution. However, existing self-evolutionary methods face a fundamental dilemma between task diversity and verification reliability: environment-bound methods obtain precise feedback but confine learning to narrow domains, while open-ended self-generation broadens the task space but lacks reliable verification, allowing misleading rewards to pollute the training loop. We identify agent skills as a powerful middle ground to reconcile this tension: each skill ensures deep, verifiable execution in a specific scenario, while dynamic routing across skills maintains open-ended task variety. Leveraging this insight, we introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a dynamic skill controller. Orchestrated via a reinforcement learning loop, these components co-evolve in a continuous self-play loop: the proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library. This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration. Empirical evaluations on tool-use and reasoning benchmarks demonstrate that Skill-SP, serving as a robust evolution engine, consistently pushes the performance ceiling of competent backbones while catalyzing striking turnarounds for initially misaligned models. Our code is available at https://github.com/Qwen-Applications/skill-self-play.
Original Article
View Cached Full Text

Cached at: 07/27/26, 05:40 AM

Paper page - Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills

Source: https://huggingface.co/papers/2607.22529 Published on Jul 24

#2 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

LLMtrainingisshiftingfrommanualdesignandannotationtointeraction-drivenself-evolution.However,existingself-evolutionarymethodsfaceafundamentaldilemmabetweentaskdiversityandverificationreliability:environment-boundmethodsobtainprecisefeedbackbutconfinelearningtonarrowdomains,whileopen-endedself-generationbroadensthetaskspacebutlacksreliableverification,allowingmisleadingrewardstopollutethetrainingloop.Weidentifyagentskillsasapowerfulmiddlegroundtoreconcilethistension:eachskillensuresdeep,verifiableexecutioninaspecificscenario,whiledynamicroutingacrossskillsmaintainsopen-endedtaskvariety.Leveragingthisinsight,weintroduceSkillSelf-Play(Skill-SP),aco-evolutionaryframeworkcomprisingaproposer,asolver,andadynamicskillcontroller.Orchestratedviaareinforcementlearningloop,thesecomponentsco-evolveinacontinuousself-playloop:theproposergenerateschallengingtasksconditionedondynamicallysampledskills;thesolverexplorescandidatesolutionstopushitscapabilityboundaries;andtheskillcontrollercollectsexecutionfeedbacktoupdateandexpandtheskilllibrary.Thisinteractiveco-evolutioneffectivelybridgesthegapbetweenstructuredverificationandopen-endedexploration.Empiricalevaluationsontool-useandreasoningbenchmarksdemonstratethatSkill-SP,servingasarobustevolutionengine,consistentlypushestheperformanceceilingofcompetentbackboneswhilecatalyzingstrikingturnaroundsforinitiallymisalignedmodels.Ourcodeisavailableathttps://github.com/Qwen-Applications/skill-self-play.

View arXiv pageView PDFGitHub0Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.22529 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.22529 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.22529 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

OpenSkill: Open-World Self-Evolution for LLM Agents

Hugging Face Daily Papers

OpenSkill is a framework for LLM agents to self-evolve skills and verification signals from open-world resources without target-task supervision, achieving high performance across benchmarks.

Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents

arXiv cs.AI

This paper introduces SkillMisevo-Gym and SkillMisevo-Bench to study how self-improving LLM agents can evolve unsafe skills from compromised experience, plus SafeEvolve as a mitigation wrapper. Experiments across 25 agent-method configurations show skill misevolution is widespread and can persist across sessions, though SafeEvolve reduces fresh-session harm significantly.