@dair_ai: An interesting idea is to continually train specialized models on skills. This paper explores that idea. They propose S…
Summary
SkillGym is a framework that converts human-written skills into training environments for LLMs, enabling fine-tuning that boosts performance on benchmarks like Terminal-Bench and SkillsBench, surpassing scores from models such as Claude Sonnet 4.6 and GPT-5.4 Mini.
View Cached Full Text
Cached at: 09/28/26, 03:39 PM
An interesting idea is to continually train specialized models on skills.
This paper explores that idea.
They propose SkillGym which turns human-written skills into 2,756 training environments with code-based checkers, then fine-tunes on 8,364 successful trajectories.
After fine-tuning, Qwen3.5-35B-A3B running in Claude Code gains 19.10 points on Terminal-Bench 2.1 and 28.13 points on skill-assisted SkillsBench v1.1, where it reaches 51.47%.
That is above the reported scores for Claude Sonnet 4.6 and GPT-5.4 Mini.
With no skills loaded, the trained model beats the base model that has the skills in context.
Paper: https://academy.dair.ai/papers/skillgym-internalizing-human-skills-into-llms-for-real-world-problem-solving-2609.27717…
SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving
Source: https://academy.dair.ai/papers/skillgym-internalizing-human-skills-into-llms-for-real-world-problem-solving-2609.27717 Agents · ReasoningChat with Paper
First page

The curator’s take
Zhilong Ge, Qin Chen and colleagues at East China Normal University present SkillGym, which converts human-written agent skills into executable, checkable training environments so the procedures end up in model weights.
Ask this paper
Question about this paper Key points01
Skill-to-task pipeline. Each skill is turned into concrete tasks with code-based outcome checkers, and contrastive runs with and without the skill measure how much the task depends on it.
02
Released resources. 2,756 environments in 12 categories and 8,364 successful trajectories from several models and harnesses, averaging 49 tool calls and over 60k logged tokens.
03
SFT gains under Claude Code. Qwen3.5-35B-A3B improves by 199 Elo on GDPval-AA v2, 19.10 points on Terminal-Bench 2.1, and 28.13 and 12.38 points on SkillsBench v1.1 with and without skills.
04
Comparison. The 35B SkillGym-Agent reaches 51.47% on skill-assisted SkillsBench, above reported scores for Claude Sonnet 4.6, GPT-5.4 Mini and DeepSeek V4 Pro.
05
Internalized procedure. Without skills it beats the skill-assisted base model under Codex and Claude Code.
AbstractHuman-written agent skills encode rich workflows for real-world problem solving, but are typically used as external inference-time instructions rather than internalized as reusable model capabilities. We introduce \texttt{SkillGym}, a framework that transforms these skills into executable, verifiable training environments for large language model agents. Its skill-to-task pipeline instantiates concrete tasks, verifies outcomes with code-based checkers, and assesses empirical skill dependence through contrastive executions. We construct and release 2,756 environments across 12 categories and collect 8,364 successful trajectories from multiple models and harnesses, averaging 49 tool calls and over 60k logged text tokens. These resources support supervised fine-tuning on verified workflows and reinforcement learning with outcome-based rewards. Under Claude Code, supervised fine-tuning improves Qwen3.5-35B-A3B by 199 Elo on GDPval-AA v2, 19.10 percentage points on Terminal-Bench 2.1, and 28.13 and 12.38 points on SkillsBench v1.1 with and without skills, respectively. Our 35B \texttt{SkillGym-Agent} reaches 51.47\% on skill-assisted SkillsBench, exceeding reported scores for Claude Sonnet 4.6, GPT-5.4 Mini, and DeepSeek V4 Pro. Without skills, it also surpasses skill-assisted bases under Codex and Claude Code, suggesting reusable procedural competence.
Similar Articles
@rohanpaul_ai: New Shanghai AI Laboratory paper finds that training on verified runs of human-written agent skills makes a model a bet…
A Shanghai AI Laboratory paper introduces SkillGym, which turns human-written agent skill files into sandboxed pass-or-fail tasks and fine-tunes models on verified runs, improving agent performance (e.g., Terminal-Bench 2.1 +19.10 points in Claude Code) even without the skill files at prompt time.
SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generation
SkillGym proposes an automatic pipeline that crawls reproducible skills from the internet, builds verifiable difficulty-controlled environments, and collects 19k verified trajectories to train skill-use agents. Fine-tuning Qwen3.5 models (2B to 122B) on these trajectories improves performance across four skill-use benchmarks, with the 9B SFT model outperforming a 397B untrained model on two of them.
SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving
SkillGym transforms human-written agent skills into executable training environments for LLMs, enabling supervised fine-tuning and reinforcement learning to enhance real-world problem-solving capabilities and performance on benchmarks.
@AlphaSignalAI: https://x.com/AlphaSignalAI/status/2069064122218717387
This article explores how AI agents can automatically write and optimize their skill files using techniques like SkillOpt from Microsoft Research, which treats skill documents as trainable state and delivers significant performance improvements. It addresses the challenge of manual skill tuning and presents frameworks like GEPA and EvoSkill as evolutionary approaches.
SkillMaster: Toward Autonomous Skill Mastery in LLM Agents
This paper introduces SkillMaster, a training framework that enables LLM agents to autonomously create, refine, and select skills through trajectory-informed review and counterfactual utility evaluation.