@dair_ai: An interesting idea is to continually train specialized models on skills. This paper explores that idea. They propose S…

X AI KOLs Timeline Papers

Summary

SkillGym is a framework that converts human-written skills into training environments for LLMs, enabling fine-tuning that boosts performance on benchmarks like Terminal-Bench and SkillsBench, surpassing scores from models such as Claude Sonnet 4.6 and GPT-5.4 Mini.

An interesting idea is to continually train specialized models on skills. This paper explores that idea. They propose SkillGym which turns human-written skills into 2,756 training environments with code-based checkers, then fine-tunes on 8,364 successful trajectories. After fine-tuning, Qwen3.5-35B-A3B running in Claude Code gains 19.10 points on Terminal-Bench 2.1 and 28.13 points on skill-assisted SkillsBench v1.1, where it reaches 51.47%. That is above the reported scores for Claude Sonnet 4.6 and GPT-5.4 Mini. With no skills loaded, the trained model beats the base model that has the skills in context. Paper: https://academy.dair.ai/papers/skillgym-internalizing-human-skills-into-llms-for-real-world-problem-solving-2609.27717…
Original Article
View Cached Full Text

Cached at: 09/28/26, 03:39 PM

An interesting idea is to continually train specialized models on skills.

This paper explores that idea.

They propose SkillGym which turns human-written skills into 2,756 training environments with code-based checkers, then fine-tunes on 8,364 successful trajectories.

After fine-tuning, Qwen3.5-35B-A3B running in Claude Code gains 19.10 points on Terminal-Bench 2.1 and 28.13 points on skill-assisted SkillsBench v1.1, where it reaches 51.47%.

That is above the reported scores for Claude Sonnet 4.6 and GPT-5.4 Mini.

With no skills loaded, the trained model beats the base model that has the skills in context.

Paper: https://academy.dair.ai/papers/skillgym-internalizing-human-skills-into-llms-for-real-world-problem-solving-2609.27717…


SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving

Source: https://academy.dair.ai/papers/skillgym-internalizing-human-skills-into-llms-for-real-world-problem-solving-2609.27717 Agents · ReasoningChat with Paper

First page

SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving

The curator’s take

Zhilong Ge, Qin Chen and colleagues at East China Normal University present SkillGym, which converts human-written agent skills into executable, checkable training environments so the procedures end up in model weights.

Ask this paper

Question about this paper Key points01

Skill-to-task pipeline. Each skill is turned into concrete tasks with code-based outcome checkers, and contrastive runs with and without the skill measure how much the task depends on it.

02

Released resources. 2,756 environments in 12 categories and 8,364 successful trajectories from several models and harnesses, averaging 49 tool calls and over 60k logged tokens.

03

SFT gains under Claude Code. Qwen3.5-35B-A3B improves by 199 Elo on GDPval-AA v2, 19.10 points on Terminal-Bench 2.1, and 28.13 and 12.38 points on SkillsBench v1.1 with and without skills.

04

Comparison. The 35B SkillGym-Agent reaches 51.47% on skill-assisted SkillsBench, above reported scores for Claude Sonnet 4.6, GPT-5.4 Mini and DeepSeek V4 Pro.

05

Internalized procedure. Without skills it beats the skill-assisted base model under Codex and Claude Code.

AbstractHuman-written agent skills encode rich workflows for real-world problem solving, but are typically used as external inference-time instructions rather than internalized as reusable model capabilities. We introduce \texttt{SkillGym}, a framework that transforms these skills into executable, verifiable training environments for large language model agents. Its skill-to-task pipeline instantiates concrete tasks, verifies outcomes with code-based checkers, and assesses empirical skill dependence through contrastive executions. We construct and release 2,756 environments across 12 categories and collect 8,364 successful trajectories from multiple models and harnesses, averaging 49 tool calls and over 60k logged text tokens. These resources support supervised fine-tuning on verified workflows and reinforcement learning with outcome-based rewards. Under Claude Code, supervised fine-tuning improves Qwen3.5-35B-A3B by 199 Elo on GDPval-AA v2, 19.10 percentage points on Terminal-Bench 2.1, and 28.13 and 12.38 points on SkillsBench v1.1 with and without skills, respectively. Our 35B \texttt{SkillGym-Agent} reaches 51.47\% on skill-assisted SkillsBench, exceeding reported scores for Claude Sonnet 4.6, GPT-5.4 Mini, and DeepSeek V4 Pro. Without skills, it also surpasses skill-assisted bases under Codex and Claude Code, suggesting reusable procedural competence.

Similar Articles

SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generation

Hugging Face Daily Papers

SkillGym proposes an automatic pipeline that crawls reproducible skills from the internet, builds verifiable difficulty-controlled environments, and collects 19k verified trajectories to train skill-use agents. Fine-tuning Qwen3.5 models (2B to 122B) on these trajectories improves performance across four skill-use benchmarks, with the 9B SFT model outperforming a 397B untrained model on two of them.

@AlphaSignalAI: https://x.com/AlphaSignalAI/status/2069064122218717387

X AI KOLs Timeline

This article explores how AI agents can automatically write and optimize their skill files using techniques like SkillOpt from Microsoft Research, which treats skill documents as trainable state and delivers significant performance improvements. It addresses the challenge of manual skill tuning and presents frameworks like GEPA and EvoSkill as evolutionary approaches.