How your agents can write and optimize their own skills

Why manually tweaking skill files holds your agents back, and what’s changing.

In 5 mins, learn why skill files make or break your AI agents and how automated optimizers like SkillOpt, GEPA, and EvoSkill are changing that.

The real bottleneck for deploying reliable agentic systems is no longer the core capabilities of the underlying language model. Today’s LLMs have a lot of power.

Instead, the quality of the agent hinges largely on the skills that you give them. In modern agent harnesses, a skill is a standalone .md file that acts as an agent’s operating procedure for a set of tasks.

This article is adapted from @bendee983’s AlphaSignal Sunday Deep Dive on how AI agents can write and optimize their own skill files.

It outlines the instructions, tool-use guidelines, formatting requirements, and failure-recovery logic that the agent should adhere to.

Optimizing these text files remains a slow and manual process. Developers must manually edit instructions, test them on a task suite, analyze failures, and rewrite the text. This manual loop is not scalable.

And unlike the underlying model, the skill document can’t be trained like a machine learning model. It lacks differentiable parameters, meaning you cannot calculate an exact gradient to guide updates.

The fundamental challenge of skill optimization

Tweaking textual instructions in skill files can also cause downstream problems. When you edit a markdown file to fix a brittle behavior in long-horizon Task A, you might cause a regression in Task B.

Without systematic tracking, it is nearly impossible to pinpoint the causal effect of individual textual changes.

The industry is responding to this problem by shifting from static manual prompting to system-driven automation. Engineers are now building optimization loops that treat the skill document as a trainable external state.

SkillOpt: a structured text-space optimizer

Developed by Microsoft Research, SkillOpt treats text documents exactly like neural network parameters.

It establishes an optimization pipeline that updates skills without altering underlying model weights.

The SkillOpt training pipeline operates via a structured loop:

• Rollout: The system executes a batch of tasks and records their execution trajectories.

• Evaluation: Trajectories receive a success or failure score from a verifier.

• Reflection: A separate LLM optimizer analyzes the minibatches of these trajectories to identify specific text components driving failures.

• Bounded edits: The optimizer proposes specific add, delete, or replace modifications. A textual learning-rate budget limits the scope of these edits to prevent volatile changes.

The proposed edits are tested rigorously to make sure they are actually effective. They are evaluated on a held-out validation set not seen during training. The system stores unsuccessful changes in a rejected-edit buffer to ensure stable performance gains.

This systematic process yields compact artifacts with high performance. On GPT-5.5, SkillOpt delivers a +23.5 point average improvement in direct chat and +24.8 points in a Codex loop.

It achieved the highest or tied performance across 52 evaluated model, benchmark, and harness settings. The optimized skill files remain highly efficient, holding a median length of roughly 920 tokens.

Experiments show that a skill document optimized by SkillOpt on a specific model and harness generalizes to other models and harnesses (though it is usually preferable to optimize your skill for your specific model/harness configuration to get the best performance).

GEPA

Beyond bounded edits on a single text file, other frameworks approach skill optimization through evolutionary programming and multi-agent synthesis.

GEPA (Genetic-Pareto) is an optimization framework that uses evolutionary algorithms to improve the instructions given to LLMs.

It can be applied to prompts, skills, and other text-based artifacts. When an agent carries out a task, GEPA uses an LLM (it can be the same one that powers the agent) to reflect on the reasoning trace, diagnose failures, and propose different “mutations” of the original artifact.

GEPA explores these different paths through “Pareto-based selection,” where it creates a list of top-candidates that perform well on different tasks.

It then uses this pool of candidates to sample a diverse set of winning strategies and explore more solutions that can generalize well across a wide range of inputs.

GEPA is very versatile and is compatible with DSPy, the popular framework used for optimizing LLM prompts.

EvoSkill

EvoSkill is a new framework that uses the idea of GEPA to discover and synthesize skills for multi-agent coding workflows.

EvoSkill uses the same fundamental idea as SkillOpt: an optimization loop that analyzes execution traces, finds error patterns and proposes fixes.

Like GEPA, EvoSkill keeps track of multiple skill candidates simultaneously, keeping them on separate Git branches and using a Pareto frontier to select the highest-performing variants.

To understand how this operates in practice, consider an engineering scenario where a coding agent keeps failing to handle nested pagination links when parsing an internal company API.

EvoSkill evaluates a branch on a held-out dataset. If the pagination accuracy surpasses the baseline, this version replaces the lowest-performing variant on the active Pareto frontier.

Trade-offs, costs, and practical considerations

Automated text-space optimization requires structural prerequisites. Systems like SkillOpt and EvoSkill cannot function on subjective, completely open-ended tasks.

They require a verifiable feedback signal and a clean, representative held-out evaluation dataset.

The primary trade-off rests entirely in upfront compute. Because an LLM optimizer must read and analyze voluminous token histories to diagnose failures, training a skill can be resource-intensive and costly.

But it offers you the option to save time and manual effort by spending more on AI optimization.

But it is worth noting that this cost only applies to the optimization phase. Because the final output is a standardized text block, it incurs no additional costs at the inference stage.

And since the output is a standard skill file, it doesn’t need any changes to the inference stack.

Loop engineering and self-optimizing agents

These specialized toolchains reflect a much broader architectural evolution. The development landscape is shifting from simple, linear prompting to “loop engineering.”

AlphaSignal AI@AlphaSignalAI·Jun 15 ArticleAll about loop engineering (including the pitfalls) In 5 min, you’ll learn the exact anatomy of a production-ready AI loop and how to keep it from spiraling out of control. Last week, OpenClaw creator Peter Steinberger posted a sharp directive for…52812611K

The main idea of loop engineering is to create a repeatable cycle with a well-defined and verifiable goal, and letting an LLM or AI agent repeat the task until it archives optimal performance.

This architecture creates an end-to-end self-improving loop. Instead of writing prompts, developers assemble control systems featuring precise evaluation metrics, memory storage, and exit conditions.

Production systems can then track live agent trajectories, flag recurring edge-case failures, and launch background optimization routines to update their own files safely.

The era of tweaking individual phrases in a system prompt or skill file is drawing to a close.

As AI agents become better at handling the small details, engineers will have the higher level role of designing systems and overseeing execution.

This article is adapted from @bendee983’s AlphaSignal Sunday Deep Dive on how AI agents can write and optimize their own skill files.

All source links are in the first reply. Full breakdown of recent updates + daily signals in our newsletter (link in bio).