Tag
A curated roundup of 10 open-source tools for training AI agents using reinforcement learning, covering frameworks like OpenPipe ART, verl-agent, Agent Lightning, and Unsloth, with details on their use cases and strengths.
Karpathy's critique of reward functions in RL is addressed by OpenPipe's ART framework using RULER, which allows natural language reward definitions evaluated by an LLM, replacing manual reward engineering.
A detailed tutorial on supervised fine-tuning (SFT) for training AI agents, built from scratch in pure PyTorch using Qwen3-0.6B, explaining the mechanics of next-token prediction and label masking.
This paper introduces SENTINEL, a failure-driven reinforcement learning framework for training tool-using language model agents. It uses a Controller-Proposer-Solver loop to generate targeted training tasks from failed trajectories, improving performance on benchmarks.
OpenEnv, a tool for creating agentic execution environments like terminals and browsers, is transitioning to a more open governance model with a committee including Hugging Face, Meta-PyTorch, Nvidia, and others to promote open-source agent training.
Socratic-SWE introduces a closed-loop self-evolution framework for software engineering agents that leverages historical solving traces to generate targeted repair tasks, achieving 50.40% on SWE-bench Verified after three iterations.
This paper investigates what makes interaction trajectories effective for training terminal-based AI agents, introducing the Terminal-Lego pipeline and revealing a pedagogical paradox where weaker agents can produce better training data. It finds that environment-grounded supervision, rather than teacher performance, is key for student generalization.
This paper proposes WRIT, a pipeline for synthesizing multi-turn agent training trajectories that balance write-intensive and read-heavy complexity. The method generates diverse tasks and simulations, enabling small models to achieve strong performance with reduced inference cost.
Microsoft open-sourced SkillOpt, a method that treats markdown skill files like neural network parameters to train AI agents without modifying model weights, using learning rates, validation checks, minibatches, and epochs.
A technical blog post that explains how to build agent training systems from first principles using a text-to-diagram agent as an example, covering environment definition, teacher trajectory generation, student fine-tuning, and reinforcement learning.
Trainer is a tool that lets users train AI agents by recording their screen, available on Product Hunt.
This paper introduces EnvSimBench, a benchmark for evaluating Large Language Models' ability to simulate environments for agent training. It identifies a 'state change cliff' in current LLMs and proposes a constraint-driven pipeline to reduce hallucinations and costs.
EnvScaler is an automated framework for scaling tool-interactive environments for LLM agents through programmatic synthesis, creating 191 diverse environments and 7K scenarios to improve agent performance on multi-turn, multi-tool interactions.
CoEvolve proposes an agent-data mutual evolution framework for training LLM agents through closed-loop, interaction-driven learning that adapts both the agent and its training data distribution. The method extracts feedback signals from rollout trajectories to guide LLM-based task synthesis, demonstrating significant improvements (15-19% absolute gains) across multiple Qwen models on AppWorld and BFCL benchmarks.
MindDR is a multi-agent deep research framework using a three-agent architecture (Planning, DeepSearch, Report) and a four-stage training pipeline, achieving competitive performance with ~30B-parameter models on multiple benchmarks. Developed by Li Auto and deployed as an online product, it also introduces MindDR Bench, a 500-query Chinese benchmark for evaluating deep research capabilities.
OpenResearcher presents a reproducible pipeline for training deep research agents using offline search environments and synthesized trajectories, achieving significant accuracy improvements on benchmark tasks like BrowseComp-Plus.
OpenAI Universe is a software platform that enables RL agents to interact with any existing computer program through a VNC-based interface, supporting parallel environment execution at 60 FPS with human demonstrations and automated reward extraction.