SEAL: Synergistic Co-Evolution of Agents and Learning Environments

Hugging Face Daily Papers Papers

Summary

SEAL is a closed-loop co-evolution framework for interactive tool-use agents that addresses Agent-Environment Misalignment by synchronizing policy and environment updates using on-policy trajectories and turn-level diagnosis.

Large Language Model (LLM) agents are increasingly improved through interaction, yet most self-evolution methods adapt either the policy or the learning environment in isolation. We identify this structural gap as Agent-Environment Misalignment: the agent's capability frontier changes during training, while the environment that provides supervision remains static or only weakly coupled to the agent's revealed failures. We propose SEAL, a closed-loop co-evolution framework for interactive tool-use agents. SEAL collects on-policy trajectories under executable verification, diagnoses failed rollouts into turn-level failure labels, and uses these diagnoses as a shared signal for both environment-side adaptation and model-side policy optimization. The environment evolves its training-time learning interface by exposing clearer tool affordance cues, constraint information, and recovery-oriented feedback, while the policy is updated with diagnosis-guided advantage reweighting. Extensive experiments across in-distribution and out-of-distribution multi-turn tool-use evaluations show that SEAL improves low-resource agent learning: with only 400 training samples, it yields +8.25 to +26.25 average-point gains across three backbones and exhibits positive out-of-distribution transfer. These results demonstrate the value of jointly adapting the learner and its training-time learning substrate for robust self-improving LLM agents.
Original Article
View Cached Full Text

Cached at: 05/26/26, 06:42 AM

Paper page - SEAL: Synergistic Co-Evolution of Agents and Learning Environments

Source: https://huggingface.co/papers/2605.24426 Large Language Model agents are increasingly improved through interaction rather than static supervision. Yet most self-evolution methods adapt either the policy or the learning environment in isolation. As the agent’s capability frontier shifts during training, the environment that provides supervision often remains static or only weakly coupled to the agent’s revealed failures. We call this mismatch Agent-Environment Misalignment.

SEAL is a closed-loop co-evolution framework for interactive tool-use agents. It collects on-policy trajectories under executable verification, diagnoses failed rollouts into turn-level labels, and uses these diagnoses as a shared signal for both learning-interface evolution and model-side policy optimization. With only 400 training samples, SEAL yields +8.25 to +26.25 average-point gains across three backbones and exhibits positive out-of-distribution transfer.

Similar Articles

SEAL: Synergistic Co-Evolution of Agents and Learning Environments

arXiv cs.CL

SEAL proposes a closed-loop framework for jointly evolving LLM agents and their training environments, using diagnosis-guided labels to align both sides. It achieves substantial gains in multi-turn tool-use tasks with only 400 training samples, demonstrating improved robustness and out-of-distribution transfer.

Self-Evolving Agents with Anytime-Valid Certificates

arXiv cs.AI

This paper introduces SEA, an architecture for self-evolving agents that confines self-modification to a steering adapter and versioned harness around a frozen base model, using anytime-valid gates to audit modifications against a fixed error budget. Experiments on SWE-bench Verified with four base models show that the suite provides a +4 to +5% improvement on strong base models while preventing regressions.