Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirements
Summary
This paper proposes Agentic ESOpt, a method using evolution strategies to enable scalable full-parameter fine-tuning of long-horizon LLM agents with minimal GPU memory requirements.
View Cached Full Text
Cached at: 08/19/26, 03:58 AM
Paper page - Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirements
Source: https://huggingface.co/papers/2608.17310
Abstract
Agentic ESOpt uses evolution strategies for scalable full-parameter fine-tuning of long-horizon LLM agents via trajectory-level reward-weighted updates and parameter-context co-evolution.
Reinforcement Learning(RL) has been promising in single-turn LLM fine-tuning. However,long-horizon agentic reasoningintroduces increasingly branching interactions and sparse rewards, exposing several limitations of RL: its heavyweight backpropagation-based training stack makes it impractical to fine-tune larger LLMs, and longer-horizon trajectories makecredit assignmentin RL substantially harder. This paper argues thatevolution strategies(ES) can be a better choice for fine-tuning long-horizon LLM agents. Compared with agentic RL, ES offers three key advantages: 1) Model Scalability: ES enablesfull-parameter optimizationwith only minimal, inference-level GPU memory, making it possible to fine-tune large LLMs. 2) Flexibility: its lightweight, black-box feedback interface makes ES fine-tuning easy to compose with prompt-space evolution (e.g., skill optimization & test-time compute); and 3) Long-Horizon Scalability: ES performs trajectory-level parameter attribution without decomposing rewards across horizons, yielding better scalability than Agentic RL as the horizon length grows. Based on this insight, we proposeAgentic ESOpt, a full-parameter agentic fine-tuning framework tailored to flexible parameter--context co-evolution. At each step,Agentic ESOptsamples perturbations around the current LLM parameters, evaluates the resulting agents with rewards, and applies an onlinereward-weighted update. To improve the exploration--adaptation trade-off,Agentic ESOptfurther introduces acosine decay scheduleof the perturbation scale σ. OnWebArena-Lite,full-parameter optimizationof Qwen-3.5-27B improves the No Skill baseline by 6.69%. In test-time automatic heuristic design,Agentic ESOptperforms online prompt--parameter co-evolution, improving its matched baseline in 28 of 36 settings.
View arXiv pageView PDFProject pageGitHub3Add to collection
Get this paper in your agent:
hf papers read 2608\.17310
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.17310 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.17310 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.17310 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Poor Man's Agentic Modeling: Simulating Large LLM-Agent Societies on a Laptop
This paper proposes a method for simulating large LLM-agent societies on a laptop by fitting low-parameter surrogate models from a few hundred queries, using a statistical-physics-based taxonomy to predict when this approximation holds. The approach is validated on EconAgent and several other simulations using DeepSeek-elicited agent behaviors.
EASy: Towards Efficient LLM-Based Agentic System
The paper proposes EASy, a trainable agentic framework that uses reinforcement learning to jointly optimize task performance and computational efficiency, introducing a milestone-plan-act workflow, dependency-aware execution graphs, and tree-structured rollout for training.
EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments
EvoArena introduces a benchmark for evaluating LLM agents in dynamic environments with progressive updates across terminal, software, and social domains, while EvoMem proposes a patch-based memory paradigm that records structured evolution; experiments show current agents achieve only 39.6% accuracy on EvoArena, and EvoMem yields average gains of 1.5% on the benchmark and improvements on GAIA and LoCoMo.
GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)
This paper introduces GenericAgent, a self-evolving LLM agent system designed to maximize context information density. It addresses long-horizon limitations through hierarchical memory, reusable SOPs, and efficient compression, achieving better performance with fewer tokens compared to leading agents.
EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimization
EnergyLens is an end-to-end framework for predictive energy-aware optimization of multi-GPU LLM inference, validated on Llama3 and Qwen3-MoE, achieving mean absolute percentage errors between 9.25% and 13.19% and revealing significant energy variation across configurations.