QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents

Hugging Face Daily Papers Papers
reinforcement-learning long-horizon-agents elastic-framework gpu-efficiency llm-agents

Summary

QwenGyre is an elastic reinforcement learning framework designed to address challenges in training extreme long-horizon LLM agents, improving GPU utilization and reducing trajectory redundancy for enhanced training efficiency.

Large language model (LLM) agents increasingly undertake extreme-long (xlong) horizon tasks, where a single execution can span hours, hundreds of model--environment interactions, and nearly 1M tokens per rollout. Applying online reinforcement learning (RL) to such executions poses two fundamental challenges: (1) severe execution variance and prolonged rollout delays cause massive GPU idling; and (2) complex non-linear branching generates massive trajectory redundancy, crippling training efficiency. To address these, we presents QwenGyre, an end-to-end framework for xlong-horizon online RL. QwenGyre elastically reallocates GPUs between rollout and training without interrupting live executions, while its trajectory processor reconstructs branching histories, scores partial progress, and deduplicates redundant paths to bound training costs. Scaled to our flagship model, Qwen~3.8 2.4T, with 700K tokens per rollout, QwenGyre yields a 6.0% absolute gain on NL2RepoBench (52.5% to 58.5%) in 48 steps. Across our evaluations on diverse domains of training datasets, QwenGyre delivers up to 1.85times and 1.78times speedups over Colocate and Async, respectively.
Original Article
View Cached Full Text

Cached at: 09/29/26, 08:10 AM

Paper page - QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents

Source: https://huggingface.co/papers/2609.33848 Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

Largelanguagemodel(LLM)agentsincreasinglyundertakeextreme-long(xlong)horizontasks,whereasingleexecutioncanspanhours,hundredsofmodel--environmentinteractions,andnearly1Mtokensperrollout.Applyingonlinereinforcementlearning(RL)tosuchexecutionsposestwofundamentalchallenges:(1)severeexecutionvarianceandprolongedrolloutdelayscausemassiveGPUidling;and(2)complexnon-linearbranchinggeneratesmassivetrajectoryredundancy,cripplingtrainingefficiency.Toaddressthese,wepresentsQwenGyre,anend-to-endframeworkforxlong-horizononlineRL.QwenGyreelasticallyreallocatesGPUsbetweenrolloutandtrainingwithoutinterruptingliveexecutions,whileitstrajectoryprocessorreconstructsbranchinghistories,scorespartialprogress,anddeduplicatesredundantpathstoboundtrainingcosts.Scaledtoourflagshipmodel,Qwen~3.82.4T,with700Ktokensperrollout,QwenGyreyieldsa6.0%absolutegainonNL2RepoBench(52.5%to58.5%)in48steps.Acrossourevaluationsondiversedomainsoftrainingdatasets,QwenGyredeliversupto1.85timesand1.78timesspeedupsoverColocateandAsync,respectively.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2609\.33848

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.33848 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.33848 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.33848 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning

arXiv cs.CL

This paper proposes the LLM-as-Environment-Engineer framework, where a policy model analyzes failures to automatically redesign the training environment for reinforcement learning, and introduces MAPF-FrozenLake as a controllable testbed. The framework, using Qwen3-4B, outperforms larger models like GPT and Gemini, showing that policy learning improves the model's ability to diagnose weaknesses.