LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget

Hugging Face Daily Papers Papers
long-context reinforcement-learning gpu-optimization training-efficiency million-tokens large-language-models

Summary

LongStraw is an architecture-aware execution stack for million-token RL post-training under a fixed GPU budget, demonstrating execution capacity up to 2.1M positions on Qwen and GLM models.

A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment. The gap is especially important for AI agents, whose observations, tool outputs, documents, and prior decisions accumulate over long trajectories. LongStraw is an architecture-aware execution stack for million-token RL post-training under a fixed GPU budget, instantiated with Group Relative Policy Optimization (GRPO). It evaluates the shared prompt without autograd, retains only model-specific state needed by later tokens, and replays short response branches one at a time, reducing the live training graph at the cost of additional replay time. We implement it for the hybrid recurrent and full-attention Qwen3.6-27B and the compressed-attention mixture-of-experts GLM-5.2. On eight H20 GPUs, LongStraw completes grouped Qwen scoring and response backward at 2.1M positions for groups of 2 and 8; increasing the group size adds only 0.21 GB of peak allocated memory, while a separate stress test reaches 4.46M positions. On 32 H20 GPUs, we validate the end-to-end LongStraw execution path for a 2.1M-token prompt across all 78 layers of GLM-5.2. These experiments establish execution capacity rather than complete training correctness because the captured prompt state is detached and some distributed forward and gradient composition paths remain incomplete.
Original Article
View Cached Full Text

Cached at: 07/20/26, 09:42 AM

Paper page - LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget

Source: https://huggingface.co/papers/2607.14952 Published on Jul 16

#1 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

AgrowinggapseparatesinferencecontextlengthsfromRLpost-training:inferencesystemsareapproachingmillion-tokencontexts,whilepost-trainingworkloadsoftenremainat256Ktokensorbelowandrelyonlengthgeneralizationatdeployment.ThegapisespeciallyimportantforAIagents,whoseobservations,tooloutputs,documents,andpriordecisionsaccumulateoverlongtrajectories.LongStrawisanarchitecture-awareexecutionstackformillion-tokenRLpost-trainingunderafixedGPUbudget,instantiatedwithGroupRelativePolicyOptimization(GRPO).Itevaluatesthesharedpromptwithoutautograd,retainsonlymodel-specificstateneededbylatertokens,andreplaysshortresponsebranchesoneatatime,reducingthelivetraininggraphatthecostofadditionalreplaytime.Weimplementitforthehybridrecurrentandfull-attentionQwen3.6-27Bandthecompressed-attentionmixture-of-expertsGLM-5.2.OneightH20GPUs,LongStrawcompletesgroupedQwenscoringandresponsebackwardat2.1Mpositionsforgroupsof2and8;increasingthegroupsizeaddsonly0.21GBofpeakallocatedmemory,whileaseparatestresstestreaches4.46Mpositions.On32H20GPUs,wevalidatetheend-to-endLongStrawexecutionpathfora2.1M-tokenpromptacrossall78layersofGLM-5.2.Theseexperimentsestablishexecutioncapacityratherthancompletetrainingcorrectnessbecausethecapturedpromptstateisdetachedandsomedistributedforwardandgradientcompositionpathsremainincomplete.

View arXiv pageView PDFGitHub10Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.14952 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.14952 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.14952 in a Space README.md to link it from this page.

Collections including this paper4

Similar Articles