GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment

Hugging Face Daily Papers Papers
long-context reinforcement-learning open-source rlvr multitask-alignment grpo post-training

Summary

GoLongRL presents an open-source approach for long-context reinforcement learning with diverse reward optimization through capability-oriented data construction and TMN-Reweight methodology.

We present GoLongRL, a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). Existing long-context RL methods often treat data construction as a matter of designing increasingly complex retrieval paths, leading to homogeneous task coverage and reward formulations that inadequately reflect practical long-context requirements. Our work offers two contributions. (1) Capability-oriented data construction with full open release. We openly release a dataset of 23K RLVR samples, the complete construction pipeline, and all training code. Guided by a taxonomy of long-context capabilities, the dataset spans 9 task types, each paired with its natural evaluation metric. It comprises curated open-source samples from established corpora and synthetic samples whose QA pairs are generated from real source documents such as books, academic papers, and multi-turn dialogues. Under the same vanilla GRPO setup, our dataset alone outperforms the closed-source QwenLong-L1.5 dataset. Moreover, our Qwen3-30B-A3B model trained on this data delivers long-context performance comparable to DeepSeek-R1-0528 and Qwen3-235B-A22B-Thinking-2507, suggesting that broader coverage and greater reward diversity substantially benefit long-context capability improvement. (2) TMN-Reweight for heterogeneous multitask optimization. To address optimization challenges from heterogeneous rewards, we propose TMN-Reweight, which combines task-level mean normalization for cross-task reward scale alignment with difficulty-adaptive weighting for more reliable advantage estimation. TMN-Reweight further improves average performance over vanilla GRPO, with general capabilities preserved or improved across reported evaluations.
Original Article
View Cached Full Text

Cached at: 05/20/26, 06:37 AM

Paper page - GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment

Source: https://huggingface.co/papers/2605.19577 Published on May 19

·

Submitted byhttps://huggingface.co/Suu

suuon May 20

#1 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

GoLongRL presents an open-source approach for long-context reinforcement learning with diverse reward optimization through capability-oriented data construction and TMN-Reweight methodology.

We present GoLongRL, a fully open-source, capability-orientedpost-training recipeforlong-contextreinforcement learningwithverifiable rewards(RLVR). Existinglong-contextRL methods often treat data construction as a matter of designing increasingly complex retrieval paths, leading to homogeneous task coverage and reward formulations that inadequately reflect practicallong-contextrequirements. Our work offers two contributions. (1) Capability-oriented data construction with full open release. We openly release a dataset of 23KRLVRsamples, the complete construction pipeline, and all training code. Guided by a taxonomy oflong-contextcapabilities, the dataset spans 9 task types, each paired with its natural evaluation metric. It comprises curated open-source samples from established corpora and synthetic samples whose QA pairs are generated from real source documents such as books, academic papers, and multi-turn dialogues. Under the same vanillaGRPOsetup, our dataset alone outperforms the closed-source QwenLong-L1.5 dataset. Moreover, our Qwen3-30B-A3B model trained on this data deliverslong-contextperformance comparable to DeepSeek-R1-0528 and Qwen3-235B-A22B-Thinking-2507, suggesting that broader coverage and greater reward diversity substantially benefitlong-contextcapability improvement. (2) TMN-Reweight for heterogeneous multitask optimization. To address optimization challenges from heterogeneous rewards, we propose TMN-Reweight, which combinestask-level mean normalizationfor cross-task reward scale alignment withdifficulty-adaptive weightingfor more reliableadvantage estimation. TMN-Reweight further improves average performance over vanillaGRPO, with general capabilities preserved or improved across reported evaluations.

View arXiv pageView PDFProject pageGitHub7Add to collection

Models citing this paper2

#### Kwai-Klear/GoLongRL-4B 4B• Updatedabout 1 hour ago • 4 #### Kwai-Klear/GoLongRL-30B-A3B 31B• Updatedabout 1 hour ago • 4

Datasets citing this paper1

#### Kwai-Klear/GoLongRL Viewer• Updatedabout 1 hour ago • 23k • 3

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.19577 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

Context-Aware RL for Agentic and Multimodal LLMs

Hugging Face Daily Papers

Introduces ContextRL, a reinforcement learning approach that teaches LLMs to identify which context supports an answer, achieving gains on agentic and multimodal benchmarks.

Learning Agent-Compatible Context Management for Long-Horizon Tasks

arXiv cs.AI

Introduces AdaCoM, an external LLM-based context manager for frozen agents, using reinforcement learning to improve long-horizon task performance by preserving task constraints and pruning stale content, with experiments on web search and deep research benchmarks.