MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement

Hugging Face Daily Papers Papers
reinforcement-learning self-improvement omni-modal scaling agentic-rl mixture-of-experts open-source

Summary

该报告介绍了MiMo-V2.6系列全模态模型，通过扩大强化学习计算规模（大批量异步训练、多样化环境、群组智能体评分）推动模型智能走向自我改进，并开源了训练动态、RL环境与框架。

Reinforcement learning (RL) is the central training paradigm for advancing large foundation models towards self-improvement. This report introduces the MiMo-V2.6 series, an omni-modal family that pushes the frontier of model intelligence by scaling RL compute. Prior to RL, we conduct mid-training on a broad multimodal corpus to provide ample exploration space, and build a solid infrastructure on the pretrained hybrid-SWA architecture to support subsequent scale-up. We scale RL compute along three dimensions: (1) larger batches and higher throughput, with an asynchronous training that consumes 1,568 samples and 2.7-3.7B tokens per step at context lengths of up to 1M; (2) more diverse and complex environments, spanning code, general, visual, and cyber domains under a mixture of agent harnesses; and (3) more grader compute, via groupwise agentic grading that yields more accurate reward signals for long-horizon tasks and steers the model towards shorter, more token-efficient solutions. To keep training stable at scale, we freeze the MoE router and establish a multi-layer defense against reward hacking. We further build infrastructure for mixed-task agentic RL, including a unified trajectory representation, high-concurrency multi-framework rollout, decoupled control and data planes, and training-inference consistency. We open-source the training dynamics, RL environments, and RL framework to facilitate reproduction and further research on scaled RL and model self-improvement.
Original Article
View Cached Full Text

Cached at: 10/09/26, 06:19 AM

Paper page - MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement

Source: https://huggingface.co/papers/2610.11959 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Reinforcementlearning(RL)isthecentraltrainingparadigmforadvancinglargefoundationmodelstowardsself-improvement.ThisreportintroducestheMiMo-V2.6series,anomni-modalfamilythatpushesthefrontierofmodelintelligencebyscalingRLcompute.PriortoRL,weconductmid-trainingonabroadmultimodalcorpustoprovideampleexplorationspace,andbuildasolidinfrastructureonthepretrainedhybrid-SWAarchitecturetosupportsubsequentscale-up.WescaleRLcomputealongthreedimensions:(1)largerbatchesandhigherthroughput,withanasynchronoustrainingthatconsumes1,568samplesand2.7-3.7Btokensperstepatcontextlengthsofupto1M;(2)morediverseandcomplexenvironments,spanningcode,general,visual,andcyberdomainsunderamixtureofagentharnesses;and(3)moregradercompute,viagroupwiseagenticgradingthatyieldsmoreaccuraterewardsignalsforlong-horizontasksandsteersthemodeltowardsshorter,moretoken-efficientsolutions.Tokeeptrainingstableatscale,wefreezetheMoErouterandestablishamulti-layerdefenseagainstrewardhacking.Wefurtherbuildinfrastructureformixed-taskagenticRL,includingaunifiedtrajectoryrepresentation,high-concurrencymulti-frameworkrollout,decoupledcontrolanddataplanes,andtraining-inferenceconsistency.Weopen-sourcethetrainingdynamics,RLenvironments,andRLframeworktofacilitatereproductionandfurtherresearchonscaledRLandmodelself-improvement.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.11959 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.11959 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.11959 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

MiMo-V2.6 update: tool-call repetition, diagnosed & fixed.

Reddit r/LocalLLaMA

The MiMo-V2.6 model series has been updated to fix tool-call repetition issues caused by a reward blind spot in scaling reinforcement learning, using a lightweight RL teacher to reduce repetition while maintaining benchmark performance.

XiaomiMiMo/MiMo-V2.6-Flash-RL · Hugging Face

Reddit r/LocalLLaMA

MiMo-V2.6-Flash-RL is a multimodal AI model that scales reinforcement learning for self-improvement, featuring a sparse mixture-of-experts architecture with 309B total parameters and 1M token context length.

MiMo v2.6 pro

Reddit r/singularity

MiMo v2.6 pro is an AI model released with open weights, and it is suggested to have impressive performance in benchmarks.