@modal: Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical st…
Summary
We trained a model on agentic coding tasks using fully asynchronous RL with mixed-precision, mixed-accelerator samplers, achieving training stability on par with homogeneous clusters at lower cost by applying off-policy correction.
View Cached Full Text
Cached at: 10/15/26, 06:07 AM
Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive.
So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.
Similar Articles
Small RL Controller, Large Language Model: RL-Guided Adaptive Sampling for Test-Time Scaling
This paper formulates adaptive sampling for large language models as a Markov decision process and trains a lightweight RL controller to balance correctness, latency, and computational cost, achieving improved trade-offs.
@ethantsliu: meta-rl induces agentic LLM exploration traditional RL trains LLM agents to use fixed policies that struggle with activ…
A new research paper reframes LLM agent training as a cross-episode Meta-RL problem, using critic-free policy gradients to enable in-context adaptation without gradient updates. The LAMER framework improves test-time performance by 11-19% over standard RL baselines on long-horizon tasks and generalizes better to unseen environments.
@svlevine: How do we run RL with real-time chunking (RTC)? In this work we figured out how to use a small RL policy with a large r…
A research collaboration introduces ARLI, a method to enable RL fine-tuning of Vision-Language-Action models with asynchronous inference using real-time chunking, improving robot behaviors by allowing the RL policy to observe more recent images.
@h100envy: Liquid AI's head of post-training explained how they built a small model that runs on-device under 1 GB in 20 minutes -…
Liquid AI's head of post-training explains how to build a sub-1GB on-device model in 20 minutes using LFM2.5, on-policy preference alignment, agentic RL, curriculum training, and iterative model merging, achieving tool-calling reliability that beats much larger models.
@charles_irl: Proper post-training RL, deployed broadly, is a key step towards a future where software systems quietly improve themse…
Modal announces an open-source library for reinforcement learning on its platform, addressing infrastructure challenges in post-training RL with scalable deployment.