DynaResize: Runtime GPU Reallocation for Disaggregated LLM Post-Training

# DynaResize : Runtime GPU Reallocation for Disaggregated LLM Post-Training
Source: [https://arxiv.org/html/2607.22614](https://arxiv.org/html/2607.22614)
Hanlin DuSKLP, Institute of Computing Technology, CASBeijingChinaUniversity of Chinese Academy of SciencesBeijingChina[duhanlin21@mails\.ucas\.ac\.cn](https://arxiv.org/html/2607.22614v1/mailto:[email protected])Zhiyuan YanSKLP, Institute of Computing Technology, CASBeijingChina[yanzhiyuan@ict\.ac\.cn](https://arxiv.org/html/2607.22614v1/mailto:[email protected]),Haiquan ChenBytedanceBeijingChina[chenhaiquan@bytedance\.com](https://arxiv.org/html/2607.22614v1/mailto:[email protected]),Jiarui FangBytedanceBeijingChina[wangjerry@bytedance\.com](https://arxiv.org/html/2607.22614v1/mailto:[email protected]),Yungang BaoSKLP, Institute of Computing Technology, CASBeijingChinaUniversity of Chinese Academy of SciencesBeijingChina[baoyg@ict\.ac\.cn](https://arxiv.org/html/2607.22614v1/mailto:[email protected])andSa WangSKLP, Institute of Computing Technology, CASBeijingChinaUniversity of Chinese Academy of SciencesBeijingChina[wangsa@ict\.ac\.cn](https://arxiv.org/html/2607.22614v1/mailto:[email protected])

###### Abstract\.

RL\-based LLM post\-training increasingly disaggregates Rollout and Training across separate GPU resources, but static GPU partitioning suffers from severe pipeline bubbles under long\-tail rollout latency\. We presentDynaResize, a runtime GPU reallocation system that dynamically switches GPUs between Rollout and Training to balance stage execution times without changing RL semantics\.DynaResizedecomposes resizing into fine\-grained operations and removes non\-startup\-critical work from the critical path through communicator reuse, bounded state staging, and hysteresis\-based resizing\. Experimental results show thatDynaResizecan improve end\-to\-end throughput by 66\.5% and reduce total execution time by 33% over the optimal static configuration, while hiding 27% of role\-switching overhead\.

## 1\.Introduction

Large Language Model \(LLM\) post\-training has become a critical phase in modern AI infrastructure\(Team,[2025](https://arxiv.org/html/2607.22614#bib.bib29); DeepSeek\-AI,[2026](https://arxiv.org/html/2607.22614#bib.bib28); Dubeyet al\.,[2024](https://arxiv.org/html/2607.22614#bib.bib30)\)\. Unlike standard pre\-training, reinforcement learning \(RL\) based post\-training follows an iterative two\-stage workflow, whereRolloutgenerates trajectories using the current policy andTrainingupdates model parameters based on the collected experience\(Ouyanget al\.,[2022](https://arxiv.org/html/2607.22614#bib.bib12); Schulmanet al\.,[2017](https://arxiv.org/html/2607.22614#bib.bib14); Caiet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib16); Yu and others,[2025](https://arxiv.org/html/2607.22614#bib.bib15); Rafailovet al\.,[2023](https://arxiv.org/html/2607.22614#bib.bib13)\)\. A straightforward sequential execution of these two stages leads to poor pipeline efficiency, as rollout and training cannot make progress concurrently\. To address this limitation, modern distributed frameworks such as veRL\(Shenget al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib5); verl community,[2026](https://arxiv.org/html/2607.22614#bib.bib1)\), slime\(Zhuet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib4)\), RLinf\(Yuet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib7); Community,[2026](https://arxiv.org/html/2607.22614#bib.bib3)\), and AReaL\(Meiet al\.,[2024](https://arxiv.org/html/2607.22614#bib.bib6); Fuet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib2)\)decouple Rollout and Training through disaggregated deployment and coordinated scheduling, enabling the two stages to overlap across different workers and thereby improving concurrency and end\-to\-end performance\.

However, this disaggregated paradigm remains fundamentally vulnerable to unpredictable Rollout latencies\(Kwonet al\.,[2023](https://arxiv.org/html/2607.22614#bib.bib22); Yuet al\.,[2022](https://arxiv.org/html/2607.22614#bib.bib21); Guldoganet al\.,[2024](https://arxiv.org/html/2607.22614#bib.bib20)\)\. Since generation length varies significantly with prompt complexity, Rollout often exhibits severe long\-tail behavior, leaving downstream Training stalled and degrading end\-to\-end system efficiency\. Recent efforts seek to hide this inefficiency throughalgorithmiccompromises, such as truncating rollout lengths\(Teamet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib31); Zhouet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib17); Fanet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib8)\)or tolerating stale policy updates\(Hanet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib9)\)\. Yet these workarounds come at the cost of deviating from the original reinforcement learning semantics, potentially compromising the final model accuracy\.

In this paper, we ask the question:Can long\-tail bubbles in disaggregated post\-training be mitigated through a system\-level solution?Specifically, we explore whether GPUs can be dynamically reassigned between Rollout and Training at runtime, allowing resources to follow the shifting bottleneck and reduce pipeline stalls caused by long\-tail Rollout latencies\. However, making this idea practical is far from straightforward\. Dynamic resizing may incur substantial overheads from worker reconfiguration, state migration, and runtime coordination, potentially diminishing or even outweighing its performance benefits\. Moreover, existing distributed frameworks provide no direct support for dynamic GPU reallocation between Rollout and Training\. If implemented naively, such reallocation would introduce three fundamental challenges\.

❶Topology Reconstruction\.Switching the role of a GPU requires destroying and rebuilding the global communication topology\. This monolithic barrier synchronization across nodes incurs pipeline stalls lasting minutes\. ❷Parameter Migration Overflow\.Dynamic reconfiguration requires the target GPU to flush old parameters and ingest new weights and optimizer states\. Under tight GPU memory constraints, this in\-place context switch may cause severe transient memory amplification, triggering Out\-of\-Memory \(OOM\) crashes\. ❸Scheduling Thrashing\.LLM generation length exhibits extreme token\-level variance\. If the system reacts passively to instantaneous load imbalances, it suffers from severethrashing\. In this state, GPUs spend more time on context\-switching than on useful computation\. This overhead completely negates the theoretical benefits of dynamic allocation\. Consequently, prior systems are forced to settle for the static paradigm\. Overcoming these systemic barriers demands a fundamental redesign of the resource scaling mechanism\.

To this end, we proposeDynaResize, a dynamic resource\-scaling system that enables low\-overhead GPU role switching between Rollout and Training on the fly for disaggregated post\-training\. The key idea ofDynaResizeis to revisit the resource resizing process by decomposing it into fine\-grained operations and analyzing their execution dependencies\. Rather than performing resizing as a monolithic stop\-and\-wait procedure,DynaResizedetermines which operations can run in parallel, which can be prewarmed ahead of topology activation, and which can be deferred until they are actually needed\. It then moves expensive but non\-startup\-critical operations, such as communicator preparation and optimizer\-state reloading, off the critical path\. In this way, dynamic GPU resizing becomes a lightweight runtime operation instead of a disruptive global reconfiguration\.

Building on this core idea,DynaResizefurther implements three mechanisms to address the key challenges of dynamic GPU reallocation:communicator reuse,state staging, andhysteresis resizing\. Communicator reuse reduces topology switching overhead by caching topology\-scoped communication metadata and prewarming inter\-role communicators off the critical path\. State staging minimizes state migration latency by streaming model weights and optimizer states through bounded host\-memory buffers, while deferring non\-critical optimizer\-state reloading until it is actually needed\. Hysteresis resizing stabilizes GPU reallocation decisions by triggering resizing only under sustained workload imbalance rather than transient latency fluctuations\.

Crucially,DynaResizedoes not reduce individual rollout latency\. Instead, it improves cluster\-wide utilization by rebalancing GPUs between Rollout and Training at runtime\. It is also orthogonal to prior algorithmic methods and can be composed with them for extreme long\-tail workloads\.

Evaluations demonstrate that compared to the optimal static configuration,DynaResizepromptly rectifies systemic imbalances at runtime\. By fluidly scaling resource partitions,DynaResizeimproves end\-to\-end post\-training throughput by 66\.5% and shortens the total execution time by 33% \(normalized to a 100\-step baseline\), while reducing resizing overhead by 27% compared to rigid refactoring\.

## 2\.Background and Motivation

Modern LLM post\-training relies on asynchronous pipelines to physically decouple the rollout stage from the training stage\. While this separation mitigates strict synchronization overhead, it exposes a critical flaw under static resource provisioning: the highly variable generation lengths lead to severe workload imbalances\. As illustrated in Figure[1](https://arxiv.org/html/2607.22614#S2.F1)\(a\), under a naive one\-step\-off\-policy asynchronous execution, rollout stragglers induce massive pipeline bubbles, leaving allocated training GPUs idling\. Furthermore, the execution time of rollout steps often significantly exceeds that of training steps \(especially for complex agentic or reasoning tasks\), exacerbating these idle periods\.

![Refer to caption](https://arxiv.org/html/2607.22614v1/x1.png)Figure 1\.Comparison of approaches for mitigating asynchronous training\-rollout imbalanceComparison of one\-step\-off\-policy and its 3 algo solutions, and our workTo alleviate this hardware under\-utilization, recent algorithmic and scheduling modifications navigate various trade\-offs \(Figure[1](https://arxiv.org/html/2607.22614#S2.F1)\(b\)\-\(d\)\)\.

- •Partial Rollout & Truncation\. \(Fig\.[1](https://arxiv.org/html/2607.22614#S2.F1)\(b\)\)Approaches like Kimi\(Teamet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib31)\)and TPPO\(Fanet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib8)\)enforce time alignment by interrupting rollouts, but this compromises the integrity of long\-form reasoning \(e\.g\., Chain\-of\-Thought\)\.
- •Fully Asynchronous RL\. \(Fig\.[1](https://arxiv.org/html/2607.22614#S2.F1)\(c\)\)Frameworks such as AsyncFlow\(Hanet al\.,[2025](https://arxiv.org/html/2607.22614#bib.bib9)\)maximize hardware overlap by decoupling stages entirely; however, this inherently introduces severe policy staleness, which risks convergence and requires careful algorithmic compensation\.
- •Advanced Repacking\. \(Fig\.[1](https://arxiv.org/html/2607.22614#S2.F1)\(d\)\)Strategies like Laminar\(Shenget al\.,[2026](https://arxiv.org/html/2607.22614#bib.bib11)\)attempt to mask latency by tightly packing requests, yet they still struggle to fully eliminate inter\-rank bubbles when intra\-batch variance is extreme\.

Moreover, methods attempting to mitigate variance by rearranging samples via curriculum\-learning\-based length prediction suffer from poor prediction accuracy and degrade the model’s long\-context capabilities\(Zhanget al\.,[2026](https://arxiv.org/html/2607.22614#bib.bib19); Gaoet al\.,[2026a](https://arxiv.org/html/2607.22614#bib.bib18)\)\. From an infrastructure perspective, recent elastic frameworks scale rollout ranks by harvesting external GPUs\(Gaoet al\.,[2026b](https://arxiv.org/html/2607.22614#bib.bib32); Xiaoet al\.,[2026](https://arxiv.org/html/2607.22614#bib.bib33); Zhang and others,[2026](https://arxiv.org/html/2607.22614#bib.bib34)\)\. Yet, relying on extra hardware solely for the rollout phase lacks the system\-level control to balance the train\-rollout loop, and fails to eliminate idle waste during pipeline stalls as well\(Or and others,[2020](https://arxiv.org/html/2607.22614#bib.bib43); Penget al\.,[2018](https://arxiv.org/html/2607.22614#bib.bib40); Thorpeet al\.,[2023](https://arxiv.org/html/2607.22614#bib.bib41); Janget al\.,[2023](https://arxiv.org/html/2607.22614#bib.bib42)\)\.

We argue that pipeline imbalance is primarily a physical resource mismatch rather than an algorithmic flaw\. Therefore, a natural system\-level approach is to dynamically reallocate resources between Training and Rollout, as shown in Figure[1](https://arxiv.org/html/2607.22614#S2.F1)\(e\)\. In particular, addressing this imbalance requires an intra\-cluster closed\-loop mechanism that can convert existing GPUs between the two stages on demand, reducing hardware idling during pipeline stalls while preserving the original algorithmic semantics\.

## 3\.DynaResize’s Design

![Refer to caption](https://arxiv.org/html/2607.22614v1/x2.png)Figure 2\.Overview ofDynaResizeThis fig shows the design of DynaResize , which contains scheduler, abstraction layer, re\-init module, comm & data store manager### 3\.1\.Overview

As illustrated in Fig\.[2](https://arxiv.org/html/2607.22614#S3.F2),DynaResizeadopts a decoupled control\-data plane architecture centered around an abstraction layer that isolates physical GPUs from logical roles, allowing on\-the\-fly computational reallocation\. On the control plane, a load\-aware monitor and a topology controller work in tandem to track pipeline bubbles and conservatively issue resize commands, which is detailed in §[3\.4](https://arxiv.org/html/2607.22614#S3.SS4)\.

![Refer to caption](https://arxiv.org/html/2607.22614v1/x3.png)Figure 3\.Fully\-overlapping workflow ofDynaResizecompared to hard stop\-and\-rebuild resizingworkflow of DynaResize and hard resize, showing overlappingFig\.[3](https://arxiv.org/html/2607.22614#S3.F3)highlights the resulting paradigm shift by contrasting the execution timelines of a traditional hard\-resize against our non\-blockingDynaResizeworkflow\. To fundamentally resolve the prohibitive overhead of the resizing process,DynaResizeabandons the monolithic,stop\-and\-rebuildlifecycle\. Instead, it pipelines role\-switching under the centralized orchestration of theFully\-overlapping Re\-init Module\. This manager concurrently overlaps most parallelizable operations not logically constrained and deferred parameter transfers with the ongoing computation\. To achieve this massive overlapping, it governs two subordinate data\-plane strategies:\(1\) Topology Reconstruction \(§[3\.2](https://arxiv.org/html/2607.22614#S3.SS2)\):The Communicator Manager pre\-warms communication metadata off the critical path to reduce rebuilding barriers\.\(2\) Parameter Migration \(§[3\.3](https://arxiv.org/html/2607.22614#S3.SS3)\):The Param Staging Manager safely pipelines weights and optimizer states via a chunked streaming mechanism to prevent memory crashes\.

### 3\.2\.Topology\-Aware Communicator Reuse

Problem\.Communication reconstruction is tightly coupled with the worker lifecycle\. In existing distributed runtimes, communication groups cannot be fully established until the corresponding workers have been initialized and their ranks, devices, and runtime contexts become available\(Paszkeet al\.,[2019](https://arxiv.org/html/2607.22614#bib.bib23); NVIDIA Corporation,[2026](https://arxiv.org/html/2607.22614#bib.bib24)\)\. As a result, each topology switch may require rebuilding inter\-role communication groups on the critical path before weight synchronization can proceed\. This creates a rigid reconfiguration overhead\. Although the cost cannot be completely eliminated due to the dependency on worker initialization, leaving it entirely on the switching path would significantly delay GPU role transitions\.

Solution\.DynaResizemitigates this overhead by separating topology\-scoped communication metadata from worker\-bound runtime states\. For each candidate topology,DynaResizecaches the metadata required to construct inter\-role communication groups, such as rank mappings and group membership\. Once the corresponding workers become available,DynaResizeprewarms the required communicators off the critical path before the new topology is published\. In this way, topology switching no longer rebuilds communication groups from scratch\. Instead, the actual switch becomes a lightweight communicator binding and activation step, which accelerates subsequent weight synchronization\.

Crucially, our communicator reuse is conservative by design\. It applies specifically to topology\-scoped inter\-role synchronization groups for model weights\. We do not claim full reuse of worker\-bound distributed runtime states \(such as the default process group or device\-mesh bindings\), as complete runtime communication reuse would require a persistent refactoring of workers, violating the modular structure of existing frameworks\. This approach recovers substantial setup costs across revisited topologies while preserving framework correctness\.

### 3\.3\.Staged State Reloading

Problem\.Elastic reconfiguration requires migrating the training state across different GPU topologies\. During resizing, training ranks must preserve the current model weights and optimizer states, and then reload the states required by the new topology\(Shoeybiet al\.,[2019](https://arxiv.org/html/2607.22614#bib.bib25); Zhaoet al\.,[2023](https://arxiv.org/html/2607.22614#bib.bib26)\)\. A naive implementation would follow a storage\-based checkpointing path: it first flushes the existing device states to external storage, releases the old workers, and then reloads the states for the new workers\. This stop\-and\-reload procedure introduces a long blocking period at the step boundary\. Since optimizer states are often several times larger than model weights, full\-state reconstruction can severely stall subsequent training steps\.

Solution\.DynaResizeavoids this storage\-based path by using host memory as a fast staging area for state migration\.DynaResizeutilizes CPU\-offloading method\(Rajbhandari and others,[2020](https://arxiv.org/html/2607.22614#bib.bib36); Renet al\.,[2021](https://arxiv.org/html/2607.22614#bib.bib37); Rajbhandariet al\.,[2021](https://arxiv.org/html/2607.22614#bib.bib38); Shenget al\.,[2023](https://arxiv.org/html/2607.22614#bib.bib39)\), directly transferring weights and optimizer states from old training ranks to new training ranks through host memory\. However, naively staging the entire state in host memory can easily cause out\-of\-memory failures\. To keep memory usage bounded,DynaResizepartitions the states into fine\-grained chunks and streams them through host\-memory buffers\. The store and reload operations are then pipelined chunk by chunk\. This design avoids full\-state accumulation in host memory, while still preserving the speed advantage of in\-memory transfer over storage\-based checkpointing\.

Crucially, the core efficiency ofDynaResizegains originate from maximizing the asynchronous overlapping of independent operations and enforcing an on\-demand optimizer reload\. Considering that optimizer states are several times larger than model weights and are only required at the end of a training step, we design an on\-demand deferred transfer mechanism\. Only the model weights are loaded during the immediate transition phase to resume step execution, while the loading and materialization of the massive optimizer states are deferred until the first step on the new topology\. This strategic deferral successfully absorbs the heaviest data\-transfer overhead from the critical path of the step boundary, enabling low\-overhead topology transitions under strict resource constraints\.

### 3\.4\.Hysteresis\-Guided Reconfiguration

Problem\.Resizing decisions are vulnerable to short\-term workload noise\. In RL post\-training, Rollout and Training costs can vary significantly across batches, causing transient throughput fluctuations\. A naive controller that reacts to every measurement may frequently switch GPU allocations back and forth\. Such reconfiguration thrashing can introduce repeated resizing overheads and may even outweigh the load\-balancing benefits of dynamic reallocation\.

Solution\.DynaResizetreats resizing as a stabilized decision problem over a discrete set of candidate topologies\. Instead of using each measurement as an immediate trigger,DynaResizefilters resizing decisions through hysteresis thresholds, dwell\-time constraints, cooldown intervals, and consecutive\-signal requirements\. As a result, resizing is triggered only by sustained directional imbalance rather than transient fluctuations\. The controller does not perform unconstrained online topology search\. It only decides when to transition among pre\-specified elastic configurations, ensuring that each resizing action can bring positive throughput gains\.

## 4\.Implementation

We implementDynaResizeatop veRL\(verl community,[2026](https://arxiv.org/html/2607.22614#bib.bib1)\), a state\-of\-the\-art LLM post\-training framework, which uses Ray\(Moritzet al\.,[2018](https://arxiv.org/html/2607.22614#bib.bib27)\)to statically provision GPU actor groups for training and rollout\. We embed ourDynaResizeengine directly into this worker lifecycle\. Instead of triggering Ray’s heavy actor teardown during a resize,DynaResizeretains the underlying placement groups\. By leveraging host\-memory staging and communicator caching,DynaResizedynamically migrates model states and transitions execution roles within existing processes, enabling heavyweight resizing operations to be executed asynchronously and fully overlapped\.

## 5\.Evaluation

Our evaluation is designed to answer three questions regarding the performance and overhead ofDynaResize\.

- •How much performance improvement canDynaResizeachieve compared to optimal static configurations?\(Exp\#1\)
- •How much reconfiguration latency canDynaResizehide compared to a naivestop\-and\-rebuildbaseline?\(Exp\#2\)
- •How many training steps are required to amortize the cost of a single resize operation, and does the hysteresis controller successfully prevent thrashing?\(Exp\#3\)

![Refer to caption](https://arxiv.org/html/2607.22614v1/x4.png)\(a\)Avg step time\.
![Refer to caption](https://arxiv.org/html/2607.22614v1/x5.png)\(b\)Time per step\.

Figure 4\.\(Exp\#1\)Time per step saved by dynamic resizing\.Exp for throughput speedup, about 1\.66x![Refer to caption](https://arxiv.org/html/2607.22614v1/x6.png)Figure 5\.\(Exp\#2\)Time breakdown forDynaResizecomparing to hard resize without on\-demand reloading\.Exp that profiles resize cost![Refer to caption](https://arxiv.org/html/2607.22614v1/x7.png)Figure 6\.\(Exp\#2\)Time flow ofDynaResizecaching and overlapping\.Exp compares breakdown cost of DynaResize with hard resizeExperimental Setup\.We evaluateDynaResizeon a GPU server with 8×\\timesNVIDIA H20 GPUs connected via NVLink and PCIe Gen5\. We use Qwen3\-8B as the training model\. To simulate real\-world post\-training imbalances, we utilize a variance prompt dataset by mixing short\-form QA tasks with long\-form reasoning and coding tasks, deliberately inducing severe generation long\-tail stragglers\.

\(Exp\#1\) End\-to\-End Performance\.We first evaluate end\-to\-end performance under different degrees of workload variance, measured by the execution time per PPO step\. Under high prompt variance,DynaResizeimproves end\-to\-end throughput by 66\.5% and reduces the total execution time by 33% compared with the optimal static baseline, as shown in Figure[4](https://arxiv.org/html/2607.22614#S5.F4)\(a\)\. These results are reported over a normalized 100\-step run, while practical post\-training jobs typically execute 500–5000 steps\. This performance gain is primarily achieved by aligning the execution times of Rollout and Training within each PPO step\.DynaResizemitigates imbalance by dynamically reallocating GPUs at runtime \(e\.g\., shifting the Train\-to\-Rollout ratio from 6:2 to 4:4 to match the current bottleneck\)\. As detailed in Figure[4](https://arxiv.org/html/2607.22614#S5.F4)\(b\), at the point of reallocation \(Step 10\), the step time temporarily spikes\. This prolonged execution is caused by the overhead of a ”mixed step” during resizing \(e\.g\., executing 2 rollouts and 1 train\)\. However, immediately after this resize step, the system stabilizes into an ideal state with the average step time drastically reduced \(from 39\.84s down to 23\.93s\)\. This effectively eliminates Rollout\-induced pipeline bubbles and maximizes global GPU utilization\.

Answer\#1:DynaResizeimproves throughput by 66\.5% by aligning Rollout and Training times through runtime GPU reallocation\.

\(Exp\#2\) Resizing Breakdown\.We next analyze the latency breakdown of a single GPU role transition, such as switching GPUs from Training to Rollout\. We compareDynaResizewith a naivestop\-and\-rebuildresizing baseline, which performs role switching without our critical\-path optimizations\.

Figures[5](https://arxiv.org/html/2607.22614#S5.F5)and[6](https://arxiv.org/html/2607.22614#S5.F6)show the detailed latency breakdown of the reconfiguration phase\. Under the naive baseline, each role switch incurs a monolithic blocking period of 255 seconds\. This delay is mainly caused by Ray actor destruction, device\-state flushing, and the blocking execution ofdist\.init\_process\_groupfor rebuilding the communication topology\. By overlapping in parallel as much as possible,DynaResizereduces this blocking cost to 187 seconds by moving non\-startup\-critical operations off the critical path\.

Answer\#2:Although role switching still involves unavoidable synchronous operations,DynaResizereduces the critical\-path blocking time from 255 seconds to 187 seconds, hiding 27% of the overhead through background execution\.

\(Exp\#3\) Cost Amortization and Thrashing Boundaries\.We further analyze whether the benefit ofDynaResizecan reliably outweigh its reconfiguration cost, and whether the resizing policy remains stable under transient workload noise\.

Cost amortization\.DynaResizebecomes beneficial once the saved pipeline idle time offsets the critical\-path switching latency\. Our evaluation identifies a break\-even point of approximately 11 sustained steps\. This brief amortization window ensures positive throughput gains in following steady steps whenever train\-rollout imbalances persist beyond a short period\.

Thrashing boundaries\.Unconstrained resizing risks reacting to short\-term fluctuations, causing reconfiguration thrashing that negates load\-balancing benefits\. To prevent this, our Hysteresis\-Guided controller enforces a ¿10\-step cooldown interval, safely exceeding the amortization threshold\. This efficiently filters high\-frequency noise, triggering migrations exclusively during sustained directional imbalances to guarantee strictly positive gains

Answer\#3:DynaResizeamortizes the cost of a single resize operation within∼\\sim11 sustained training steps\. The hysteresis controller can prevent thrashing\.

## 6\.Conclusion

DynaResizeenables runtime dynamic GPU resizing for RL\-based LLM post\-training, reallocating GPUs between Rollout and Training to mitigate pipeline bubbles without algorithmic approximations\. Although pre\-warming and on\-demand loading only moderately reduce the critical\-path resizing latency, this reduction is sufficient to yield substantial end\-to\-end gains in pipeline\-stall\-dominated post\-training workloads\. In our evaluation, hiding 27% of role\-switching overhead enablesDynaResizeto improve end\-to\-end throughput by 66\.5% over the optimal static configuration\. These results suggest that runtime resource reallocation is a promising system\-level direction for improving post\-training efficiency, while preserving RL semantics and algorithmic effectiveness\.

## References

- Y\. Cai, S\. Cai,et al\.\(2025\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- R\. Community \(2026\)RLinf: reinforcement learning infrastructure for embodied and agentic ai\.Note:https://github\.com/RLinf/RLinfCited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- DeepSeek\-AI \(2026\)DeepSeek\-v4: towards highly efficient million\-token context intelligence\.Note:https://huggingface\.co/deepseek\-ai/DeepSeek\-V4\-ProCited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Yang, A\. Fan,et al\.\(2024\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- T\. Fan, L\. Liu, Y\. Yue, J\. Chen, C\. Wang, Q\. Yu, C\. Zhang, Z\. Lin, R\. Zhu, Y\. Yuan, X\. Zuo, B\. Ma, M\. Zhang, G\. Liu, R\. Zhang, H\. Zhou, C\. Xie, R\. Zhu, Z\. Zhang, X\. Liu, M\. Wang, L\. Yan, and Y\. Wu \(2025\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p2.1),[1st item](https://arxiv.org/html/2607.22614#S2.I1.i1.p1.1)\.
- W\. Fu, J\. Gao, X\. Shen, C\. Zhu, Z\. Mei, C\. He, S\. Xu, G\. Wei, J\. Mei, J\. Wang, T\. Yang, B\. Yuan, and Y\. Wu \(2025\)AReaL: a large\-scale asynchronous reinforcement learning system for language reasoning\.External Links:2505\.24298,[Link](https://arxiv.org/abs/2505.24298)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- W\. Gao, Y\. Zhao, D\. An, T\. Wu, L\. Cao, S\. Xiong, J\. Huang, W\. Wang, S\. Yang, W\. Su, J\. Wang, L\. Qu, B\. Zheng, and W\. Wang \(2026a\)RollPacker: taming Long\-Tail rollouts for RL Post\-Training with tail batching\.In23rd USENIX Symposium on Networked Systems Design and Implementation \(NSDI 26\),Renton, WA,pp\. 849–866\.External Links:ISBN 978\-1\-939133\-54\-0,[Link](https://www.usenix.org/conference/nsdi26/presentation/gao-wei)Cited by:[§2](https://arxiv.org/html/2607.22614#S2.p3.1)\.
- W\. Gao, Y\. Zhao, D\. Muhtar, D\. An, X\. Shang, T\. Wu, L\. Cao, S\. Xiong, W\. Wang, J\. Huang, T\. Ma, S\. Yang, J\. Wang, L\. Qu, B\. Zheng, and W\. Wang \(2026b\)Cited by:[§2](https://arxiv.org/html/2607.22614#S2.p3.1)\.
- O\. Guldogan, J\. Kunde, K\. Lee, and R\. Pedarsani \(2024\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p2.1)\.
- Z\. Han, A\. You, H\. Wang, K\. Luo, G\. Yang, W\. Shi, M\. Chen, S\. Zhang, Z\. Lan, C\. Deng, H\. Ji, W\. Liu, Y\. Huang, Y\. Zhang, C\. Pan, J\. Wang, X\. Huang, C\. Li, and J\. Wu \(2025\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p2.1),[2nd item](https://arxiv.org/html/2607.22614#S2.I1.i2.p1.1)\.
- I\. Jang, Z\. Yang, Z\. Zhang, X\. Jin, and M\. Chowdhury \(2023\)Oobleck: resilient distributed training of large models using pipeline templates\.InProceedings of the 29th Symposium on Operating Systems Principles,SOSP ’23,New York, NY, USA,pp\. 382–395\.External Links:ISBN 9798400702297,[Link](https://doi.org/10.1145/3600006.3613152),[Document](https://dx.doi.org/10.1145/3600006.3613152)Cited by:[§2](https://arxiv.org/html/2607.22614#S2.p3.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p2.1)\.
- Z\. Mei, W\. Fu, K\. Li, G\. Wang, H\. Zhang, and Y\. Wu \(2024\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- P\. Moritz, R\. Nishihara, S\. Wang, A\. Tumanov, R\. Liaw, E\. Liang, M\. Elibol, Z\. Yang, W\. Paul, M\. I\. Jordan, and I\. Stoica \(2018\)Ray: a distributed framework for emerging AI applications\.In13th USENIX Symposium on Operating Systems Design and Implementation \(OSDI 18\),Carlsbad, CA,pp\. 561–577\.External Links:ISBN 978\-1\-939133\-08\-3,[Link](https://www.usenix.org/conference/osdi18/presentation/moritz)Cited by:[§4](https://arxiv.org/html/2607.22614#S4.p1.1)\.
- NVIDIA Corporation \(2026\)NVIDIA collective communication library \(nccl\)\.Note:[https://github\.com/NVIDIA/nccl](https://github.com/NVIDIA/nccl)Optimized primitives for collective multi\-GPU communicationCited by:[§3\.2](https://arxiv.org/html/2607.22614#S3.SS2.p1.1)\.
- A\. Oret al\.\(2020\)TorchElastic: elastic and fault\-tolerant distributed training\.Note:PyTorch BlogCited by:[§2](https://arxiv.org/html/2607.22614#S2.p3.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. F\. Christiano, J\. Leike, and R\. Lowe \(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),Vol\.35,pp\. 27730–27744\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/file/b1efde53be364a73914f58805a001731-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- A\. Paszke, S\. Gross, F\. Massa, A\. Lerer, J\. Bradbury, G\. Chanan, T\. Killeen, Z\. Lin, N\. Gimelshein, L\. Antiga, A\. Desmaison, A\. Kopf, E\. Yang, Z\. DeVito, M\. Raison, A\. Tejani, S\. Chilamkurthy, B\. Steiner, L\. Fang, J\. Bai, and S\. Chintala \(2019\)PyTorch: an imperative style, high\-performance deep learning library\.InAdvances in Neural Information Processing Systems,H\. Wallach, H\. Larochelle, A\. Beygelzimer, F\. d'Alché\-Buc, E\. Fox, and R\. Garnett \(Eds\.\),Vol\.32,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf)Cited by:[§3\.2](https://arxiv.org/html/2607.22614#S3.SS2.p1.1)\.
- Y\. Peng, Y\. Bao, Y\. Chen, C\. Wu, and C\. Guo \(2018\)Optimus: an efficient dynamic resource scheduler for deep learning clusters\.InProceedings of the Thirteenth EuroSys Conference,EuroSys ’18,New York, NY, USA\.External Links:ISBN 9781450355841,[Link](https://doi.org/10.1145/3190508.3190517),[Document](https://dx.doi.org/10.1145/3190508.3190517)Cited by:[§2](https://arxiv.org/html/2607.22614#S2.p3.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 53728–53741\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/a85b405ed65c6477a4fe8302b5e06ce7-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- S\. Rajbhandariet al\.\(2020\)ZeRO: memory optimizations toward training trillion parameter models\.InProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis \(SC20\),Cited by:[§3\.3](https://arxiv.org/html/2607.22614#S3.SS3.p2.1)\.
- S\. Rajbhandari, O\. Ruwase, J\. Rasley, S\. Smith, and Y\. He \(2021\)ZeRO\-infinity: breaking the gpu memory wall for extreme scale deep learning\.InProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis,SC ’21,New York, NY, USA\.External Links:ISBN 9781450384421,[Link](https://doi.org/10.1145/3458817.3476205),[Document](https://dx.doi.org/10.1145/3458817.3476205)Cited by:[§3\.3](https://arxiv.org/html/2607.22614#S3.SS3.p2.1)\.
- J\. Ren, S\. Rajbhandari, R\. Y\. Aminabadi, O\. Ruwase, S\. Yang, M\. Zhang, D\. Li, and Y\. He \(2021\)ZeRO\-Offload: democratizing Billion\-Scale model training\.In2021 USENIX Annual Technical Conference \(USENIX ATC 21\),pp\. 551–564\.External Links:ISBN 978\-1\-939133\-23\-6,[Link](https://www.usenix.org/conference/atc21/presentation/ren-jie)Cited by:[§3\.3](https://arxiv.org/html/2607.22614#S3.SS3.p2.1)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- G\. Sheng, Y\. Tong, B\. Wan, W\. Zhang, C\. Jia, X\. Wu, Y\. Wu, X\. Li, C\. Zhang, Y\. Peng, H\. Lin, X\. Liu, and C\. Wu \(2026\)Laminar: a scalable asynchronous rl post\-training framework\.InProceedings of the 21st European Conference on Computer Systems,EUROSYS ’26,New York, NY, USA,pp\. 400–422\.External Links:ISBN 9798400722127,[Link](https://doi.org/10.1145/3767295.3803580),[Document](https://dx.doi.org/10.1145/3767295.3803580)Cited by:[3rd item](https://arxiv.org/html/2607.22614#S2.I1.i3.p1.1)\.
- G\. Sheng, C\. Zhang, Z\. Ye, X\. Wu, W\. Zhang, R\. Zhang, Y\. Peng, H\. Lin, and C\. Wu \(2025\)HybridFlow: a flexible and efficient rlhf framework\.InProceedings of the Twentieth European Conference on Computer Systems,EuroSys ’25,New York, NY, USA,pp\. 1279–1297\.External Links:ISBN 9798400711961,[Link](https://doi.org/10.1145/3689031.3696075),[Document](https://dx.doi.org/10.1145/3689031.3696075)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- Y\. Sheng, L\. Zheng, B\. Yuan, Z\. Li, M\. Ryabinin, B\. Chen, P\. Liang, C\. Ré, I\. Stoica, and C\. Zhang \(2023\)FlexGen: high\-throughput generative inference of large language models with a single gpu\.InProceedings of the 40th International Conference on Machine Learning,ICML’23\.Cited by:[§3\.3](https://arxiv.org/html/2607.22614#S3.SS3.p2.1)\.
- M\. Shoeybi, M\. Patwary, R\. Puri, P\. LeGresley, J\. Casper, and B\. Catanzaro \(2019\)Cited by:[§3\.3](https://arxiv.org/html/2607.22614#S3.SS3.p1.1)\.
- K\. Team, A\. Du, B\. Gao, B\. Xing,et al\.\(2025\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p2.1),[1st item](https://arxiv.org/html/2607.22614#S2.I1.i1.p1.1)\.
- Q\. Team \(2025\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- J\. Thorpe, P\. Zhao, J\. Eyolfson, Y\. Qiao, Z\. Jia, M\. Zhang, R\. Netravali, and G\. H\. Xu \(2023\)Bamboo: making preemptible instances resilient for affordable training of large DNNs\.In20th USENIX Symposium on Networked Systems Design and Implementation \(NSDI 23\),Boston, MA,pp\. 497–513\.External Links:ISBN 978\-1\-939133\-33\-5,[Link](https://www.usenix.org/conference/nsdi23/presentation/thorpe)Cited by:[§2](https://arxiv.org/html/2607.22614#S2.p3.1)\.
- verl community \(2026\)Verl: volcano engine reinforcement learning for llms\.Note:https://github\.com/verl\-project/verlCited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1),[§4](https://arxiv.org/html/2607.22614#S4.p1.1)\.
- J\. Xiao, M\. Chen, Q\. Ren, J\. Song, J\. Huang, Y\. Deng, C\. Tong, W\. Chen, S\. Wang, Z\. Bi, S\. Lu, Y\. Duan, X\. Wang, R\. Yu, W\. Yang, L\. Ai, E\. Yang, and B\. Shi \(2026\)Cited by:[§2](https://arxiv.org/html/2607.22614#S2.p3.1)\.
- C\. Yu, Y\. Wang, Z\. Guo, H\. Lin, S\. Xu, H\. Zang, Q\. Zhang, Y\. Wu, C\. Zhu, J\. Hu,et al\.\(2025\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- G\. Yu, J\. S\. Jeong, G\. Kim, S\. Kim, and B\. Chun \(2022\)Orca: a distributed serving system for Transformer\-Based generative models\.In16th USENIX Symposium on Operating Systems Design and Implementation \(OSDI 22\),Carlsbad, CA,pp\. 521–538\.External Links:ISBN 978\-1\-939133\-28\-1,[Link](https://www.usenix.org/conference/osdi22/presentation/yu)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p2.1)\.
- Q\. Yuet al\.\(2025\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.
- H\. Zhanget al\.\(2026\)Cited by:[§2](https://arxiv.org/html/2607.22614#S2.p3.1)\.
- Y\. Zhang, H\. Jiang, X\. Luo, Z\. Yang, C\. Zhang, Y\. Shen, Y\. Y\. Dongsheng Li, L\. Qiu, and Y\. You \(2026\)Cited by:[§2](https://arxiv.org/html/2607.22614#S2.p3.1)\.
- Y\. Zhao, A\. Gu, R\. Vasanth, O\. Ruwase, J\. Johnson, A\. Nayak, B\. Fineran,et al\.\(2023\)Cited by:[§3\.3](https://arxiv.org/html/2607.22614#S3.SS3.p1.1)\.
- Y\. Zhou, J\. Li,et al\.\(2025\)Cited by:[§1](https://arxiv.org/html/2607.22614#S1.p2.1)\.
- Z\. Zhu, C\. Xie, X\. Lv, and slime Contributors \(2025\)Slime: an llm post\-training framework for rl scaling\.Note:[https://github\.com/THUDM/slime](https://github.com/THUDM/slime)GitHub repository\. Corresponding author: Xin LvCited by:[§1](https://arxiv.org/html/2607.22614#S1.p1.1)\.