GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving

# GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving
Source: [https://arxiv.org/html/2609.03494](https://arxiv.org/html/2609.03494)
Qiankun MaAffiliation:Shenzhen Institutes of Advanced Technology, Chinese Academy of SciencesAffiliation:Pengcheng LaboratoryAffiliation:University of Chinese Academy of SciencesZinan XiongAffiliation:Pengcheng LaboratoryHaofei WangAffiliation:Pengcheng LaboratoryZhen SongAffiliation:Pengcheng LaboratoryYang XiangAffiliation:Pengcheng LaboratoryZiyao ZhangHairong ZhengEmail:[maqiankun25@mails\.ucas\.ac\.cn†Corresponding authors](mailto:Corresponding%20authors)Affiliation:Shenzhen Institutes of Advanced Technology, Chinese Academy of SciencesAffiliation:Pengcheng Laboratory

###### Abstract

Long\-output reasoning has made the key–value \(KV\) cache a critical memory bottleneck for efficient LLM serving\. Existing KV compression methods usually rely on a predefined per\-request budget and adjust only which KV states are retained, leaving the total capacity fixed throughout decoding\. However, reasoning workloads exhibit substantial demand variation: different requests require different KV capacities, and the attention demand of an individual request evolves during generation\. We introduceGrowPage, an on\-demand KV budgeting framework that treats KV capacity as a runtime resource\. GrowPage maintains lightweight dual\-timescale query summaries to capture recent and long\-term attention behaviors, and uses their relative attention working sets to estimate demand evolution\. At each capacity boundary, GrowPage either compresses KV states within the current allocation or acquires an additional physical page when broader demand emerges\. By integrating with PagedAttention’s page\-level memory abstraction, GrowPage preserves continuous batching and prefix caching\. Experiments on reasoning benchmarks across multiple models show that GrowPage achieves a superior performance–throughput trade\-off over existing approaches\.

## 1Introduction

Large reasoning models\([Yuan et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib1);[Guo et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib2)\)achieve strong performance on challenging tasks such as mathematical reasoning and code generation by producing long chains of thought \(CoTs\) that iteratively explore, verify, and revise candidate solutions\([Wei et al\., 2022](https://arxiv.org/html/2609.03494#bib.bib3)\)\. This capability, however, comes with substantial serving cost\. Reasoning models can generate thousands or even tens of thousands of tokens, continuously expanding their key–value \(KV\) caches\. When multiple long\-running reasoning requests are served together, the resulting memory footprint limits the number of requests that can remain resident on GPU, making the KV cache a major bottleneck for efficient LLM serving\.\([Hu et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib4);[Chitty\-Venkata et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib5)\)\.

This pressure has motivated extensive research on KV cache compression\. Early methods mainly reduce prompt KV states\([Li et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib6);[Cai et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib7);[Feng et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib8)\), while recent approaches extend token eviction to decoding to bound cache growth during long\-output generation\([Ghadia et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib9);[Cai et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib10);[Liao et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib11);[Ramachandran et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib12)\)\. However, many such methods are difficult to integrate with modern serving mechanisms such as continuous batching and prefix caching, so their algorithmic memory savings do not necessarily translate into higher serving efficiency\. Zipage\([Liao et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib14)\)bridges this gap through Compressed PagedAttention, combining token\-wise KV eviction with PagedAttention while preserving key serving capabilities\.

Yet Zipage, like most existing KV compression methods, still operates under a predefined per\-request KV capacity bound\. This underutilizes PagedAttention’s block\-based memory management, which naturally supports incremental physical\-page allocation, and mismatches the heterogeneous memory demands of reasoning requests\. Existing adaptive methods may change which KV states are retained or how a given budget is distributed, but typically leave the total per\-request capacity fixed\. GrowPage instead makes this capacity itself an online decision variable\. As illustrated in Figure[1](https://arxiv.org/html/2609.03494#S1.F1)\(a\), a generous budget over\-provisions low\-demand requests, whereas an aggressive budget can under\-provision demanding requests and evict reasoning\-critical history\. Tuning a fixed budget therefore merely shifts the operating point between wasted memory and insufficient capacity\.

![Refer to caption](https://arxiv.org/html/2609.03494v1/figure1.png)Figure 1:Illustrative comparison of KV budgeting strategies for LLM reasoning\. \(a\) Fixed\-budget Zipage and on\-demand GrowPage for low\- and high\-demand requests\. \(b\) Reasoning performance vs\. throughput on AMC23 with Qwen3\-8B\.Our analyses in Section[2](https://arxiv.org/html/2609.03494#S2)reveal that this mismatch arises at two levels: the minimum sufficient KV budget varies substantially across requests, and attention concentration within an individual request also evolves throughout decoding\. These observations motivate treating per\-request KV capacity as a runtime quantity rather than a static budget chosen in advance\. We theoretically connect attention concentration to the KV capacity required to preserve attention outputs, and empirically show that the relative working\-set difference induced by recent and long\-term query summaries provides a lightweight signal for tracking demand evolution\.

Based on these insights, we introduceGrowPage, an on\-demand KV budgeting framework for efficient LLM reasoning serving\. GrowPage maintains lightweight dual\-timescale query summaries, using the long\-timescale summary as a historical reference and the short\-timescale summary to capture recent attention behavior\. Their relative working sets estimate the current demand trend\. At each capacity boundary, GrowPage either compresses historical KV states within the existing allocation or acquires an additional physical page\. By coupling this decision with PagedAttention memory management, GrowPage adapts per\-request KV capacity while preserving continuous batching and prefix caching\. Across multiple mathematical reasoning and code\-generation benchmarks and diverse model architectures, GrowPage consistently improves the reasoning performance–throughput trade\-off; Figure[1](https://arxiv.org/html/2609.03494#S1.F1)\(b\) shows a representative result on AMC23 with Qwen3\-8B\.

Our contributions are summarized as follows:

- •We identify the demand mismatch underlying fixed KV budgeting in reasoning serving\.KV requirements vary both across requests and throughout decoding, making static capacity bounds prone to either memory over\-provisioning or insufficient retention\.
- •We propose GrowPage, an on\-demand KV budgeting framework for reasoning LLMs\.GrowPage uses dual\-timescale query summaries to estimate evolving attention demand and guide online compression or incremental physical\-page allocation\.
- •We co\-design on\-demand KV budgeting with PagedAttention\-based serving\.GrowPage translates online demand estimates into page\-level memory actions while preserving continuous batching and prefix caching, yielding a better reasoning performance–throughput trade\-off under LLM reasoning serving\.

## 2Why On\-Demand KV Budgeting?

Static KV budgeting assumes that memory demand can be adequately captured by a capacity fixed in advance for the entire decoding process\. We examine this assumption from two complementary perspectives: inter\-request heterogeneity and intra\-request temporal variation\.

Observation 1: Minimum sufficient KV budgets vary substantially across requests\.We first quantify the KV capacity required by individual reasoning requests\. For each request–budget pair, we perform multiple independent generations and aggregate their correctness to reduce sampling variance\. Given a set of budgetsℬ\\mathcal\{B\}, we define the*minimum sufficient KV budget*of requestiias:

Bi⋆=min\{B∈ℬ\|ci\(B′\)=1,∀B′∈ℬ,B′≥B\},B\_\{i\}^\{\\star\}=\\min\\left\\\{B\\in\\mathcal\{B\}\\;\\middle\|\\;c\_\{i\}\(B^\{\\prime\}\)=1,\\ \\forall B^\{\\prime\}\\in\\mathcal\{B\},B^\{\\prime\}\\geq B\\right\\\},\(1\)whereci​\(B\)c\_\{i\}\(B\)indicates whether requestiiis answered correctly under budgetBB, as determined from multiple independent generations\. Requiring correctness to hold under all larger tested budgets reduces the influence of occasional non\-monotonic outcomes\. Requests that are not consistently solved under any tested budget are categorized as*Unsolved*\.

Figure 2:Distribution of minimum sufficient KV budgets for Qwen3 8B\.As shown in Figure[2](https://arxiv.org/html/2609.03494#S2.F2),Bi⋆B\_\{i\}^\{\\star\}spans the entire tested range across AIME24, AMC23, and LiveCode\. This wide distribution indicates substantial inter\-request heterogeneity: a small fixed budget under\-provisions demanding requests, whereas a large one unnecessarily reserves KV memory for requests requiring much less capacity\. Appendix[I](https://arxiv.org/html/2609.03494#A9)further shows that GrowPage generally allocates larger adaptive capacities to requests with higher minimum sufficient budgets, validating its request\-level demand adaptation\. This inter\-request heterogeneity already challenges the use of a single fixed budget\. However, the mismatch is not limited to differences across requests\. KV demand can also evolve over time within the same request\.

Observation 2: KV demand varies substantially during decoding\.Prior work has shown that long\-form reasoning traverses distinct reasoning stages with substantially different attention sparsity patterns\([Ramachandran et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib12)\)\.

Figure 3:Attention sparsity throughout decoding for DeepSeek\-R1 Distill Llama 8B on LiveCodeBench\.Consistent with this observation, Figure[3](https://arxiv.org/html/2609.03494#S2.F3)shows that the attention sparsity of a representative request changes throughout decoding, repeatedly transitioning between concentrated and diffuse regimes\. Such temporal variation suggests that the amount of historical KV information required by a request is itself non\-stationary, making a capacity fixed before decoding unnecessarily conservative at some stages yet potentially insufficient at others\.

Together, these observations reveal that KV demand varies both across requests and over the course of decoding\. They motivate treating KV capacity as an on\-demand resource rather than a fixed reservation\. In the next section, we formalize the connection between attention concentration and the KV capacity required to preserve attention outputs, providing the theoretical basis for GrowPage’s online capacity control\.

## 3Methodology

### 3\.1Theoretical Analysis

Section[2](https://arxiv.org/html/2609.03494#S2)shows that attention concentration varies substantially throughout decoding\. We now formalize why such variation implies changing KV capacity requirements\. At decoding steptt, let𝐚t=\(at,1,…,at,Nt\)\\mathbf\{a\}\_\{t\}=\(a\_\{t,1\},\\ldots,a\_\{t,N\_\{t\}\}\)denote the normalized attention distribution overNtN\_\{t\}historical KV states, withat,\(1\)≥⋯≥at,\(Nt\)a\_\{t,\(1\)\}\\geq\\cdots\\geq a\_\{t,\(N\_\{t\}\)\}denoting the sorted attention weights\. For a target coveragep∈\(0,1\)p\\in\(0,1\), we define the*pp\-coverage attention demand*as:

Dp​\(𝐚t\)=min⁡\{k:∑i=1kat,\(i\)≥p\}\.D\_\{p\}\(\\mathbf\{a\}\_\{t\}\)=\\min\\left\\\{k:\\sum\_\{i=1\}^\{k\}a\_\{t,\(i\)\}\\geq p\\right\\\}\.\(2\)Intuitively, concentrated attention yields a smallerDpD\_\{p\}, whereas diffuse attention requires a larger working set\. We next connect this quantity to attention\-output approximation\. Let𝐨t\\mathbf\{o\}\_\{t\}denote the full\-cache attention output and𝐨^t​\(S\)\\widehat\{\\mathbf\{o\}\}\_\{t\}\(S\)the output obtained by retaining and renormalizing a subsetSS:

𝐨t=∑i=1Ntat,i​𝐯i,𝐨^t​\(S\)=∑i∈Sat,imt​\(S\)​𝐯i,mt​\(S\)=∑i∈Sat,i\.\\mathbf\{o\}\_\{t\}=\\sum\_\{i=1\}^\{N\_\{t\}\}a\_\{t,i\}\\mathbf\{v\}\_\{i\},\\qquad\\widehat\{\\mathbf\{o\}\}\_\{t\}\(S\)=\\sum\_\{i\\in S\}\\frac\{a\_\{t,i\}\}\{m\_\{t\}\(S\)\}\\mathbf\{v\}\_\{i\},\\quad m\_\{t\}\(S\)=\\sum\_\{i\\in S\}a\_\{t,i\}\.\(3\)
Under the bounded\-value assumption‖𝐯i‖2≤Vmax\\\|\\mathbf\{v\}\_\{i\}\\\|\_\{2\}\\leq V\_\{\\max\}, we define the*worst\-case KV capacity*:

Cδwc​\(𝐚t\)=minS⁡\{\|S\|:sup‖𝐯i‖2≤Vmax‖𝐨t−𝐨^t​\(S\)‖2≤δ\}\.C\_\{\\delta\}^\{\\mathrm\{wc\}\}\(\\mathbf\{a\}\_\{t\}\)=\\min\_\{S\}\\left\\\{\|S\|:\\sup\_\{\\\|\\mathbf\{v\}\_\{i\}\\\|\_\{2\}\\leq V\_\{\\max\}\}\\left\\\|\\mathbf\{o\}\_\{t\}\-\\widehat\{\\mathbf\{o\}\}\_\{t\}\(S\)\\right\\\|\_\{2\}\\leq\\delta\\right\\\}\.\(4\)For any retained subsetSS, its worst\-case approximation error is exactly:

sup‖𝐯i‖2≤Vmax‖𝐨t−𝐨^t​\(S\)‖2=2​Vmax​\(1−mt​\(S\)\)\.\\sup\_\{\\\|\\mathbf\{v\}\_\{i\}\\\|\_\{2\}\\leq V\_\{\\max\}\}\\left\\\|\\mathbf\{o\}\_\{t\}\-\\widehat\{\\mathbf\{o\}\}\_\{t\}\(S\)\\right\\\|\_\{2\}=2V\_\{\\max\}\\bigl\(1\-m\_\{t\}\(S\)\\bigr\)\.\(5\)The upper bound follows from the triangle inequality, and is tight by assigning retained and discarded values to opposite directions with normVmaxV\_\{\\max\}; the full proof is provided in Appendix[C](https://arxiv.org/html/2609.03494#A3)\.

###### Theorem 3\.1\.

Assume‖𝐯i‖2≤Vmax\\\|\\mathbf\{v\}\_\{i\}\\\|\_\{2\}\\leq V\_\{\\max\}for allii\. For any approximation tolerance0<δ<2​Vmax0<\\delta<2V\_\{\\max\}, the minimum worst\-case KV capacity satisfiesCδwc​\(𝐚t\)=D1−δ/\(2​Vmax\)​\(𝐚t\)C\_\{\\delta\}^\{\\mathrm\{wc\}\}\(\\mathbf\{a\}\_\{t\}\)=D\_\{1\-\\delta/\(2V\_\{\\max\}\)\}\(\\mathbf\{a\}\_\{t\}\)\.

Theorem[3\.1](https://arxiv.org/html/2609.03494#S3.Thmtheorem1)follows because guaranteeing error at mostδ\\deltais equivalent to retaining attention massmt​\(S\)≥1−δ/\(2​Vmax\)m\_\{t\}\(S\)\\geq 1\-\\delta/\(2V\_\{\\max\}\), and the maximum mass achievable withkkretained states is obtained by selecting the top\-kkattention weights\. Thus, under the same tolerance, diffuse attention requires a larger robust KV capacity, whereas concentrated attention can be preserved with fewer states\. For the realized value vectors of a particular request, the required capacity may be smaller; Theorem[3\.1](https://arxiv.org/html/2609.03494#S3.Thmtheorem1)characterizes the worst\-case requirement determined solely by the attention distribution\.

Combined with the non\-stationary attention behavior observed in Section[2](https://arxiv.org/html/2609.03494#S2), this result motivates adapting KV capacity as attention demand evolves during decoding\. We next describe how GrowPage tracks such changes online and translates them into on\-demand capacity control\.

### 3\.2Online Demand Estimation

Figure[4](https://arxiv.org/html/2609.03494#S3.F4)illustrates the overall workflow of GrowPage\. During decoding, each request maintains short\- and long\-timescale query summaries\. When the current KV capacity is exhausted, GrowPage compares their induced historical attention working sets to estimate the demand trend and decide whether to compress the existing KV states or acquire an additional physical page\.

#### Dual\-timescale query summaries\.

GrowPage maintains two exponentially smoothed query representations for each request and attention layer\. For clarity, we omit layer and head indices in Eq\.[6](https://arxiv.org/html/2609.03494#S3.E6)\. Let𝐪t\\mathbf\{q\}\_\{t\}denote the current query after query normalization but before RoPE:

𝐪¯tx=βx​𝐪¯t−1x\+\(1−βx\)​𝐪t,x∈\{S,L\},\\bar\{\\mathbf\{q\}\}\_\{t\}^\{x\}=\\beta\_\{x\}\\bar\{\\mathbf\{q\}\}\_\{t\-1\}^\{x\}\+\(1\-\\beta\_\{x\}\)\\mathbf\{q\}\_\{t\},\\qquad x\\in\\\{S,L\\\},\(6\)whereβS<βL\\beta\_\{S\}<\\beta\_\{L\}; we useβS=0\.9\\beta\_\{S\}=0\.9andβL=0\.999\\beta\_\{L\}=0\.999by default\. The long\-timescale summary serves as a historical reference, while the short\-timescale summary captures recent attention behavior\. Their relative behavior therefore provides a lightweight estimate of demand evolution\. Both summaries are maintained in the pre\-RoPE space to avoid averaging queries under different rotary transformations\. At each decision point, we calibrate their per\-head RMS scales and apply RoPE at the current position before attention estimation\.

Figure 4:Overview of GrowPage\. At each KV capacity boundary, GrowPage estimates the request’s attention\-demand trend and dynamically chooses between*Compress & Hold*and*Grow by One Page*\.
#### Attention\-demand trend\.

At a capacity boundary, GrowPage constructs a historical candidate set𝒞t\\mathcal\{C\}\_\{t\}and excludes the most recent KV block to reduce local\-attention dominance\. For layerℓ\\elland KV headhh, the aligned short\- and long\-timescale queries induce normalized attention distributions𝐚t,ℓ,hS\\mathbf\{a\}\_\{t,\\ell,h\}^\{S\}and𝐚t,ℓ,hL\\mathbf\{a\}\_\{t,\\ell,h\}^\{L\}over𝒞t\\mathcal\{C\}\_\{t\}\. Following Section[3\.1](https://arxiv.org/html/2609.03494#S3.SS1), we compute

Kt,ℓ,hx=min⁡\{k:∑i=1kat,ℓ,h,\(i\)x≥p\},rt,ℓ,hx=Kt,ℓ,hx\|𝒞t\|,x∈\{S,L\},K\_\{t,\\ell,h\}^\{x\}=\\min\\left\\\{k:\\sum\_\{i=1\}^\{k\}a\_\{t,\\ell,h,\(i\)\}^\{x\}\\geq p\\right\\\},\\qquad r\_\{t,\\ell,h\}^\{x\}=\\frac\{K\_\{t,\\ell,h\}^\{x\}\}\{\|\\mathcal\{C\}\_\{t\}\|\},\\quad x\\in\\\{S,L\\\},\(7\)whereat,ℓ,h,\(i\)xa\_\{t,\\ell,h,\(i\)\}^\{x\}denotes the attention weights sorted in descending order, and we setp=0\.99p=0\.99by default\. Since page growth is a request\-level decision, we follow the mean\-aggregation convention of prior KV\-cache methods and aggregate the working\-set ratios across layers and KV heads:

rtx=1∑ℓ∈ℒ\|ℋℓ\|​∑ℓ∈ℒ∑h∈ℋℓrt,ℓ,hx,Δt=rtS−rtL,r\_\{t\}^\{x\}=\\frac\{1\}\{\\sum\_\{\\ell\\in\\mathcal\{L\}\}\|\\mathcal\{H\}\_\{\\ell\}\|\}\\sum\_\{\\ell\\in\\mathcal\{L\}\}\\sum\_\{h\\in\\mathcal\{H\}\_\{\\ell\}\}r\_\{t,\\ell,h\}^\{x\},\\qquad\\Delta\_\{t\}=r\_\{t\}^\{S\}\-r\_\{t\}^\{L\},\(8\)whereℒ\\mathcal\{L\}denotes the participating layers andℋℓ\\mathcal\{H\}\_\{\\ell\}the KV heads in layerℓ\\ell\. A negativeΔt\\Delta\_\{t\}indicates increasing attention concentration, whereas a positive value indicates an expanding working set\. To translate this trend into a page\-level decision, GrowPage comparesΔt\\Delta\_\{t\}with a demand thresholdτ\\tau: it selects*Grow by One Page*whenΔt\>τ\\Delta\_\{t\}\>\\tau, and otherwise performs*Compress & Hold*\. We useτ=0\\tau=0by default; its sensitivity is analyzed in Appendix[D\.1](https://arxiv.org/html/2609.03494#A4.SS1)\.

### 3\.3Trend\-Guided Capacity Control

GrowPage invokes capacity control only when the current KV allocation is exhausted\. At each boundary, it chooses between*Compress & Hold*and*Grow by One Page*, naturally aligning adaptation with PagedAttention’s page granularity\. GrowPage adopts monotonic capacity growth rather than actively shrinking live requests, since page reclamation requires additional irreversible KV eviction; bidirectional resizing is studied in Appendix[H\.1](https://arxiv.org/html/2609.03494#A8.SS1)\.

#### Compress & Hold\.

When recent demand becomes more concentrated, GrowPage retains the current allocation and creates space by compacting historical KV states\. Unlike the request\-level demand signal, KV selection is performed independently for each layer and KV head\. For candidate tokeniiat layerℓ\\elland headhh, we combine its importance under the two temporal summaries as:

si,ℓ,h=max⁡\(ai,ℓ,hS,ai,ℓ,hL\)\.s\_\{i,\\ell,h\}=\\max\\left\(a\_\{i,\\ell,h\}^\{S\},a\_\{i,\\ell,h\}^\{L\}\\right\)\.\(9\)Thus, different layer–head pairs can retain different token subsets, while the maximum across temporal views preserves tokens important to either recent or long\-term attention\. GrowPage releases exactly one page KV states at each compression event\. LetPtP\_\{t\}denote the current number of allocated pages,bbthe page size, andℛt\\mathcal\{R\}\_\{t\}the protected recent tokens\. The resulting historical retention budget is therefore determined by the target capacity\(Pt−1\)​b\(P\_\{t\}\-1\)b, rather than treated as a tunable hyperparameter\. We denote this derived budget byKtK\_\{t\}\. To prevent compression from over\-concentrating on a few historical regions, GrowPage combines block\-local retention with global selection independently for each layer–head pair\. Letℬj\\mathcal\{B\}\_\{j\}denote the candidate positions from historical blockjjandklock\_\{\\mathrm\{loc\}\}the minimum local retention quota\. For each\(ℓ,h\)\(\\ell,h\),

Slocℓ,h\\displaystyle S\_\{\\mathrm\{loc\}\}^\{\\ell,h\}=⋃jTopKkloc⁡\(ℬj;sℓ,h\),\\displaystyle=\\bigcup\_\{j\}\\operatorname\{TopK\}\_\{k\_\{\\mathrm\{loc\}\}\}\(\\mathcal\{B\}\_\{j\};s\_\{\\ell,h\}\),\(10\)Sℓ,h\\displaystyle S^\{\\ell,h\}=Slocℓ,h∪TopKKt−\|Slocℓ,h\|⁡\(𝒞t∖Slocℓ,h,sℓ,h\),\\displaystyle=S\_\{\\mathrm\{loc\}\}^\{\\ell,h\}\\cup\\operatorname\{TopK\}\_\{K\_\{t\}\-\|S\_\{\\mathrm\{loc\}\}^\{\\ell,h\}\|\}\\bigl\(\\mathcal\{C\}\_\{t\}\\setminus S\_\{\\mathrm\{loc\}\}^\{\\ell,h\};s\_\{\\ell,h\}\\bigr\),wheresℓ,h=\{si,ℓ,h\}is\_\{\\ell,h\}=\\\{s\_\{i,\\ell,h\}\\\}\_\{i\}\. Thus,klock\_\{\\mathrm\{loc\}\}controls only the balance between block\-local coverage and global importance, while the total number of retained historical tokens is fixed by the requirement to create one page worth of free slots\. The effect ofklock\_\{\\mathrm\{loc\}\}is studied in Appendix[D\.2](https://arxiv.org/html/2609.03494#A4.SS2)\. The retained KV states are then compacted within the existing physical\-page allocation\.

#### Grow by One Page\.

When recent demand becomes more diffuse, GrowPage acquires one additional physical KV page instead of further compressing the historical working set\. The page is appended to the request’s block table, and decoding continues until the next capacity boundary\.

## 4System Integration

Section[3](https://arxiv.org/html/2609.03494#S3)describes how GrowPage estimates the evolving KV demand of each request and converts it into page\-level capacity decisions\. We now describe how these decisions are integrated into a PagedAttention\-based serving runtime with low additional state overhead and minimal disruption to batched scheduling\.

### 4\.1Lightweight Per\-Request State Management

GrowPage introduces only a small amount of auxiliary state for online demand estimation\. For each active request, the runtime maintains the short\- and long\-timescale query summaries described in Section[3\.2](https://arxiv.org/html/2609.03494#S3.SS2)\. These states are allocated together with the request metadata, updated in place during decoding, and recycled when the request completes\. Unlike the KV cache itself, their size is independent of sequence length, avoiding additional memory growth as reasoning proceeds\.

To avoid frequent GPU\-side memory allocation during continuous batching, GrowPage pre\-allocates the storage required by these summaries for active request slots and reuses it across requests\. We slightly modify the attention kernel to update the short\- and long\-timescale summaries from the normalized pre\-RoPE queries during the regular decoding path\. The more expensive attention\-based demand estimation is invoked only at capacity boundaries, so most decoding steps require only lightweight in\-place state updates\.

### 4\.2Scheduler\-Aware Page Allocation and Fallback

GrowPage maps the capacity decisions in Section[3\.3](https://arxiv.org/html/2609.03494#S3.SS3)directly onto PagedAttention’s physical\-page abstraction\. For*Compress & Hold*, the request keeps its existing physical\-page allocation while selected historical KV states are compacted to release slots for subsequent tokens\. For*Grow by One Page*, the scheduler obtains an additional page from the global free\-page pool and appends its physical page identifier to the request’s block table\. The request can then continue decoding without changing the underlying attention or block\-table interface\.

Page growth is coordinated with global memory availability\. When a free physical page is available, the growth request is satisfied immediately\. Under memory pressure, however, granting additional capacity to one request may reduce the memory available to other concurrent requests\. GrowPage therefore falls back to compression when additional page allocation cannot be safely satisfied, allowing the request to continue within its current allocation\. If global memory pressure remains unresolved, the existing serving scheduler can further apply controlled request preemption and rescheduling\. This keeps per\-request adaptation subordinate to the global memory manager rather than allowing individual requests to monopolize the shared KV pool\.

Importantly, GrowPage preserves the page/block abstraction of PagedAttention rather than introducing a separate memory layout\. It therefore remains compatible with serving mechanisms such as continuous batching, prefix caching, and CUDA Graph execution\. More details see Appendix[E](https://arxiv.org/html/2609.03494#A5)\. GrowPage additionally inherits the asynchronous compression pipeline of Zipage\([Liao et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib14)\), allowing KV compaction to overlap with serving execution and reducing compression\-induced stalls on the critical decoding path\. Together, these design choices allow online KV\-capacity adaptation to be incorporated into PagedAttention\-based serving without sacrificing the system optimizations of the underlying engine\.

## 5Experiments

### 5\.1Experimental Setup

#### Models and benchmarks\.

We evaluate GrowPage on two representative reasoning models: DeepSeek\-R1\-Distill\-Llama\-8B\([Guo et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib2)\)and Qwen3\-8B\([Yang et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib15)\)\. Following previous KV\-cache compression studies, we consider both mathematical reasoning and code generation tasks, including GSM8K\([Cobbe et al\., 2021](https://arxiv.org/html/2609.03494#bib.bib16)\), MATH500\([Hendrycks et al\., 2021](https://arxiv.org/html/2609.03494#bib.bib17)\), AMC23, AIME24, and LiveCodeBench\([Jain et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib18)\)\. These benchmarks cover diverse reasoning workloads with varying output lengths and KV\-cache demands\.

Table 1:Main comparison on reasoning benchmarks\. We report pass@1 accuracy \(%\) and decoding throughput \(tokens/s\)\. The results are evaluated on DeepSeek\-R1\-Distill\-Llama\-8B and Qwen3\-8B\.Pass@1 Accuracy \(%\)↑\\uparrowTPS \(tokens/s\)↑\\uparrowMethodGSM8KMATH500AMC23AIME24LiveCodeAvg\.GSM8KMATH500AMC23AIME24LiveCodeAvg\.DeepSeek\-R1\-Distill\-Llama\-8BFullKV \(vLLM\)88\.286\.681\.343\.546\.569\.227051515123576611421472FullKV \(nano\-vLLM\)88\.086\.280\.943\.246\.168\.925761442118272210861401MorphKV\(ICML’25\)85\.575\.870\.435\.833\.960\.3507235189119178245R\-KV\(NeurIPS’25\)87\.184\.777\.839\.942\.266\.3450218177102162221G\-KV\(arXiv’25\)87\.785\.178\.340\.542\.866\.9472228185111171233Zipage\(ACL’26\)88\.085\.079\.041\.444\.667\.6301821742060115812711936GrowPage \(Ours\)88\.085\.879\.444\.745\.468\.7383325992398146517942417Qwen3\-8BFullKV \(vLLM\)95\.796\.192\.575\.583\.988\.726581382106460710761357FullKV \(nano\-vLLM\)95\.695\.892\.175\.083\.588\.42486130399556810151273MorphKV\(ICML’25\)94\.282\.571\.857\.564\.874\.2463425321168330341R\-KV\(NeurIPS’25\)95\.291\.684\.769\.478\.083\.8448396307151305321G\-KV\(arXiv’25\)95\.192\.584\.268\.978\.683\.9424408309154317322Zipage\(ACL’26\)95\.793\.789\.572\.780\.786\.5310720761805100712711853GrowPage \(Ours\)95\.994\.991\.473\.881\.587\.5328025532261119015892174

#### Evaluation metrics and implementation\.

All experiments are conducted on a single NVIDIA A100 GPU with 80 GB memory\. Following Zipage, we implement GrowPage on nano\-vLLM with PagedAttention\-based KV management and integrate the proposed capacity control mechanism into the decoding pipeline\. Unless otherwise specified, we follow the same system configuration as Zipage, fixing the KV block sizebbto 256 tokens and the protected recent\-window sizeℛt\\mathcal\{R\}\_\{t\}to 16 tokens\. We fix the GPU memory utilization ratio to 0\.9 for all vLLM/nano\-vLLM\-based experiments to ensure a consistent memory budget\. We measure serving efficiency using decoding throughput \(TPS\), defined as the total number of output tokens generated across all requests divided by the wall\-clock time from the start of decoding until all requests complete\. Additional experimental details, including implementation settings and dataset configurations, are provided in Appendix[F\.1](https://arxiv.org/html/2609.03494#A6.SS1)\.

### 5\.2Main Results

Figure 5:Accuracy–throughput trade\-off on the mixed GSM8K, AMC23, and AIME24 workload with Qwen3\-8B\.#### Benchmark comparison\.

Table[1](https://arxiv.org/html/2609.03494#S5.T1)compares GrowPage with full KV inference and representative KV\-cache compression methods\. For fixed\-budget baselines, we use the dataset\-specific KV budgets reported in Table[5](https://arxiv.org/html/2609.03494#A6.T5)\. To keep the main comparison concise, Table[1](https://arxiv.org/html/2609.03494#S5.T1)reports only Pass@1 accuracy and decoding throughput \(TPS\), while additional serving metrics and detailed analysis are provided in Appendix[G](https://arxiv.org/html/2609.03494#A7)\. All serving\-system baselines use the same GPU memory budget, while MorphKV, R\-KV, and G\-KV use the largest executable batch size under the same memory constraint to maximize throughput\. GrowPage consistently achieves a better reasoning performance–throughput trade\-off across both models\. Compared with full KV inference, GrowPage improves the average TPS by 64\.3% on DeepSeek\-R1\-Distill\-Llama\-8B and 60\.2% on Qwen3\-8B while maintaining comparable pass@1 accuracy\. Although MorphKV, R\-KV, and G\-KV substantially reduce KV memory, their lack of serving\-system co\-design limits the resulting throughput gains even under maximized batching\. Compared with Zipage, which preserves PagedAttention compatibility, GrowPage further improves throughput by adapting KV capacity to request\-specific demand rather than relying on a fixed budget\.

#### Pareto Analysis under Mixed Workloads\.

To evaluate on\-demand KV budgeting under heterogeneous request demands, we construct a mixed workload combining GSM8K, AMC23, and AIME24, and compare GrowPage with FullKV and Zipage under different KV budgets\. Details are provided in Appendix[F\.2](https://arxiv.org/html/2609.03494#A6.SS2)\. As shown in Figure[5](https://arxiv.org/html/2609.03494#S5.F5),CCdenotes the average number of concurrent requests resident in GPU memory during serving\. Increasing the fixed budget of Zipage improves accuracy but reducesCC, resulting in lower throughput\. GrowPage achieves 84\.9% pass@1 accuracy at 2213 tokens/s with an average KV budget of 3442 andC=121C=121\. It outperforms Zipage\-4096 in accuracy \(84\.9% vs\. 83\.7%\), throughput \(2213 vs\. 1690 tokens/s\), and resident concurrency \(121121vs\.100100\), while approaching the accuracy of Zipage\-8192 \(85\.1%\) with nearly1\.9×1\.9\\timeshigher throughput and1\.8×1\.8\\timesmore resident requests\. These results show that adaptive KV budgeting better translates limited GPU memory into serving concurrency and throughput under heterogeneous workloads\.

### 5\.3Analysis and Discussion

![Refer to caption](https://arxiv.org/html/2609.03494v1/figure6.png)Figure 6:Correlation between the GrowPage signalΔt\\Delta\_\{t\}and future attention\-demand change on Qwen3\-8B with AMC23\.
Figure 7:Counterfactual validation of GrowPage adaptive capacity decisions on Qwen3\-8B with AMC23\.
#### Future Demand Evolution Prediction\.

We first evaluate whetherΔt\\Delta\_\{t\}reflects the actual change of future KV demand\. Specifically, we define the future demand change as the variation between the current attention working set and the actual attention working set of tokens generated within the next physical KV page\. A positive value indicates that future tokens require a broader historical working set, while a negative value indicates increasing attention concentration\. Figure[7](https://arxiv.org/html/2609.03494#S5.F7)shows the correlation betweenΔt\\Delta\_\{t\}and future demand change on Qwen3\-8B with AMC23\. The two variables exhibit a strong monotonic relationship, achieving a Spearman correlation coefficient ofρ=0\.78\\rho=0\.78and a sign agreement of 80\.3%\. This demonstrates that the dual\-timescale query summaries effectively capture the temporal evolution of attention demand\.

Table 2:Ablation of dual\-timescale demand estimation on Qwen3\-8B\.βS\\beta\_\{S\}βL\\beta\_\{L\}MetricAMC23AIME24Pass@1TPSPass@1TPS00\.999Top\-pp89\.9239471\.2126400\.99Top\-pp89\.1247869\.913170\.90\.99Top\-pp90\.3236271\.712560\.990\.999Top\-pp90\.9230872\.912180\.90\.999Entropy90\.5217972\.411360\.90\.999Top\-pp91\.4226173\.81190

#### Capacity Expansion Benefit Analysis\.

We further investigate whetherΔt\\Delta\_\{t\}can identify when allocating additional KV capacity is beneficial\. At each capacity boundary, we construct two counterfactual KV states corresponding to*Compress & Hold*and*Grow by One Page*\. To avoid the influence of different generation trajectories, we first obtain a reference decoding trace using full KV inference and replay the subsequent tokens under the two counterfactual KV states\. Both branches share the same model parameters and future token sequence from the reference trajectory, differing only in the available KV capacity\. We evaluate the prediction quality of the replayed tokens using teacher\-forced negative log\-likelihood \(NLL\)\. LetLcompressL\_\{\\mathrm\{compress\}\}andLgrowL\_\{\\mathrm\{grow\}\}denote the average NLL under*Compress & Hold*and*Grow by One Page*, respectively\. We define the growth advantage asAt=Lcompress−LgrowA\_\{t\}=L\_\{\\mathrm\{compress\}\}\-L\_\{\\mathrm\{grow\}\}, where a largerAtA\_\{t\}indicates a larger reduction in prediction loss from allocating additional KV capacity\. Figure[7](https://arxiv.org/html/2609.03494#S5.F7)groups samples according to differentΔt\\Delta\_\{t\}ranges and reports the corresponding growth advantage\. The growth advantage consistently increases asΔt\\Delta\_\{t\}becomes larger, demonstrating that the proposed demand\-trend signal not only captures attention\-demand variation but also provides a reliable indicator for capacity expansion decisions\.

Figure 8:Runtime overhead analysis of GrowPage on Qwen3\-8B with AMC23\.
#### Dual\-Timescale Signal Ablation\.

We further ablate the design of the dual\-timescale demand signal on Qwen3\-8B, with results reported in Table[2](https://arxiv.org/html/2609.03494#S5.T2.fig1)\. Here,βS=0\\beta\_\{S\}=0reduces the short\-term summary to the current query\. Across both AMC23 and AIME24, using the current query directly yields higher throughput but noticeably lower reasoning accuracy, while introducing moderate short\-term smoothing improves the accuracy–throughput trade\-off\. A longer\-term referenceβL=0\.999\\beta\_\{L\}=0\.999also consistently improves accuracy overβL=0\.99\\beta\_\{L\}=0\.99under the same short\-term setting\. However, overly smoothing the short\-term summaryβS=0\.99\\beta\_\{S\}=0\.99weakens its responsiveness and performs worse than the default\(βS,βL\)=\(0\.9,0\.999\)\(\\beta\_\{S\},\\beta\_\{L\}\)=\(0\.9,0\.999\), which achieves the highest pass@1 accuracy of 91\.4% on AMC23 and 73\.8% on AIME24\. Replacing the Top\-ppworking\-set metric with attention entropy further degrades both accuracy and throughput under the same EMA configuration\. These results support the use of well\-separated temporal summaries together with the Top\-ppworking set for robust online demand estimation\.

#### Runtime Overhead Analysis\.

GrowPage invokes demand estimation and capacity control only at capacity boundaries rather than at every decoding step\. Figure[8](https://arxiv.org/html/2609.03494#S5.F8)shows that Online Demand, Attention Score, Window Mask, Top\-k Selection, and KV Compaction incur only 5\.84, 8\.54, 0\.70, 8\.72, and 19\.05 ms per event, respectively, accounting for less than 0\.32% of the corresponding decoding time\. KV Compaction dominates the additional overhead, while the overall runtime cost remains negligible relative to long\-output decoding\.

## 6Related Work

#### KV\-cache compression for LLM inference\.

KV\-cache compression has become an important direction for reducing the memory footprint of long\-context and reasoning inference\. Early approaches mainly focus on retaining important tokens under a limited cache budget, such as StreamingLLM\([Xiao et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib21)\), SnapKV\([Li et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib6)\), and PyramidKV\([Cai et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib7)\)\. Recent studies extend KV eviction to the decoding stage to control cache growth during long\-form generation\. Representative methods include H2O\([Zhang et al\., 2023](https://arxiv.org/html/2609.03494#bib.bib19)\), Quest\([Tang et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib20)\), R\-KV\([Cai et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib10)\), G\-KV\([Liao et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib11)\), and ThinkKV\([Ramachandran et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib12)\)\. These methods mainly improve token retention or eviction policies under a predefined KV budget\. However, the allocated capacity itself is typically fixed during decoding, which limits their ability to adapt to heterogeneous and evolving KV demands across reasoning requests\.

#### System\-aware KV optimization\.

Beyond token selection algorithms, recent works explore integrating KV management with practical LLM serving systems\. PagedAttention\([Kwon et al\., 2023](https://arxiv.org/html/2609.03494#bib.bib13)\)introduces a page\-based memory abstraction that enables efficient KV management under continuous batching\. Recent system\-level KV management approaches further investigate efficient page organization and KV scheduling for long\-sequence inference\([Mao et al\.,](https://arxiv.org/html/2609.03494#bib.bib22)\)\. Zipage\([Liao et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib14)\)integrates token\-level KV eviction with PagedAttention through Compressed PagedAttention, preserving important serving features such as prefix caching\. Nevertheless, existing system\-compatible approaches still assume a predefined per\-request KV capacity\. In contrast, GrowPage treats KV capacity itself as a runtime resource and dynamically maps demand estimation into page\-level allocation decisions\.

## 7Conclusion

Large reasoning models introduce substantial KV\-cache pressure due to their long chain\-of\-thought generation, making efficient memory management critical for LLM reasoning serving\. Existing KV compression methods typically rely on fixed per\-request capacity budgets, which cannot accommodate the heterogeneous and evolving KV demands of reasoning requests\. In this work, we present GrowPage, an on\-demand KV budgeting framework that treats KV capacity as a runtime resource\. GrowPage uses lightweight dual\-timescale query summaries to estimate demand evolution and dynamically chooses between KV compaction within the current allocation and incremental physical\-page expansion\. By integrating this capability with PagedAttention\-based serving, GrowPage preserves key system optimizations including continuous batching and prefix caching while achieving improved reasoning performance–throughput trade\-offs across diverse benchmarks\.

## References

- Adnanet al\.\(2024\)M\. Adnan, A\. Arunkumar, G\. Jain, P\. J\. Nair, I\. Soloveychik, and P\. KamathKeyformer: kv cache reduction through key tokens selection for efficient generative inference\.Proceedings of Machine Learning and Systems6,pp\. 114–127\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.
- Caiet al\.\(2026\)Z\. Cai, W\. Xiao, H\. Sun, Y\. Zhang, K\. Wan, Y\. Li, Y\. Zhou, L\. Chang, J\. Gu, Z\. Dong,et al\.R\-kv: redundancy\-aware kv cache compression for reasoning models\.Advances in neural information processing systems38,pp\. 60980–61005\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p2.1),[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px1.p1.1)\.
- Caiet al\.\(2024\)Z\. Cai, Y\. Zhang, B\. Gao, Y\. Liu, Y\. Li, T\. Liu, K\. Lu, W\. Xiong, Y\. Dong, J\. Hu,et al\.Pyramidkv: dynamic kv cache compression based on pyramidal information funneling\.arXiv preprint arXiv:2406\.02069\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p2.1),[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2024\)Y\. Chen, G\. Wang, J\. Shang, S\. Cui, Z\. Zhang, T\. Liu, S\. Wang, Y\. Sun, D\. Yu, and H\. WuNACL: a general and effective kv cache eviction framework for llm at inference time\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 7913–7926\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.
- Chenget al\.\(2026\)P\. Cheng, J\. Wang, T\. Chen, B\. Liu, X\. Hou, and J\. LiuDesireKV: decoupling sensitivity and importance for reasoning\-aware kv cache compression\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 20518–20526\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px2.p1.1)\.
- Chitty\-Venkataet al\.\(2026\)K\. T\. Chitty\-Venkata, J\. Ye, S\. Raskar, A\. Kougkas, X\. Sun, M\. Emani, V\. Vishwanath, and B\. NicolaePagedeviction: structured block\-wise kv cache pruning for efficient large language model inference\.InFindings of the Association for Computational Linguistics: EACL 2026,pp\. 3207–3218\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p1.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§5\.1](https://arxiv.org/html/2609.03494#S5.SS1.SSS0.Px1.p1.1)\.
- Fenget al\.\(2026\)Y\. Feng, J\. Lv, Y\. Cao, X\. Xie, and S\. K\. ZhouAda\-kv: optimizing kv cache eviction by adaptive budget allocation for efficient llm inference\.Advances in Neural Information Processing Systems38,pp\. 113152–113188\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p2.1)\.
- Geet al\.\(2024\)S\. Ge, Y\. Zhang, L\. Liu, M\. Zhang, J\. Han, and J\. GaoModel tells you what to discard: adaptive kv cache compression for llms\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 22975–22988\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.
- Ghadiaet al\.\(2025\)R\. Ghadia, A\. Kumar, G\. Jain, P\. Nair, and P\. DasDialogue without limits: constant\-sized kv caches for extended responses in llms\.arXiv preprint arXiv:2503\.00979\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p2.1)\.
- Guoet al\.\(2025\)D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p1.1),[§5\.1](https://arxiv.org/html/2609.03494#S5.SS1.SSS0.Px1.p1.1)\.
- Hanet al\.\(2026\)S\. Han, Y\. Wu, J\. Yu, and O\. IsayevKARA: efficient reasoning llm serving via sliding\-window kv cache compression\.arXiv preprint arXiv:2607\.01237\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px2.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the math dataset\.arXiv preprint arXiv:2103\.03874\.Cited by:[§5\.1](https://arxiv.org/html/2609.03494#S5.SS1.SSS0.Px1.p1.1)\.
- Hooperet al\.\(2024\)C\. Hooper, S\. Kim, H\. Mohammadzadeh, M\. W\. Mahoney, Y\. S\. Shao, K\. Keutzer, and A\. GholamiKvquant: towards 10 million context length llm inference with kv cache quantization\.Advances in Neural Information Processing Systems37,pp\. 1270–1303\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.
- Huet al\.\(2024\)C\. Hu, H\. Huang, J\. Hu, J\. Xu, X\. Chen, T\. Xie, C\. Wang, S\. Wang, Y\. Bao, N\. Sun,et al\.Memserve: context caching for disaggregated llm serving with elastic memory pool\.arXiv preprint arXiv:2406\.17565\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px3.p1.1)\.
- Huet al\.\(2025\)J\. Hu, W\. Huang, W\. Wang, Z\. Li, T\. Hu, Z\. Liu, X\. Chen, T\. Xie, and Y\. ShanRaas: reasoning\-aware attention sparsity for efficient llm reasoning\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 2577–2590\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p1.1)\.
- Jainet al\.\(2025\)N\. Jain, A\. Gu, W\. Li, F\. Yan, T\. Zhang, S\. Wang, A\. Solar\-Lezama, K\. Sen, and I\. StoicaLivecodebench: holistic and contamination free evaluation of large language models for code\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 58791–58831\.Cited by:[§5\.1](https://arxiv.org/html/2609.03494#S5.SS1.SSS0.Px1.p1.1)\.
- Kaiet al\.\(2026\)J\. Kai, Z\. Xiao, A\. Birch, and Z\. LinInformation\-aware kv cache compression for long reasoning\.arXiv preprint arXiv:2606\.26875\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px2.p1.1)\.
- Kanget al\.\(2024\)H\. Kang, Q\. Zhang, S\. Kundu, G\. Jeong, Z\. Liu, T\. Krishna, and T\. ZhaoGear: an efficient kv cache compression recipe for near\-lossless generative inference of llm\.arXiv preprint arXiv:2403\.05527\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.
- Kwonet al\.\(2023\)W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. Gonzalez, H\. Zhang, and I\. StoicaEfficient memory management for large language model serving with pagedattention\.InProceedings of the 29th symposium on operating systems principles,pp\. 611–626\.Cited by:[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px2.p1.1)\.
- Leeet al\.\(2024\)W\. Lee, J\. Lee, J\. Seo, and J\. Sim\{\\\{infinigen\}\\\}: Efficient generative inference of large language models with dynamic\{\\\{kv\}\\\}cache management\.In18th USENIX symposium on operating systems design and implementation \(OSDI 24\),pp\. 155–172\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px3.p1.1)\.
- Liet al\.\(2026\)M\. Li, Y\. Feng, X\. Xie, and W\. J\. SongREAL: retrieval\-reasoning and logic\-constructed attention behaviors for long\-context kv cache compression\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 39035–39052\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2024\)Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. ChenSnapkv: llm knows what you are looking for before generation\.Advances in Neural Information Processing Systems37,pp\. 22947–22970\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p2.1),[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px1.p1.1)\.
- Liaoet al\.\(2026\)M\. Liao, L\. Wang, C\. Zhang, B\. Qiao, S\. Qin, Q\. Lin, S\. Rajmohan, D\. Zhang, and H\. WanZipage: maintain high request concurrency for llm reasoning through compressed pagedattention\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 7716–7737\.Cited by:[§E\.2](https://arxiv.org/html/2609.03494#A5.SS2.p1.1),[§E\.3](https://arxiv.org/html/2609.03494#A5.SS3.p1.1),[Appendix E](https://arxiv.org/html/2609.03494#A5.p1.1),[§1](https://arxiv.org/html/2609.03494#S1.p2.1),[§4\.2](https://arxiv.org/html/2609.03494#S4.SS2.p3.1),[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px2.p1.1)\.
- Liaoet al\.\(2025\)M\. Liao, L\. Wang, C\. Zhang, Z\. Shen, X\. Mao, S\. Qin, Q\. Lin, S\. Rajmohan, D\. Zhang, and H\. WanG\-kv: decoding\-time kv cache eviction with global attention\.arXiv preprint arXiv:2512\.00504\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p2.1),[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2024a\)A\. Liu, J\. Liu, Z\. Pan, Y\. He, G\. Haffari, and B\. ZhuangMinicache: kv cache compression in depth dimension for large language models\.Advances in Neural Information Processing Systems37,pp\. 139997–140031\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2024b\)Y\. Liu, H\. Li, Y\. Cheng, S\. Ray, Y\. Huang, Q\. Zhang, K\. Du, J\. Yao, S\. Lu, G\. Ananthanarayanan,et al\.Cachegen: kv cache compression and streaming for fast large language model serving\.InProceedings of the ACM SIGCOMM 2024 Conference,pp\. 38–56\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px3.p1.1)\.
- Liuet al\.\(2023\)Z\. Liu, A\. Desai, F\. Liao, W\. Wang, V\. Xie, Z\. Xu, A\. Kyrillidis, and A\. ShrivastavaScissorhands: exploiting the persistence of importance hypothesis for llm kv cache compression at test time\.Advances in Neural Information Processing Systems36,pp\. 52342–52364\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2024c\)Z\. Liu, J\. Yuan, H\. Jin, S\. Zhong, Z\. Xu, V\. Braverman, B\. Chen, and X\. HuKivi: a tuning\-free asymmetric 2bit quantization for kv cache\.arXiv preprint arXiv:2402\.02750\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.
- \[30\]Y\. Mao, Q\. Wang, M\. Ester, and K\. LiEfficient and accurate kv\-cache management for long\-sequence llms\.InES\-FoMo III: 3rd Workshop on Efficient Systems for Foundation Models,Cited by:[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px2.p1.1)\.
- Prabhuet al\.\(2025\)R\. Prabhu, A\. Nayak, J\. Mohan, R\. Ramjee, and A\. PanwarVattention: dynamic memory management for serving llms without pagedattention\.InProceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1,pp\. 1133–1150\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px3.p1.1)\.
- Qinet al\.\(2026\)R\. Qin, Z\. Li, W\. He, J\. Cui, H\. Tang, F\. Ren, T\. Ma, S\. Cai, Y\. Zhang, M\. Zhang,et al\.Mooncake: a kvcache\-centric disaggregated architecture for llm serving\.ACM Transactions on Storage22\(4\),pp\. 1–38\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px3.p1.1)\.
- Ramachandranet al\.\(2026\)A\. Ramachandran, M\. Neseem, C\. Sakr, R\. Venkatesan, B\. Khailany, and T\. KrishnaThinkv: thought\-adaptive kv cache compression for efficient reasoning models\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 110072–110106\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p2.1),[§2](https://arxiv.org/html/2609.03494#S2.p4.1),[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px1.p1.1)\.
- Suet al\.\(2026\)Y\. Su, Z\. Tian, D\. Qiao, Y\. Zhou, J\. Li, and M\. ZhangLongFlow: efficient kv cache compression for reasoning models\.arXiv preprint arXiv:2603\.11504\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px2.p1.1)\.
- Tanget al\.\(2024\)J\. Tang, Y\. Zhao, K\. Zhu, G\. Xiao, B\. Kasikci, and S\. HanQuest: query\-aware sparsity for efficient long\-context llm inference\.arXiv preprint arXiv:2406\.10774\.Cited by:[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px1.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p1.1)\.
- Xiaoet al\.\(2025\)G\. Xiao, J\. Tang, J\. Zuo, J\. Guo, S\. Yang, H\. Tang, Y\. Fu, and S\. HanDuoattention: efficient long\-context llm inference with retrieval and streaming heads\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 37228–37253\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.
- Xiaoet al\.\(2024\)G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. LewisEfficient streaming language models with attention sinks\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 21875–21895\.Cited by:[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px1.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§5\.1](https://arxiv.org/html/2609.03494#S5.SS1.SSS0.Px1.p1.1)\.
- Yanget al\.\(2024\)J\. Y\. Yang, B\. Kim, J\. Bae, B\. Kwon, G\. Park, E\. Yang, S\. J\. Kwon, and D\. LeeNo token left behind: reliable kv cache compression via importance\-aware mixed precision quantization\.arXiv preprint arXiv:2402\.18096\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.
- Yuanet al\.\(2025\)J\. Yuan, H\. Gao, D\. Dai, J\. Luo, L\. Zhao, Z\. Zhang, Z\. Xie, Y\. Wei, L\. Wang, Z\. Xiao,et al\.Native sparse attention: hardware\-aligned and natively trainable sparse attention\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 23078–23097\.Cited by:[§1](https://arxiv.org/html/2609.03494#S1.p1.1)\.
- Zhanget al\.\(2026\)H\. Zhang, H\. Zhang, X\. Ma, J\. Zhang, and S\. GuoLazyeviction: lagged kv eviction with attention pattern observation for efficient long reasoning\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 36335–36352\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2023\)Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. Ré, C\. Barrett,et al\.H2o: heavy\-hitter oracle for efficient generative inference of large language models\.Advances in neural information processing systems36,pp\. 34661–34710\.Cited by:[§6](https://arxiv.org/html/2609.03494#S6.SS0.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2026\)J\. Zhao, Z\. Fang, S\. Li, S\. Yang, and S\. HeBuzz: beehive\-structured sparse kv cache with segmented heavy hitters for efficient llm inference\.ACM Transactions on Intelligent Systems and Technology17\(4\),pp\. 1–22\.Cited by:[Appendix B](https://arxiv.org/html/2609.03494#A2.SS0.SSS0.Px1.p1.1)\.

## Appendix

## Appendix AOverview of Mathematical Notation

For clarity, Table[3](https://arxiv.org/html/2609.03494#A1.T3)summarizes the key notation used in the theoretical analysis and the GrowPage capacity\-control framework\.

Table 3:Summary of key notation used in the paper\.SymbolDescriptionBi⋆B\_\{i\}^\{\\star\}Minimum sufficient KV budget of requestii𝐚t\\mathbf\{a\}\_\{t\}Attention distribution over historical KV states at decoding stepttDp​\(𝐚t\)D\_\{p\}\(\\mathbf\{a\}\_\{t\}\)Minimum number of KV states required to cover attention massppSSSubset of retained historical KV statesmt​\(S\)m\_\{t\}\(S\)Attention mass retained by subsetSS𝐨t,𝐨^t​\(S\)\\mathbf\{o\}\_\{t\},\\widehat\{\\mathbf\{o\}\}\_\{t\}\(S\)Full\-cache and subset\-renormalized attention outputsCδwc​\(𝐚t\)C\_\{\\delta\}^\{\\mathrm\{wc\}\}\(\\mathbf\{a\}\_\{t\}\)Minimum worst\-case KV capacity under approximation toleranceδ\\delta𝐪t\\mathbf\{q\}\_\{t\}Current normalized pre\-RoPE query𝐪¯tS,𝐪¯tL\\bar\{\\mathbf\{q\}\}\_\{t\}^\{S\},\\bar\{\\mathbf\{q\}\}\_\{t\}^\{L\}Short\- and long\-timescale EMA query summariesβS,βL\\beta\_\{S\},\\beta\_\{L\}EMA decay factors for the two temporal query summaries𝒞t\\mathcal\{C\}\_\{t\}Historical candidate KV set for online demand estimationKt,ℓ,hxK\_\{t,\\ell,h\}^\{x\}Top\-ppworking\-set size for temporal viewx∈\{S,L\}x\\in\\\{S,L\\\}rtS,rtLr\_\{t\}^\{S\},r\_\{t\}^\{L\}Request\-level short\- and long\-timescale working\-set ratiosΔt\\Delta\_\{t\}Demand\-trend signal, defined asrtS−rtLr\_\{t\}^\{S\}\-r\_\{t\}^\{L\}τ\\tauThreshold controlling compression versus capacity growthsi,ℓ,hs\_\{i,\\ell,h\}KV importance score for tokeniiat layerℓ\\elland headhhPtP\_\{t\}Number of physical KV pages allocated to the requestbbNumber of KV\-token slots in one physical pageℛt\\mathcal\{R\}\_\{t\}Protected recent\-token set excluded from compressionKtK\_\{t\}Historical KV retention budget at a compression eventklock\_\{\\mathrm\{loc\}\}Block\-local KV retention quota

## Appendix BExtended Related Works

#### Fine\-Grained KV Compression\.

Beyond the representative KV\-eviction methods discussed in the main text, a broad body of work explores finer\-grained cache adaptation and alternative compression dimensions\. Scissorhands\([Liu et al\., 2023](https://arxiv.org/html/2609.03494#bib.bib23)\)exploits the persistence of token importance to probabilistically retain historically influential KV states, while Keyformer\([Adnan et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib24)\)identifies a compact set of key tokens that dominate attention during generation\. Rather than applying a uniform policy to all attention heads, FastGen\([Ge et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib25)\)profiles their attention structures and assigns different cache strategies accordingly, and DuoAttention\([Xiao et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib26)\)separates retrieval heads requiring long\-context access from streaming heads that can operate with compact caches\. NACL\([Chen et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib27)\)combines proxy\-token\-based importance with randomized eviction, whereas BUZZ\([Zhao et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib28)\)preserves recent context together with segmented historical heavy hitters\. A complementary direction reduces the representation cost of retained KV states\. KIVI\([Liu et al\., 2024c](https://arxiv.org/html/2609.03494#bib.bib29)\)introduces asymmetric low\-bit quantization tailored to the distinct distributions of keys and values, and KVQuant\([Hooper et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib30)\)combines pre\-RoPE key quantization, non\-uniform datatypes, and explicit outlier handling\. MiKV\([Yang et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib31)\)preserves important KV states at higher precision while representing less important states at lower precision, and GEAR\([Kang et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib32)\)combines quantization with low\-rank and sparse error correction\. Cross\-layer redundancy has also been exploited by MiniCache\([Liu et al\., 2024a](https://arxiv.org/html/2609.03494#bib.bib33)\), which merges similar KV states across neighboring layers\. These approaches adapt*which*states are represented and*how*they are represented, but do not directly address online request\-level decisions about how much total KV capacity should be allocated as generation evolves\.

#### KV Compression for Long Reasoning\.

Recent work has increasingly specialized KV\-cache compression for long\-output reasoning, where token importance can change substantially over an extended generation trajectory\. LazyEviction\([Zhang et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib34)\)identifies*token importance recurrence*, observing that previously unimportant tokens may regain high attention after many decoding steps, and therefore delays irreversible eviction through an observation window\. DesireKV\([Cheng et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib35)\)jointly considers attention\-derived importance and quantization sensitivity, selectively retaining, quantizing, or evicting reasoning tokens according to their different roles\. LongFlow\([Su et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib36)\)derives token\-importance estimates directly from intermediate attention computations and co\-designs them with fused kernels to reduce the overhead of continuous importance evaluation during long reasoning\. InfoKV\([Kai et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib37)\)further argues that attention alone may overlook tokens with long\-range future influence and supplements attention scores with information\-theoretic signals related to predictive uncertainty\. Kara\([Han et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib38)\)restricts compression to a sliding window of recently generated reasoning context and develops a serving implementation compatible with paged KV management\. REAL\([Li et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib39)\)instead analyzes heterogeneous attention\-head behaviors in both successful and failed retrieval\-reasoning cases to construct more robust eviction policies\. Together, these studies reinforce that reasoning\-time KV importance is highly dynamic\. Nevertheless, their adaptation primarily concerns importance estimation, eviction timing, token grouping, or representation precision; GrowPage operates on an orthogonal dimension by adapting the*total per\-request capacity*according to the evolving demand of the request\.

#### System\-Level KV Management\.

Another related direction optimizes where and how KV states are physically stored and transferred in serving systems\. vAttention\([Prabhu et al\., 2025](https://arxiv.org/html/2609.03494#bib.bib40)\)uses CUDA virtual memory to support dynamic physical\-memory allocation while retaining a contiguous virtual KV layout, illustrating that the physical storage backing an active sequence can be managed independently from its logical address space\. For memory hierarchies extending beyond GPU memory, InfiniGen\([Lee et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib41)\)predicts important KV entries and selectively fetches them from CPU memory, thereby reducing the transfer overhead of conventional KV offloading\. CacheGen\([Liu et al\., 2024b](https://arxiv.org/html/2609.03494#bib.bib42)\)compresses reusable KV caches into compact bitstreams for efficient context loading and adapts compression to network conditions\. At cluster scale, MemServe\([Hu et al\., 2024](https://arxiv.org/html/2609.03494#bib.bib43)\)introduces an elastic distributed memory pool for context caching and disaggregated inference, while Mooncake\([Qin et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib44)\)builds a KV\-cache\-centric serving architecture that exploits heterogeneous CPU, DRAM, SSD, and network resources together with a global scheduler\. These systems mainly improve KV placement, movement, reuse, or physical memory provisioning\. GrowPage addresses a different but complementary problem: before allocating additional GPU\-resident KV storage to an active reasoning request, it estimates whether the request’s evolving attention demand justifies expanding its logical cache capacity\. This demand\-aware capacity control allows the page\-based memory manager to respond not only to sequence growth, but also to heterogeneous and time\-varying information requirements\.

## Appendix CProof of Theorem[3\.1](https://arxiv.org/html/2609.03494#S3.Thmtheorem1)

For a retained subsetSS, let

m⁡\(S\)=∑i∈Sat,i\.m\(S\)=\\sum\_\{i\\in S\}a\_\{t,i\}\.The difference between the full and renormalized attention outputs is

𝐨t−𝐨^t​\(S\)=∑i∉Sat,i​𝐯i−1−m⁡\(S\)m⁡\(S\)​∑i∈Sat,i​𝐯i\.\\mathbf\{o\}\_\{t\}\-\\widehat\{\\mathbf\{o\}\}\_\{t\}\(S\)=\\sum\_\{i\\notin S\}a\_\{t,i\}\\mathbf\{v\}\_\{i\}\-\\frac\{1\-m\(S\)\}\{m\(S\)\}\\sum\_\{i\\in S\}a\_\{t,i\}\\mathbf\{v\}\_\{i\}\.Using‖𝐯i‖2≤Vmax\\\|\\mathbf\{v\}\_\{i\}\\\|\_\{2\}\\leq V\_\{\\max\}and the triangle inequality,

‖𝐨t−𝐨^t​\(S\)‖2\\displaystyle\\left\\\|\\mathbf\{o\}\_\{t\}\-\\widehat\{\\mathbf\{o\}\}\_\{t\}\(S\)\\right\\\|\_\{2\}≤∑i∉Sat,i​Vmax\+1−m⁡\(S\)m⁡\(S\)​∑i∈Sat,i​Vmax\\displaystyle\\leq\\sum\_\{i\\notin S\}a\_\{t,i\}V\_\{\\max\}\+\\frac\{1\-m\(S\)\}\{m\(S\)\}\\sum\_\{i\\in S\}a\_\{t,i\}V\_\{\\max\}=2​Vmax​\(1−m⁡\(S\)\)\.\\displaystyle=2V\_\{\\max\}\(1\-m\(S\)\)\.
This bound is tight\. Let𝐮\\mathbf\{u\}be any unit vector and choose𝐯i=−Vmax​𝐮\\mathbf\{v\}\_\{i\}=\-V\_\{\\max\}\\mathbf\{u\}fori∈Si\\in Sand𝐯i=Vmax​𝐮\\mathbf\{v\}\_\{i\}=V\_\{\\max\}\\mathbf\{u\}fori∉Si\\notin S\. Then

‖𝐨t−𝐨^t​\(S\)‖2=2​Vmax​\(1−m⁡\(S\)\)\.\\left\\\|\\mathbf\{o\}\_\{t\}\-\\widehat\{\\mathbf\{o\}\}\_\{t\}\(S\)\\right\\\|\_\{2\}=2V\_\{\\max\}\(1\-m\(S\)\)\.Hence,

sup‖𝐯i‖2≤Vmax‖𝐨t−𝐨^t​\(S\)‖2=2​Vmax​\(1−m⁡\(S\)\)\.\\sup\_\{\\\|\\mathbf\{v\}\_\{i\}\\\|\_\{2\}\\leq V\_\{\\max\}\}\\left\\\|\\mathbf\{o\}\_\{t\}\-\\widehat\{\\mathbf\{o\}\}\_\{t\}\(S\)\\right\\\|\_\{2\}=2V\_\{\\max\}\(1\-m\(S\)\)\.Therefore, guaranteeing error at mostδ\\deltafor all admissible value vectors is equivalent to

m⁡\(S\)≥1−δ2​Vmax\.m\(S\)\\geq 1\-\\frac\{\\delta\}\{2V\_\{\\max\}\}\.Among all subsets of cardinalitykk, the maximum retained attention mass is obtained by selecting thekklargest attention weights\. Thus, the minimum cardinality satisfying the above condition is

Cδwc​\(𝐚t\)=D1−δ/\(2​Vmax\)​\(𝐚t\),C\_\{\\delta\}^\{\\mathrm\{wc\}\}\(\\mathbf\{a\}\_\{t\}\)=D\_\{1\-\\delta/\(2V\_\{\\max\}\)\}\(\\mathbf\{a\}\_\{t\}\),which proves the theorem\.

Figure 9:Accuracy–throughput trade\-offs under different demand thresholdsτ\\tauon Qwen3\-8B with AMC23 and AIME24\.
## Appendix DHyperparameter Sensitivity

### D\.1Effect of Demand Threshold

We further study the sensitivity of GrowPage to the demand thresholdτ\\tau, which controls the trade\-off between KV compression and capacity expansion\. As shown by the counterfactual analysis in Figure[7](https://arxiv.org/html/2609.03494#S5.F7), the benefit of allocating additional KV capacity changes noticeably asΔt\\Delta\_\{t\}crosses the region around zero\. We therefore focus on the fine\-grained interval around\[−0\.005,0\.005\]\[\-0\.005,0\.005\]and varyτ\\taufrom−0\.003\-0\.003to0\.0030\.003with a step size of0\.0010\.001\. All experiments are conducted on Qwen3\-8B using AMC23 and AIME24\. Figure[9](https://arxiv.org/html/2609.03494#A3.F9)shows thatτ\\tauprovides a direct mechanism for controlling the accuracy–throughput operating point of GrowPage\. A smaller threshold makes capacity expansion more permissive, causing more requests to acquire additional KV pages\. This preserves a larger historical working set and generally improves reasoning accuracy, but increases per\-request KV consumption and consequently reduces serving throughput\. Conversely, a largerτ\\taumakes GrowPage more conservative about expansion, resulting in more frequent*Compress & Hold*decisions and higher throughput at the cost of more aggressive information removal\. Across both AMC23 and AIME24,τ=0\\tau=0provides a stable operating point with a balanced trade\-off between reasoning accuracy and serving throughput\. We therefore adoptτ=0\\tau=0as the default threshold throughout our experiments\.

Table 4:Effect of the local retention quotaklock\_\{\\mathrm\{loc\}\}on Qwen3\-8B\.klock\_\{\\mathrm\{loc\}\}AMC23AIME24Pass@1Pass@11690\.372\.43290\.973\.16491\.473\.812890\.672\.8

### D\.2Effect of Local Retention Quota

As described in Section[3\.3](https://arxiv.org/html/2609.03494#S3.SS3),klock\_\{\\mathrm\{loc\}\}specifies the minimum number of tokens retained from each historical block before the remaining budget is filled by global importance\-based selection\. We evaluate its sensitivity on Qwen3\-8B using AMC23 and AIME24\. We report only Pass@1 because differentklock\_\{\\mathrm\{loc\}\}values yield nearly identical TPS;klock\_\{\\mathrm\{loc\}\}mainly affects which KV states are retained rather than the total retained capacity, and therefore primarily influences reasoning accuracy\. As shown in Table[4](https://arxiv.org/html/2609.03494#A4.T4.fig1), performance first improves asklock\_\{\\mathrm\{loc\}\}increases from 16 to 64, indicating that sufficient block\-local coverage helps prevent globally important regions from dominating the retained KV states\. However, further increasingklock\_\{\\mathrm\{loc\}\}to 128 slightly degrades accuracy, since an overly large local quota leaves less flexibility for global selection and forces more tokens to be preserved from relatively unimportant blocks\. Overall,kloc=64k\_\{\\mathrm\{loc\}\}=64achieves the best performance on both benchmarks, and we therefore use it as the default setting throughout our experiments\.

## Appendix ESystem Implementation Details

This section provides additional implementation details for the system integration described in Section[4](https://arxiv.org/html/2609.03494#S4)\. We first describe the runtime management and computation of the dual\-timescale query states\. Our support for prefix caching and asynchronous KV compression largely follows the system design of Zipage\([Liao et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib14)\), with the compression target adapted to GrowPage’s dynamically determined page capacity\.

### E\.1Dual\-Timescale Query Management and Online Demand Estimation

#### Pre\-allocated query\-state pool\.

To avoid request\-level GPU allocation and deallocation, GrowPage pre\-allocates a fixed query\-state tensor when the inference engine is initialized\. Its layout is

𝒬∈ℝL×M×2×Hq×dh,\\mathcal\{Q\}\\in\\mathbb\{R\}^\{L\\times M\\times 2\\times H\_\{q\}\\times d\_\{h\}\},\(11\)whereLLis the number of attention layers,MMis the maximum number of query slots,HqH\_\{q\}is the number of query heads per GPU, anddhd\_\{h\}is the head dimension\. The third dimension stores the short\- and long\-timescale EMA queries\. We setMMto the serving engine’s maximum number of concurrent sequences \(max\_num\_seqs\), which is 512 by default\.

Query slots are bound to requests only when they enter the running queue\. The scheduler maintains a pool of free slot identifiers and assigns one identifier to each newly admitted running request; requests waiting to be scheduled therefore consume no query\-state slot\. When a request finishes, its identifier is returned to the free pool without releasing the underlying GPU tensor\. The same procedure is applied when a request is preempted; if it is scheduled again, it acquires a new slot and reinitializes its summaries\. This slot\-based design avoids frequent CUDA memory allocation and allows the pre\-allocated storage to be continuously reused across requests\. The resulting auxiliary memory is fixed with respect to sequence length and scales as2​M​L​Hq​dh2MLH\_\{q\}d\_\{h\}elements\.

#### Initialization and online update\.

The query summaries are updated only during decoding\. For the first decoding step of a newly assigned slot, both temporal states are initialized directly from the current normalized pre\-RoPE query,𝐪¯S=𝐪¯L=𝐪t\\bar\{\\mathbf\{q\}\}^\{S\}=\\bar\{\\mathbf\{q\}\}^\{L\}=\\mathbf\{q\}\_\{t\}, thereby overwriting any stale values left by the previous owner of the slot without explicitly clearing the tensor\. Subsequent decoding steps update the two states in place according to Eq\.[6](https://arxiv.org/html/2609.03494#S3.E6), usingβS=0\.9\\beta\_\{S\}=0\.9andβL=0\.999\\beta\_\{L\}=0\.999\. No historical query sequence is materialized: each request stores only the two running summaries\.

#### Demand estimation at capacity boundaries\.

The regular decoding path only updates the two EMA states\. The remaining demand\-estimation operations are executed when a request reaches a capacity boundary\. Because EMA averaging changes the magnitude of the stored queries, we first calibrate each temporal summary to the RMS scale of the current normalized query\. Forx∈\{S,L\}x\\in\\\{S,L\\\}, we compute

γtx=RMS⁡\(𝐪t\)RMS⁡\(𝐪¯tx\),𝐪~tx=γtx​𝐪¯tx\.\\gamma\_\{t\}^\{x\}=\\frac\{\\operatorname\{RMS\}\(\\mathbf\{q\}\_\{t\}\)\}\{\\operatorname\{RMS\}\(\\bar\{\\mathbf\{q\}\}\_\{t\}^\{x\}\)\},\\qquad\\widetilde\{\\mathbf\{q\}\}\_\{t\}^\{x\}=\\gamma\_\{t\}^\{x\}\\bar\{\\mathbf\{q\}\}\_\{t\}^\{x\}\.\(12\)The calibrated queries are then transformed by RoPE using the*current*decoding position\. This preserves the temporal information accumulated in the common pre\-RoPE space while aligning both summaries with the positional frame used by the current attention computation\.

We exclude the most recent physical KV block from the historical candidate set, corresponding to 256 tokens under our default block size, and independently compute the short\- and long\-timescale attention distributions over the remaining historical keys\. The native GQA head mapping is preserved when matching query heads to KV heads\. For each temporal view, the attention weights are sorted and accumulated until reaching the Top\-ppcoverage threshold \(p=0\.99p=0\.99\), yielding the working\-set ratios in Eq\.[7](https://arxiv.org/html/2609.03494#S3.E7)\. These ratios are then aggregated across layers and KV heads according to Eq\.[8](https://arxiv.org/html/2609.03494#S3.E8), producing the request\-level signalΔt\\Delta\_\{t\}\. Finally,Δt\\Delta\_\{t\}is compared with the demand thresholdτ\\tauto select between*Compress & Hold*and*Grow by One Page*\. Thus, attention scoring, sorting, and working\-set estimation are kept off the critical path of ordinary decoding steps and are invoked only when a capacity decision is required\.

### E\.2Prefix\-Caching Support

GrowPage inherits the prefix\-preserving compression mechanism of Zipage\([Liao et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib14)\)\. Prefix sharing is tracked at physical\-block granularity using reference counts, and a block referenced by multiple requests is never overwritten in place during compression\. Instead, compression writes the retained KV states into a set of target blocks while leaving shared prefix blocks intact for the other requests\. We adapt the target size to GrowPage’s dynamic capacity\. For a request currently occupyingPtP\_\{t\}physical pages,*Compress & Hold*keeps the physical allocation unchanged but compacts the retained KV states into at most\(Pt−1\)​b\(P\_\{t\}\-1\)boccupied slots, leaving one page worth of free slots for subsequent decoding\. If shared prefix blocks occupy part of the original allocation, sufficient new target blocks are allocated to replace those shared blocks, while reusable non\-shared blocks are retained whenever possible\. After compaction, the request releases its references to the original shared blocks and their reference counts are updated accordingly\.*Grow by One Page*does not modify the shared prefix and simply appends the newly allocated page to the request’s block table\. As in Zipage, temporary target\-block allocation may precede the release of shared blocks; under severe memory pressure, the scheduler can therefore invoke preemption to avoid allocation deadlock\.

### E\.3Asynchronous Decoding and Compression

We also adopt the asynchronous decoding–compression pipeline of Zipage\([Liao et al\., 2026](https://arxiv.org/html/2609.03494#bib.bib14)\)\. At a given scheduling step, only a subset of requests reaching capacity boundaries require KV compaction\. Executing compression synchronously would stall the remaining decode\-ready requests and serialize a relatively small compression batch with the main decoding workload\. Instead, requests selected for*Compress & Hold*enter the asynchronous compression path, while other ready requests continue decoding\. Once compaction completes, the compressed requests rejoin a subsequent decoding batch\. Requests taking*Grow by One Page*require only page allocation and block\-table update and can resume decoding after the new page is installed\. This preserves Zipage’s overlap between decoding and KV compaction while allowing GrowPage’s capacity decisions to remain request\-specific\.

## Appendix FExperimental Details

### F\.1Datasets and Evaluation Protocol

We evaluate GrowPage on five reasoning benchmarks covering mathematical reasoning and code generation\. Table[5](https://arxiv.org/html/2609.03494#A6.T5)summarizes the dataset size, number of independent generations per question, maximum output length, and the KV budget used by fixed\-budget compression baselines\. Unless otherwise specified, we use a sampling temperature of 0\.6\. For benchmarks with multiple generations per question, each generation is evaluated independently and the reported Pass@1 is averaged over all sampled outputs\. The dataset\-specific KV budgets follow the configurations used in our fixed\-budget comparisons; GrowPage does not use these predefined budgets and instead adapts its KV capacity online\.

Table 5:Evaluation settings for individual reasoning benchmarks\. The KV budget denotes the per\-request capacity used by fixed\-budget compression baselines\.WorkloadNumber of QuestionsSample TimesMax Output LengthKV BudgetAMC234032163844096AIME243032327684096MATH5005008163844096GSM8K13194163844096LiveCodeBench v14008163848192

The sampling multiplicity is chosen according to the size and difficulty of each benchmark\. For the smaller competition\-level mathematics datasets AMC23 and AIME24, we use 32 independent generations per question to reduce sampling variance\. For the larger MATH500, GSM8K, and LiveCodeBench datasets, fewer repetitions provide sufficient coverage while keeping the total evaluation cost manageable\. The larger KV budget for LiveCodeBench reflects its substantially longer code\-generation trajectories\.

### F\.2Mixed\-Workload Construction

For the heterogeneous\-workload experiment in Figure[5](https://arxiv.org/html/2609.03494#S5.F5), we combine AMC23, AIME24, and GSM8K into a single serving workload\. Because these datasets differ substantially in the number of available questions, directly using each question once would cause GSM8K to dominate the workload\. We therefore use different sampling multiplicities to obtain a more balanced contribution from the three datasets\. The resulting workload contains 3559 requests, as summarized in Table[6](https://arxiv.org/html/2609.03494#A6.T6.fig1)\. All requests use a maximum output length of 32768 tokens\.

Table 6:Composition of the mixed reasoning workload\.DatasetNumber of QuestionsSamplesTotal RequestsAMC2340321280AIME243032960GSM8K131911319

This construction prevents the much larger GSM8K dataset from overwhelmingly dominating the mixed workload and yields comparable numbers of requests from the three constituent benchmarks\. Unlike the individual benchmark comparison in Table[5](https://arxiv.org/html/2609.03494#A6.T5), we do not assign a single fixed KV budget to this workload, since the purpose of this experiment is to explicitly evaluate Zipage across multiple fixed budgets and compare its resulting accuracy–throughput Pareto frontier with GrowPage\.

## Appendix GExtended Analysis of Main Results

### G\.1Additional Mixed\-Workload Results

To examine whether the mixed\-workload behavior in Figure[5](https://arxiv.org/html/2609.03494#S5.F5)generalizes across model architectures, we repeat the same experiment on DeepSeek\-R1\-Distill\-Llama\-8B\. We use the identical mixed workload described in Appendix[F\.2](https://arxiv.org/html/2609.03494#A6.SS2), and compare GrowPage with FullKV and Zipage under fixed KV budgets ranging from 1024 to 8192 tokens\.

Figure 10:Accuracy–throughput trade\-off on the mixed GSM8K, AMC23, and AIME24 workload with DeepSeek\-R1\-Distill\-Llama\-8B\.CCdenotes the average number of requests resident in GPU memory\.As shown in Figure[10](https://arxiv.org/html/2609.03494#A7.F10), DeepSeek\-R1\-Distill\-Llama\-8B exhibits a trend consistent with the Qwen3\-8B results in Figure[5](https://arxiv.org/html/2609.03494#S5.F5)\. Increasing the fixed KV budget of Zipage improves reasoning accuracy but progressively reduces resident concurrency and serving throughput\. GrowPage instead reaches 64\.3% Pass@1 at 2451 tokens/s with an average KV budget of 3234 tokens andC=135C=135\. Compared with Zipage\-4096, GrowPage improves accuracy from 63\.5% to 64\.3% while increasing throughput from 2098 to 2451 tokens/s and resident concurrency from 107 to 135\. Compared with the more conservative Zipage\-8192 setting, GrowPage remains within 0\.6 percentage points in accuracy while achieving1\.75×1\.75\\timeshigher throughput and1\.71×1\.71\\timesmore resident requests\. It also closely matches FullKV accuracy \(64\.3% vs\. 64\.5%\) while providing2\.11×2\.11\\timeshigher throughput\. These results further show that on\-demand KV budgeting achieves a favorable accuracy–efficiency trade\-off across different reasoning\-model architectures rather than being specific to Qwen3\-8B\.

### G\.2Detailed Serving Metrics

Table[7](https://arxiv.org/html/2609.03494#A7.T7)provides additional efficiency metrics for the main comparison in Table[1](https://arxiv.org/html/2609.03494#S5.T1)\. We report the total inference time for each benchmark and the time per output token \(TPOT\)\. The total time is converted from seconds to hours for readability\. Since different benchmarks contain different numbers of sampled requests, total inference time should be compared across methods within the same benchmark rather than across datasets\.

Table 7:Extended efficiency comparison on reasoning benchmarks\. We report total inference time \(hours\) and time per output token \(TPOT, ms/token\); lower is better for both metrics\.Total Inference Time \(h\)↓\\downarrowTPOT \(ms/token\)↓\\downarrowMethodGSM8KMATH500AMC23AIME24LiveCodeAvg\.GSM8KMATH500AMC23AIME24LiveCodeAvg\.DeepSeek\-R1\-Distill\-Llama\-8BFullKV \(vLLM\)0\.613\.302\.005\.278\.103\.86117522994783579599\.0FullKV \(nano\-vLLM\)0\.643\.442\.065\.258\.513\.981235461027814598621\.6MorphKV\(ICML’25\)3\.9525\.9916\.0944\.4966\.1331\.33136157156241248187\.6R\-KV\(NeurIPS’25\)4\.2525\.3415\.5240\.7764\.3830\.05151169167254272202\.6G\-KV\(arXiv’25\)4\.1024\.5914\.7837\.3360\.8128\.32145163162247260195\.4Zipage\(ACL’26\)0\.552\.631\.233\.487\.753\.1381128127240225160\.2GrowPage \(Ours\)0\.492\.181\.192\.935\.752\.5178110112207155132\.4Qwen3\-8BFullKV \(vLLM\)1\.234\.092\.666\.586\.354\.18139381826938496556\.0FullKV \(nano\-vLLM\)1\.324\.332\.827\.016\.734\.441484048721003525590\.4MorphKV\(ICML’25\)8\.2917\.0810\.7635\.6426\.6919\.69170165169250232197\.2R\-KV\(NeurIPS’25\)8\.4116\.6310\.1330\.9425\.6818\.36176181181263254211\.0G\-KV\(arXiv’25\)8\.6316\.0110\.2331\.2524\.3318\.09181174176256246206\.6Zipage\(ACL’26\)1\.063\.111\.634\.195\.393\.0879141138249177156\.8GrowPage \(Ours\)1\.042\.601\.423\.774\.672\.7075106122207145131\.0

#### End\-to\-end efficiency\.

Table[7](https://arxiv.org/html/2609.03494#A7.T7)further confirms that the throughput improvements of GrowPage translate into shorter workload completion time\. GrowPage achieves the lowest total inference time across all five benchmarks on both models\. Compared with Zipage, its average inference time decreases from 3\.13 to 2\.51 hours on DeepSeek\-R1\-Distill\-Llama\-8B and from 3\.08 to 2\.70 hours on Qwen3\-8B\. The improvement becomes particularly pronounced on longer and more demanding workloads; for example, on LiveCodeBench with DeepSeek\-R1\-Distill\-Llama\-8B, GrowPage reduces the total inference time from 7\.75 to 5\.75 hours\. These results indicate that adapting per\-request KV capacity reduces unnecessary memory residency and enables the serving engine to complete batched reasoning workloads more efficiently\.

#### Per\-token decoding efficiency\.

GrowPage also consistently achieves the lowest TPOT across all evaluated datasets\. Averaged over the five benchmarks, GrowPage reduces TPOT from 160\.2 to 132\.4 ms/token relative to Zipage on DeepSeek\-R1\-Distill\-Llama\-8B, and from 156\.8 to 131\.0 ms/token on Qwen3\-8B\. This suggests that on\-demand capacity control not only increases aggregate concurrency but also reduces the attention cost associated with unnecessarily large KV working sets\. Interestingly, MorphKV, R\-KV, and G\-KV achieve substantially lower TPOT than FullKV on several long\-generation workloads, yet exhibit much lower aggregate TPS in Table[1](https://arxiv.org/html/2609.03494#S5.T1)\. This discrepancy highlights that reducing per\-token computation alone does not necessarily translate into higher serving throughput: without system\-level integration with continuous batching and paged memory management, algorithmic KV compression may fail to efficiently exploit the freed memory under high concurrency\. GrowPage combines both advantages, maintaining low per\-token latency while translating KV savings into higher aggregate serving throughput\.

## Appendix HAdditional Capacity\-Control Analysis

### H\.1Alternative Capacity\-Control Policies

We further investigate whether the performance gains of GrowPage arise specifically from its demand\-guided capacity control, rather than merely from changing the KV capacity or the overall frequency of compression and growth\. We compare the default GrowPage policy with four alternative capacity\-control strategies on Qwen3\-8B with AMC23\.

#### Compared policies\.

GrowPage\-Fixeddisables capacity expansion and maintains a fixed KV budget of 4096 tokens; whenever the current capacity is exhausted, it applies*Compress & Hold*\.GrowPage\-Randomremoves the dependence onΔt\\Delta\_\{t\}and randomly selects*Compress & Hold*with probability0\.70\.7and*Grow by One Page*otherwise\. The probability is chosen to approximately match the empirical compression frequency of the default GrowPage policy, providing a frequency\-matched control for evaluating whether the timing of capacity decisions matters\.GrowPage\-Inversereverses the default interpretation of the demand signal: it grows whenΔt≤0\\Delta\_\{t\}\\leq 0and compresses whenΔt\>0\\Delta\_\{t\}\>0\. Finally,GrowPage\-Shrinkintroduces explicit bidirectional resizing with three actions:

πshrink​\(Δt\)=\{*Shrink*,Δt<−0\.005,*Compress & Hold*,−0\.005≤Δt≤0,*Grow by One Page*,Δt\>0\.\\pi\_\{\\mathrm\{shrink\}\}\(\\Delta\_\{t\}\)=\\begin\{cases\}\\text\{\\emph\{Shrink\}\},&\\Delta\_\{t\}<\-0\.005,\\\\ \\text\{\\emph\{Compress \\& Hold\}\},&\-0\.005\\leq\\Delta\_\{t\}\\leq 0,\\\\ \\text\{\\emph\{Grow by One Page\}\},&\\Delta\_\{t\}\>0\.\\end\{cases\}\(13\)Here,*Shrink*compacts two pages worth of historical KV states and returns one physical page to the global memory pool, while leaving one page worth of free slots for subsequent decoding\. We impose a minimum KV capacity of 1024 tokens to prevent excessive contraction\. The threshold−0\.005\-0\.005is motivated by the low\-demand region identified in the counterfactual analysis of Figure[7](https://arxiv.org/html/2609.03494#S5.F7)\.

Table 8:Comparison of alternative capacity\-control policies on Qwen3\-8B with AMC23\. Total time denotes the end\-to\-end inference time of the workload\. Lower is better for Avg\. KV Budget, TPOT, and Total Time\.MethodAvg\. KV Budget↓\\downarrowPass@1 \(%\)↑\\uparrowTPS \(tokens/s\)↑\\uparrowTPOT \(ms/token\)↓\\downarrowTotal Time \(h\)↓\\downarrowGrowPage\-Fixed409689\.918741361\.59GrowPage\-Shrink209984\.225461071\.44GrowPage\-Random372888\.320081371\.57GrowPage\-Inverse616790\.813401752\.13Zipage\(ACL’26\)409689\.518051381\.63GrowPage \(Ours\)316491\.422611221\.42

#### Effect of demand\-guided adaptation\.

Table[8](https://arxiv.org/html/2609.03494#A8.T8)shows that dynamically adapting KV capacity is important for obtaining a favorable accuracy–efficiency trade\-off\. Compared withGrowPage\-Fixed, the default policy reduces the average KV budget from 4096 to 3164 tokens while improving TPS from 1874 to 2261 and Pass@1 from 89\.9% to 91\.4%\. This indicates that the benefit cannot be reproduced by simply maintaining a static capacity and repeatedly compressing within it\.

More importantly,GrowPage\-Randomproduces an action frequency close to that of the default policy, but performs substantially worse\. Despite using a similar fraction of compression and growth decisions, the random policy reaches only 88\.3% Pass@1 at 2008 tokens/s, compared with 91\.4% at 2261 tokens/s for GrowPage\. It also maintains a larger average KV budget of 3728 tokens\. This result shows that the gain does not arise merely from the marginal frequency of compression and growth;*when*each action is applied is critical\.

Reversing the demand signal leads to the opposite behavior\.GrowPage\-Inverseexpands much more aggressively, increasing the average KV budget to 6167 tokens and reducing throughput to 1340 tokens/s, while still achieving lower accuracy than the default GrowPage policy\. This result complements the counterfactual analysis in Figure[7](https://arxiv.org/html/2609.03494#S5.F7)and confirms that the direction ofΔt\\Delta\_\{t\}carries meaningful information for capacity allocation rather than serving only as a generic activity signal\.

#### Why not aggressively shrink capacity?

GrowPage\-Shrinkachieves the smallest average KV budget and the highest raw throughput, but its Pass@1 drops sharply to 84\.2%,7\.27\.2percentage points below the default GrowPage policy\. The result exposes the cost of aggressively reclaiming physical pages from active reasoning requests: once additional historical KV states are irreversibly removed, later reasoning stages may again require information that is no longer available\. GrowPage therefore adopts the more conservative*Compress & Hold*operation in the main design, retaining the current physical allocation while creating room for subsequent tokens\. This yields a substantially better balance between memory efficiency, serving throughput, and reasoning accuracy\.

### H\.2System\-Level Event Analysis

To better understand how different capacity\-control policies affect the serving runtime, we further record the number of page\-growth, compression, fallback, and request\-preemption events\. A*fallback*occurs when a requested page expansion cannot be safely satisfied by the global free\-page pool and is therefore converted to compression\. A*preemption*is triggered under severe memory pressure when page expansion fails and the subsequent fallback compression still cannot provide sufficient GPU memory; the scheduler then removes an active decoding request from the running batch, releases its occupied resources, and returns it to the waiting queue for later rescheduling\. ForGrowPage\-Shrink, we separately report explicit*Shrink*events; these events are included in the total compression count recorded by the runtime but are separated from ordinary*Compress & Hold*events in Table[9](https://arxiv.org/html/2609.03494#A8.T9)\.

Table 9:Runtime capacity\-control events on Qwen3\-8B with AMC23\. Grow Ratio is computed over the primary capacity decisions before fallback handling\. ForGrowPage\-Shrink, explicit Shrink events are reported separately from Compress & Hold\.MethodGrowGrow Ratio \(%\)Compress & HoldShrinkFallbackPreemptionsGrowPage\-Fixed00\.022285–019GrowPage\-Shrink1790945\.51254089271034668GrowPage\-Random934530\.421399–9441894GrowPage\-Inverse1988166\.79931–2862997GrowPage \(Ours\)950030\.921203–5971361

#### Decision frequency versus decision timing\.

The runtime statistics provide a particularly controlled comparison between GrowPage andGrowPage\-Random\. The two policies have almost identical primary action distributions: GrowPage grows at 30\.9% of capacity boundaries, while the random policy grows at 30\.4%\. Their total numbers of primary capacity decisions are also nearly identical\. Nevertheless, GrowPage achieves substantially higher accuracy and throughput in Table[8](https://arxiv.org/html/2609.03494#A8.T8)\. This frequency\-matched comparison further demonstrates that the effectiveness of GrowPage comes from usingΔt\\Delta\_\{t\}to determine*where*expansion is needed, rather than merely choosing an appropriate global compression\-to\-growth ratio\. The demand\-guided policy is also accompanied by fewer fallback and preemption events than its random counterpart\.

#### Cost of dynamic capacity adaptation\.

Dynamic capacity adaptation also introduces additional interactions with the global memory manager\. UnlikeGrowPage\-Fixed, which never requests new physical pages and therefore incurs almost no fallback or preemption events, GrowPage triggers 597 fallbacks and 1361 preemptions as requests dynamically compete for additional KV capacity\. This represents a system\-level cost of on\-demand expansion under memory pressure\. Nevertheless, GrowPage still achieves higher throughput \(2261 vs\. 1874 tokens/s\) and accuracy \(91\.4% vs\. 89\.9%\) while using a smaller average KV budget \(3164 vs\. 4096\), indicating that the benefit of demand\-aware capacity allocation outweighs the additional scheduling pressure in our setting\.

#### Capacity over\-expansion and bidirectional resizing\.

The event distribution ofGrowPage\-Inversefurther illustrates the importance of the direction of the demand signal\. Reversing the policy raises the growth ratio from 30\.9% to 66\.7%, leading to substantially larger average KV residency and lower serving throughput\. Although this policy retains more KV information, the additional capacity does not translate into higher reasoning accuracy, indicating that indiscriminate expansion wastes shared GPU memory\.

GrowPage\-Shrinkexhibits a different failure mode\. It triggers 8927 explicit shrink operations while also performing 17909 subsequent growth operations, resulting in substantially more bidirectional capacity transitions\. This aggressive resizing is accompanied by 4668 request preemptions, considerably more than the 1361 observed with the default GrowPage policy\. Together with the significant accuracy degradation in Table[8](https://arxiv.org/html/2609.03494#A8.T8), these results suggest that aggressively reclaiming pages from live reasoning requests can introduce unnecessary capacity churn and irreversible information loss\. The default GrowPage policy therefore favors conservative*Compress & Hold*and incremental growth, which provides more stable online capacity adaptation under PagedAttention\-based serving\.

## Appendix IRequest\-Level Capacity Alignment

Figure[2](https://arxiv.org/html/2609.03494#S2.F2)shows that the minimum sufficient fixed KV budget varies substantially across requests\. We further examine whether GrowPage’s online capacity allocation reflects this request\-level demand heterogeneity\. Specifically, we group requests according to the minimum sufficient budgetBi⋆B\_\{i\}^\{\\star\}identified in Figure[2](https://arxiv.org/html/2609.03494#S2.F2), and rerun the same requests with GrowPage using the same random seed\. For each request, we record its average allocated KV capacity over the decoding trajectory\. Figure[11](https://arxiv.org/html/2609.03494#A9.F11)reports the resulting request\-level capacities together with the mean and 95% confidence interval for each fixed\-budget class\.

Across the populated budget classes, requests requiring larger fixed budgets generally receive larger adaptive capacities from GrowPage\. This trend is particularly clear on LiveCodeBench, where the average adaptive capacity increases consistently from the 512\-token class to the 8K\-token class\. AMC23 shows the same tendency over its populated 512, 1K, and 2K classes\. AIME24 also exhibits an overall positive relationship, although the sparsely populated 4K and 8K classes show larger variance\. These results indicate that GrowPage does not allocate capacity uniformly across requests, but instead adapts its runtime KV allocation in accordance with their underlying memory demands\.

Importantly, the adaptive capacity is not expected to numerically matchBi⋆B\_\{i\}^\{\\star\}\. The latter denotes the minimum*fixed*capacity required throughout an entire generation, whereas GrowPage changes the allocated capacity online as demand evolves\. Consequently, a request belonging to a high\-budget class can still maintain a substantially smaller average capacity by using additional pages only during high\-demand stages\. This distinction highlights the advantage of on\-demand budgeting over static reservation\.

We also observe that requests categorized as*Unresolved*tend to receive relatively large adaptive capacities\. Since these requests are not consistently solved under any tested fixed budget, this behavior suggests that GrowPage responds to their elevated attention demand rather than to the eventual correctness outcome\. Overall, the alignment betweenBi⋆B\_\{i\}^\{\\star\}and GrowPage’s adaptive allocation provides request\-level evidence that the proposed demand signal translates heterogeneous reasoning demands into differentiated KV capacity\.

Figure 11:Alignment between minimum sufficient fixed\-budget classes and GrowPage’s adaptive KV capacity\. Requests are grouped by the minimum sufficient budget identified in Figure[2](https://arxiv.org/html/2609.03494#S2.F2), and each point shows the average capacity allocated by GrowPage to the same request under the same random seed\. Markers and error bars denote the class mean and 95% confidence interval\.