Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches

# Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches
Source: [https://arxiv.org/html/2607.20538](https://arxiv.org/html/2607.20538)
###### Abstract

Long\-context Transformer inference increasingly relies on KV\-cache compression or quantization\. Rotation and transform\-coding results indicate that the channel basis of each key/value vector can affect how faithfully a fixed backend preserves model behavior\. We introduce*Codec\-Gauge*, a post\-training cache\-coordinate layer that learns small orthogonal channel transforms, or gauges, around existing backends\. Its frequency\-distribution objective combines a token–channel DCT spectral\-centroid loss with a smooth rate proxy to concentrate KV energy in low\-frequency codec\-facing layouts\. We evaluate actual compression/decompression with measured bytes and rolling compressed\-history scoring\. Across six models at 3, 4, and 6 bits/value, learned gauges reduce zfp KL divergence by44\.0%44\.0\\%on average relative to raw coordinates and outperform random, Hadamard, DCT, and PCA/KLT controls; the same gauges improve quality preservation for block\-uniform and KIVI\-style quantization\. A 27B extension and task\-prompt likelihoods reproduce the quality trend, while serial storage/timing measurements validate the implemented compressed\-cache paths\.

## Introduction

Long\-context Transformer inference is increasingly constrained by the memory and bandwidth cost of KV caches\. During autoregressive decoding, each layer stores past key and value tensors, and this cache grows linearly with batch size, context length, number of layers, number of KV heads, and head dimension\. Attention kernels and memory managers improve access locality, while multi\-query and grouped\-query attention reduce stored KV heads \(\(Shazeer[2019](https://arxiv.org/html/2607.20538#bib.bib1); Ainslieet al\.[2023](https://arxiv.org/html/2607.20538#bib.bib2); Daoet al\.[2022](https://arxiv.org/html/2607.20538#bib.bib3); Kwonet al\.[2023](https://arxiv.org/html/2607.20538#bib.bib4)\)\); nevertheless, the cache remains a large runtime object that must be stored, moved, and repeatedly consumed\. This paper isolates a complementary variable: the coordinate basis presented to a compression or quantization backend\. Codec\-Gauge learns this basis from frozen\-model KV tensors to improve fidelity at the same measured rate while leaving model weights, attention semantics, and backend coding rules unchanged \(Figure[1](https://arxiv.org/html/2607.20538#Sx1.F1)\)\.

![Refer to caption](https://arxiv.org/html/2607.20538v1/x1.png)Figure 1:Overview of Codec\-Gauge\. A one\-time post\-training stage learns orthogonal gauges from frozen\-model KV tensors\. At inference, the gauge maps KV tensors into codec\-facing coordinates before compression or quantization, and the inverse gauge restores recovered KV states before attention\.A large body of work reduces this cost by changing cache precision, cache retention, or model representation \(\(Liuet al\.[2024c](https://arxiv.org/html/2607.20538#bib.bib5); Hooperet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib6); Kanget al\.[2024](https://arxiv.org/html/2607.20538#bib.bib7); Yanget al\.[2024](https://arxiv.org/html/2607.20538#bib.bib8); Linet al\.[2025b](https://arxiv.org/html/2607.20538#bib.bib9); Zhanget al\.[2023](https://arxiv.org/html/2607.20538#bib.bib17); Xiaoet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib18); Liet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib20); DeepSeek\-AI[2024](https://arxiv.org/html/2607.20538#bib.bib29); Changet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib30); Nawrotet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib33); Gelberget al\.[2026](https://arxiv.org/html/2607.20538#bib.bib34)\)\)\. Codec\-Gauge is motivated by two related observations: transform coding shows that a basis change can concentrate signal energy into coefficients that are easier to preserve at a fixed rate \(\(Ahmedet al\.[1974](https://arxiv.org/html/2607.20538#bib.bib39); Lindstrom[2014](https://arxiv.org/html/2607.20538#bib.bib38)\)\), and LLM quantization shows that orthogonal channel rotations can change low\-bit error behavior without changing the represented function \(\(Cheeet al\.[2023](https://arxiv.org/html/2607.20538#bib.bib10); Tsenget al\.[2024](https://arxiv.org/html/2607.20538#bib.bib11); Ashkbooset al\.[2024](https://arxiv.org/html/2607.20538#bib.bib12); Liuet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib13); Suet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib15); Saxena and Roy[2025](https://arxiv.org/html/2607.20538#bib.bib16)\)\)\. Together, these observations motivate learning, after pretraining, the representation geometry presented to a fixed backend\. Although model channels have no native spatial order, a codec consumes a concrete memory layout, so the learned channel basis can change the smoothness and coefficient structure of the stored KV field\.

This paper studies geometry as a first\-class compression variable\.*Codec\-Gauge*learns an orthogonal gauge over KV channel groups so that existing compression and quantization backends see a more compression\-friendly representation, while model weights, attention semantics, and backend implementations remain fixed\. By KV coordinates, we mean the channel basis used to represent each key or value vector within a head\. The gauge is an invertible change of this basis: it can mix channel coordinates within each group, but it never mixes tokens, heads, or layers, and it never removes cache entries\. At inference time, cached keys and values are mapped into the learned gauge before compression or quantization and mapped back before attention\. Along a no\-error path, the gauge and inverse preserve the cache up to numerical precision; under lossy backends, the gauge changes the geometry seen by the backend and therefore changes recovery error and quality retained after decompression\.

We use a matched\-rate experimental design\. Within each comparison, the model, token stream, backend, measured bit budget, and evaluator are fixed, and only the cache\-coordinate map is varied\. This makes identity, random orthogonal, fixed transform, data\-derived, and learned gauges comparable under the same backend, so improvements can be attributed to cache geometry rather than to a changed quantizer, eviction rule, packing format, kernel, or scheduler\.

Our main target is fixed\-rate numerical compression of continuous KV tensors\. We use GPU zfp compression as the primary codec because its block\-transform structure makes coefficient concentration and local smoothness directly relevant to reconstruction quality at a fixed rate \(\(Lindstrom[2014](https://arxiv.org/html/2607.20538#bib.bib38)\)\)\. The key technical choice is a frequency\-distribution loss on the codec\-facing token–channel layout: a DCT spectral\-centroid term moves KV energy toward lower frequencies, while a smooth log\-amplitude proxy favors coefficient distributions with fewer significant transform coefficients\. All quality claims use actual compression/decompression and rolling compressed\-history scoring, where newly created KV entries traverse the same gauge and backend path\.

The codec remains outside the training loop: fitting the gauge uses no reconstruction, language\-modeling, logit, or task\-output loss\.

We use coordinate controls to identify the effect of this training objective\. Identity uses the raw cache basis, while random orthogonal gauges capture generic rotation and outlier\-redistribution effects known from quantization \(\(Cheeet al\.[2023](https://arxiv.org/html/2607.20538#bib.bib10); Tsenget al\.[2024](https://arxiv.org/html/2607.20538#bib.bib11); Ashkbooset al\.[2024](https://arxiv.org/html/2607.20538#bib.bib12); Liuet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib13); Sunet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib14); Suet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib15); Saxena and Roy[2025](https://arxiv.org/html/2607.20538#bib.bib16)\)\)\. Hadamard and DCT provide fixed structured transforms, PCA/KLT provides a data\-derived basis, and learned gauges test the proposed frequency\-distribution objective under the same backend and measured rates\. These controls separate codec\-facing gauge learning from both raw\-coordinate compression and generic orthogonal preprocessing\.

#### Backend compatibility\.

The learned gauge is a fixed per\-checkpoint transform, not a per\-token payload, and changes neither a backend’s coding rule nor its bit budget\. It is applied at the KV tensor interface; our measurements use explicit recovery and include that cost in timing, tying quality, storage, and latency to the same implemented path\.

Across six language models and multiple matched bits\-per\-value settings, learned gauges reduce KV reconstruction error and rolling suffix degradation for zfp beyond identity, random, fixed\-transform, and PCA/KLT controls\. In the same trained coordinates, block\-uniform and KIVI\-style quantizers \(\(Liuet al\.[2024c](https://arxiv.org/html/2607.20538#bib.bib5)\)\) also preserve output quality better than their raw\-coordinate versions\. A larger 27B model check and task\-prompt likelihoods reproduce the coordinate effect, while serial context\-length measurements verify realized storage and recovery cost\. For zfp, learned gauges reduce KL divergence, logit MSE, top\-1 flip rate, and KV NRMSE by44\.0%44\.0\\%,43\.3%43\.3\\%,24\.5%24\.5\\%, and18\.3%18\.3\\%at the main operating points\.

This paper makes four contributions\. First, we formulate KV\-cache coordinate geometry as a post\-training variable for improving compression fidelity under fixed backends\. Second, we introduce Codec\-Gauge, an orthogonal cache\-coordinate layer that wraps existing compression and quantization backends without changing attention semantics\. Third, we design a frequency\-distribution objective that trains this layer from frozen KV tensors using codec\-facing token–channel spectral structure, without language\-modeling loss, logit matching, task supervision, or model\-weight updates\. Fourth, we provide paired evidence that learned gauges improve actual zfp GPU compression under matched measured rates and also improve quality retention for block\-uniform and KIVI\-style quantization paths\.

## Related Work

#### KV\-cache quantization and rotations\.

KV\-cache quantization reduces stored key/value precision through asymmetric granularity, outlier handling, correction terms, mixed precision, or serving co\-design \(\(Liuet al\.[2024c](https://arxiv.org/html/2607.20538#bib.bib5); Hooperet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib6); Kanget al\.[2024](https://arxiv.org/html/2607.20538#bib.bib7); Yanget al\.[2024](https://arxiv.org/html/2607.20538#bib.bib8); Linet al\.[2025b](https://arxiv.org/html/2607.20538#bib.bib9)\)\)\. Coordinate transforms are also central to LLM quantization, including incoherence processing, outlier\-removing rotations, learned rotations, distribution flattening, adaptive KV rotations, and Hadamard\-based correction \(\(Cheeet al\.[2023](https://arxiv.org/html/2607.20538#bib.bib10); Tsenget al\.[2024](https://arxiv.org/html/2607.20538#bib.bib11); Ashkbooset al\.[2024](https://arxiv.org/html/2607.20538#bib.bib12); Liuet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib13); Sunet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib14); Suet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib15); Saxena and Roy[2025](https://arxiv.org/html/2607.20538#bib.bib16)\)\)\. Codec\-Gauge uses a cache\-geometric frequency objective for a fixed numerical codec rather than a scalar quantization loss, then reuses the same trained coordinates for scalar low\-bit paths\.

#### Token, page, and dynamic cache selection\.

Another line of work reduces the number of cache entries retained or accessed, using heavy hitters, attention sinks, model\-internal patterns, prompt observations, layer\-wise budgets, or query\-aware sparsity \(\(Zhanget al\.[2023](https://arxiv.org/html/2607.20538#bib.bib17); Xiaoet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib18); Geet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib19); Liet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib20); Caiet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib21); Tanget al\.[2024](https://arxiv.org/html/2607.20538#bib.bib22)\)\)\. InfiniGen, RocketKV, and CacheGen emphasize dynamic cache management, multi\-stage compression, or cache streaming \(\(Leeet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib23); Behnamet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib24); Liuet al\.[2024b](https://arxiv.org/html/2607.20538#bib.bib28)\)\)\. Codec\-Gauge instead operates on the dense tensor coordinates of retained entries, so it can be layered with retention and paging policies\.

#### Transform and numerical KV compression\.

Frequency\-domain and transform\-coding methods treat KV caches as structured numerical signals\. FreqKV, FAEDKV, and KVTC exploit frequency or transform\-domain redundancy \(\(Kaiet al\.[2026](https://arxiv.org/html/2607.20538#bib.bib25); Liet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib26); Staniszewski and Łańcucki[2026](https://arxiv.org/html/2607.20538#bib.bib27)\)\), while DCT and zfp show how spectral concentration and local smoothness affect array rate\-distortion behavior \(\(Ahmedet al\.[1974](https://arxiv.org/html/2607.20538#bib.bib39); Lindstrom[2014](https://arxiv.org/html/2607.20538#bib.bib38)\)\)\. Codec\-Gauge contributes a learned coordinate layer for this setting: the gauge shapes the numerical field seen by an existing codec’s token–channel layout, and compression is performed by the backend itself\.

#### Architectural, low\-rank, and training\-aware KV representations\.

Several methods reduce KV state by changing model structure or learning a different cache representation, including latent KV states, low\-rank projection, depth redundancy, dynamic memory compression, training for compressible KV, and adaptive orthogonal projections \(\(DeepSeek\-AI[2024](https://arxiv.org/html/2607.20538#bib.bib29); Changet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib30); Liuet al\.[2024a](https://arxiv.org/html/2607.20538#bib.bib32); Nawrotet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib33); Gelberget al\.[2026](https://arxiv.org/html/2607.20538#bib.bib34); Linet al\.[2025a](https://arxiv.org/html/2607.20538#bib.bib31)\)\)\. Codec\-Gauge keeps the checkpoint frozen: a small invertible gauge is trained for an existing model and evaluated around an explicit compression/decompression path\.

## Problem Statement and Overview

Table 1:Notation for cache gauge training\.Table[1](https://arxiv.org/html/2607.20538#Sx3.T1)summarizes the notation\. We consider the standard key/value cache produced by softmax\-attention Transformers during autoregressive inference\. For layerll, the cache for the previousTTtokens is:

Kl,Vl∈ℝB×Hl×T×dl,ℳT=\{\(Kl,Vl\)\}l=1L\.K\_\{l\},V\_\{l\}\\in\\mathbb\{R\}^\{B\\times H\_\{l\}\\times T\\times d\_\{l\}\},\\qquad\\mathcal\{M\}\_\{T\}=\\\{\(K\_\{l\},V\_\{l\}\)\\\}\_\{l=1\}^\{L\}\.The number of cache elements in FP16 or BF16 storage is:

NKV​\(T\)=∑l=1L2​B​Hl​T​dl,N\_\{\\mathrm\{KV\}\}\(T\)=\\sum\_\{l=1\}^\{L\}2BH\_\{l\}Td\_\{l\},which grows linearly with context length\. We only consider key/value tensors exposed by standard softmax attention; recurrent or linear\-attention states are outside the gauge and compression object studied here\.

Codec\-Gauge learns a cache\-coordinate gauge without changing tokens, attention architecture, or model weights\. LetXl,sX\_\{l,s\}denote a layer\-llcache tensor withs∈\{K,V\}s\\in\\\{K,V\\\}\. For each layer, cache type, head, and channel group, we learn an invertible gaugeGl,s,h,rG\_\{l,s,h,r\}\. The group dimensionggpartitions the head dimension intoRl=dl/gR\_\{l\}=d\_\{l\}/ggroups; wheng=dlg=d\_\{l\}, the gauge acts on the full head\. For groupr=1,…,Rlr=1,\\ldots,R\_\{l\},

Xl,s​\[b,h,t,r\]∈ℝg,X\_\{l,s\}\[b,h,t,r\]\\in\\mathbb\{R\}^\{g\},and the transformed cache is:

Zl,s​\[b,h,t,r\]=Gl,s,h,r​Xl,s​\[b,h,t,r\]\.Z\_\{l,s\}\[b,h,t,r\]=G\_\{l,s,h,r\}X\_\{l,s\}\[b,h,t,r\]\.The transform mixes channels only within a group; it neither mixes nor removes tokens\.

At inference time, the backend observes the transformed\-coordinate cache rather than the raw cache:

Z\\displaystyle Z=G​X,\\displaystyle=GX,Bc\\displaystyle B\_\{c\}=C​\(Z\),\\displaystyle=C\(Z\),Z^\\displaystyle\\hat\{Z\}=D​\(Bc\),\\displaystyle=D\(B\_\{c\}\),X^\\displaystyle\\hat\{X\}=G−1​Z^,\\displaystyle=G^\{\-1\}\\hat\{Z\},whereCCandDDare the compression/decompression or quantization/reconstruction maps of a fixed backend, andBcB\_\{c\}is the compressed buffer consumed only through the decoded cacheX^\\hat\{X\}\. Without backend error,G−1​G​X=XG^\{\-1\}GX=Xand model behavior is unchanged up to numerical precision\. Under lossy backends, the gauge leaves attention semantics fixed but changes the numerical geometry seen by the backend, and therefore the structure of recovery error and downstream output perturbation, meaning changes in likelihoods, logits, and top\-token rankings relative to the full\-cache reference\.

The evaluation target is direct: for a frozen Transformer, fixed data distribution, fixed backend, and matched actual bits/value, a learned gauge should reduce KV reconstruction error and rolling suffix output perturbation relative to raw and control coordinates\.

## Method and Evaluation Design

### Gauge Parameterization and Training

Given a frozen Transformerfθf\_\{\\theta\}, Codec\-Gauge learns only cache\-coordinate parameters and does not update model weights\. For each layerll, cache types∈\{K,V\}s\\in\\\{K,V\\\}, headhh, and channel grouprr, we parameterize an orthogonal gauge as:

Gl,s,h,r=exp⁡\(Al,s,h,r−Al,s,h,r⊤\),Gl,s,h,r−1=Gl,s,h,r⊤,G\_\{l,s,h,r\}=\\exp\(A\_\{l,s,h,r\}\-A\_\{l,s,h,r\}^\{\\top\}\),\\qquad G\_\{l,s,h,r\}^\{\-1\}=G\_\{l,s,h,r\}^\{\\top\},whereexp⁡\(⋅\)\\exp\(\\cdot\)is the matrix exponential\. SinceA−A⊤A\-A^\{\\top\}is skew\-symmetric, the resulting small group\-level gauge is orthogonal and invertible without an orthogonality penalty\. The group dimensionggsets the channel subspace covered by each gauge: smaller groups restrict mixing, while full\-head gauges provide higher expressivity\. We evaluateg=8g=8,1616,3232, and full\-head gauges, and reportg=16g=16as the representative grouped setting\.

The gauge is a fixed model\-side object, not a per\-token cache payload\. Explicit application costsO​\(NKV​g\)O\(N\_\{\\mathrm\{KV\}\}g\)multiply\-adds, and learned parameters scale asO​\(∑l2​Hl​dl​g\)O\(\\sum\_\{l\}2H\_\{l\}d\_\{l\}g\)rather than with context length\. Some recovery operations can be folded into downstream projections; key\-side fusion depends on positional encoding and kernel layout\. Our implementation uses explicit key/value recovery and charges that cost in timing\.

Gauge training uses FineWeb\-Edu sample\-10BT \(\(Penedoet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib37)\)\)\. For each model, we collect6464non\-overlapping 4096\-token windows, run frozen\-model prefill, collect standard softmax\-attention KV tensors, and optimize the gauge for2525epochs with AdamW\. This one\-time per\-checkpoint step usesTb=16T\_\{b\}=16,λf=1\.0\\lambda\_\{f\}=1\.0,λr=0\.02\\lambda\_\{r\}=0\.02, andτ=0\.04\\tau=0\.04for every checkpoint, with no model\-specific tuning\. Evaluation windows are recorded from disjoint FineWeb\-Edu documents with a different seed\. The objective depends only on cache geometry; it does not include language\-modeling loss, logit matching, or task\-output supervision\.

### Gauge Training Objective

For a transformed cache tensorX′=G​XX^\{\\prime\}=GX, we partition the token dimension into blocks of lengthTbT\_\{b\}and apply a two\-dimensional orthonormal DCT to eachTb×dlT\_\{b\}\\times d\_\{l\}token–channel block:

U=DCT2​D​\(block​\(X′\)\)\.U=\\mathrm\{DCT\}\_\{2D\}\(\\mathrm\{block\}\(X^\{\\prime\}\)\)\.The training objective is

Ltrain=λf​Lfreq\+λr​Lrate\.L\_\{\\mathrm\{train\}\}=\\lambda\_\{f\}L\_\{\\mathrm\{freq\}\}\+\\lambda\_\{r\}L\_\{\\mathrm\{rate\}\}\.
The frequency term is a spectral centroid under the token–channel layout exposed to the codec\. Although the gauge may be grouped, this statistic spans the complete head layout presented to zfp\. The DCT supplies a differentiable statistic, and the gauge chooses a channel basis whose observed KV energy becomes concentrated in low token and layout frequencies\. Channel frequency refers to memory order after the basis change, not intrinsic spatial order; token order remains unchanged\. For coefficient position\(u,v\)\(u,v\), define the normalized radius

ρ​\(u,v\)=12​\(uTb−1\)2\+\(vdl−1\)2\.\\rho\(u,v\)=\\frac\{1\}\{\\sqrt\{2\}\}\\sqrt\{\\left\(\\frac\{u\}\{T\_\{b\}\-1\}\\right\)^\{2\}\+\\left\(\\frac\{v\}\{d\_\{l\}\-1\}\\right\)^\{2\}\}\.With coefficient energyEu,v=Uu,v2E\_\{u,v\}=U\_\{u,v\}^\{2\}, we define

Lfreq=∑u,vEu,v​ρ​\(u,v\)∑u,vEu,v\+ϵ\.L\_\{\\mathrm\{freq\}\}=\\frac\{\\sum\_\{u,v\}E\_\{u,v\}\\rho\(u,v\)\}\{\\sum\_\{u,v\}E\_\{u,v\}\+\\epsilon\}\.Minimizing this term shifts energy toward lower radial frequencies in the codec\-facing layout\. Since the gauge acts only on channel groups, it learns how each token\-frequency component is represented across channel\-layout modes, making neighboring coordinates in the fixed codec layout carry more correlated, low\-frequency energy\. Token order, channel grouping, and the zfp field layout are fixed across identity, random, fixed\-transform, PCA/KLT, and learned conditions, so the coordinate map is the isolated variable\. We track spectral concentration as1−Lfreq1\-L\_\{\\mathrm\{freq\}\}\.

The rate term is a smooth log\-amplitude proxy:

Lrate=𝔼u,v​\[log⁡\(1\+\|Uu,v\|τ\)\],L\_\{\\mathrm\{rate\}\}=\\mathbb\{E\}\_\{u,v\}\\left\[\\log\\left\(1\+\\frac\{\|U\_\{u,v\}\|\}\{\\tau\}\\right\)\\right\],withτ=0\.04\\tau=0\.04\. Because the gauge and DCT are energy\-preserving transforms, this concave penalty favors concentrating energy into fewer significant coefficients\. Together withLfreqL\_\{\\mathrm\{freq\}\}, it provides a differentiable codec\-layout objective; held\-out evaluation uses measured backend bytes and post\-decode model behavior rather than the proxy itself\. After training, the gauge is fixed for all evaluation windows\.

### Compression Conditions

The main evaluation grid evaluates each backend under identity, random, and learned coordinates\. Identity compresses raw KV cache; random uses a sampled orthogonal gauge with the same block structure as the learned gauge; learned uses the trained gauge\. Random captures the known benefits of generic rotation and outlier redistribution in quantization \(\(Cheeet al\.[2023](https://arxiv.org/html/2607.20538#bib.bib10); Ashkbooset al\.[2024](https://arxiv.org/html/2607.20538#bib.bib12); Liuet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib13); Suet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib15); Saxena and Roy[2025](https://arxiv.org/html/2607.20538#bib.bib16)\)\), so gains beyond it test the value of the learned codec\-layout geometry\.

The primary backend is GPU zfp compression applied to continuous KV tensors at 2\-, 3\-, 4\-, 6\-, and 8\-bit/value\. The implementation encodes contiguous FP32\-transformed tensors with native CUDA zfp, decodes them, applies the inverse gauge, and records the actual compressed bytes\. The CUDA extension presents each tensor as a two\-dimensional zfp field withnx=dln\_\{x\}=d\_\{l\}andny=numel/dln\_\{y\}=\\mathrm\{numel\}/d\_\{l\}; reported bytes include the zfp payload and a fixed shape header\. Block\-uniform uses one range perTb×dlT\_\{b\}\\times d\_\{l\}block, whereas the KIVI\-style path uses per\-channel key and per\-token value ranges \(\(Liuet al\.[2024c](https://arxiv.org/html/2607.20538#bib.bib5)\)\); both use the same bit settings\. Each main evaluation batch contains4949conditions: one full\-cache baseline, three no\-compression sanity rows, and1515rows each for zfp, block\-uniform quantization, and KIVI\-style quantization\. A separate zfp coordinate\-control run evaluates identity, learned, three random gauges, Hadamard, DCT, and PCA/KLT controls at33,44, and66bits/value\. PCA/KLT uses an uncentered second\-moment eigenbasis computed at the same layer, K/V, head, and group granularity\.

![Refer to caption](https://arxiv.org/html/2607.20538v1/x2.png)Figure 2:Aggregate output\-distribution perturbation underg=16g=16gauges at33,44, and66bits/value\. Lower is better; KL divergence and logit MSE use log axes, and top\-1 flip rate is a percentage\. Error bars summarize the standard error across six model means\. Annotations report the mean paired learned\-gauge reduction relative to identity coordinates across the six models\.
### Evaluation Design

We evaluate six checkpoints spanning multiple providers and cache implementations: Qwen3\-0\.6B, Qwen3\.5\-0\.8B, Llama\-3\.2\-1B, Gemma\-3\-1B, Phi\-4 Mini Instruct, and Ministral\-3\-3B\-Base\-2512\. The set includes grouped\-query softmax\-attention decoders, hybrid\-cache behavior where only standard tensor KV layers are gauged, and long\-context/windowed attention variants\. We additionally run a larger Gemma\-3\-27B model under zfp at the intermediate rates\.

Output quality is measured with paired rolling compressed\-history scoring over 256 evaluation windows per model\. Each window has 2,048 prefix tokens, one teacher\-forced rolling input token, and 384 scored target tokens\. All conditions consume the same ground\-truth input token, but compressed conditions maintain their own histories, so every KV token passes through the condition’s gauge, backend, decode, and inverse\-gauge path\.

Cache\-side metrics include compressed bytes, actual bits/value, KV MSE, KV NRMSE, key/value NRMSE, and maximum absolute error\. Output\-side metrics include paired delta\-NLL:

Δ​NLL=NLLcompressed−NLLfull,\\Delta\\mathrm\{NLL\}=\\mathrm\{NLL\}\_\{\\mathrm\{compressed\}\}\-\\mathrm\{NLL\}\_\{\\mathrm\{full\}\},as well asKL​\(pfull∥pcompressed\)\\mathrm\{KL\}\(p\_\{\\mathrm\{full\}\}\\\|p\_\{\\mathrm\{compressed\}\}\), logit MSE, top\-1 flip rate, and top\-5 overlap\. These paired metrics keep the model, input tokens, and reference logits fixed while measuring loss, distribution shift, and top\-kkstability\. The evaluator writes raw sums and counts\. Actual bits/value are computed from measured compressed bytes\. Paper figures post\-process the released raw files as:

Δ​NLL/token\\displaystyle\\Delta\\mathrm\{NLL/token\}=∑Δ​NLL∑target tokens,\\displaystyle=\\frac\{\\sum\\Delta\\mathrm\{NLL\}\}\{\\sum\\text\{target tokens\}\},KV​NRMSE\\displaystyle\\mathrm\{KV\\ NRMSE\}=∑KV​SSE∑KV​reference​SSE,\\displaystyle=\\sqrt\{\\frac\{\\sum\\mathrm\{KV\\ SSE\}\}\{\\sum\\mathrm\{KV\\ reference\\ SSE\}\}\},top1​flip​rate\\displaystyle\\mathrm\{top1\\ flip\\ rate\}=∑top1​flips∑target tokens\.\\displaystyle=\\frac\{\\sum\\mathrm\{top1\\ flips\}\}\{\\sum\\text\{target tokens\}\}\.This aggregation is invariant to model\-specific batching\. Main analyses use 3, 4, and 6 bits/value; 2\-bit and 8\-bit results remain in the released outputs as boundary conditions\.

Sanity rows validate no\-compression invariants: identity\-clone checks cache cloning, and random/learned inverse rows check theG−1​GG^\{\-1\}Ground trip\. In theg=16g=16run, identity clone has zero cache error, while random and learned inverse rows have maximum cache NRMSE below5\.1×10−85\.1\\times 10^\{\-8\}\.

We also run a serial context\-length evaluation for full\-cache, zfp\-4, block\-uniform\-4, and KIVI\-4 paths\. The 4\-bit setting provides a matched comparison between fixed\-rate numerical compression and low\-bit quantization\. We record KV allocation growth relative to a one\-token origin and total decode\-plus\-model time normalized by the full\-cache path, including compression\-path recovery\.

For task\-prompt checks, we score RULER needle retrieval and variable tracking \(\(Hsiehet al\.[2024](https://arxiv.org/html/2607.20538#bib.bib36)\)\) together with LongBench\-v2 multiple\-choice prompts \(\(Baiet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib35)\)\) under the same zfp rates and coordinate conditions\.

## Results

Across implemented backend paths, learned gauges reduce cache reconstruction error and output perturbation\. Coordinate controls and held\-out spectra identify the source of this effect; a 27B extension and task prompts test its extent, while serial measurements verify the realized storage path\.

![Refer to caption](https://arxiv.org/html/2607.20538v1/x3.png)Figure 3:Coordinate\-control checks for zfp compression at33,44, and66bits/value\. Bars report error reduction relative to identity coordinates; higher is better, and error bars are standard errors across the1818model–rate pairs\. Random averages three orthogonal gauges\. Hadamard, DCT, and PCA/KLT provide fixed or data\-derived structured controls\.![Refer to caption](https://arxiv.org/html/2607.20538v1/x4.png)Figure 4:Gauge training dynamics and spectral cache geometry\. Left: mean objective terms across six models with standard\-error bands\. Right: representative normalized DCT energy maps from Qwen3\-0\.6B; the reported concentration means and standard errors aggregate all268268standard\-attention K/V layer maps across the six models\. Learned gauges concentrate energy in lower token–channel frequencies\.### Compression Quality Under Matched Rates

The main quantitative analyses use33,44, and66bits/value\. The22\-bit setting often enters a severe\-distortion regime, while88bits/value approaches a high\-fidelity boundary where transform\-only numerical noise can become comparable to the remaining compression perturbation; both boundary settings are retained in released outputs\.

Figure[2](https://arxiv.org/html/2607.20538#Sx4.F2)shows absolute paired quality metrics; percentage labels summarize reductions at matched settings\. For fixed\-rate zfp compression, learned gauges reduce KL divergence, logit MSE, and top\-1 flip rate by44\.0%44\.0\\%,43\.3%43\.3\\%, and24\.5%24\.5\\%on average relative to identity coordinates\. The corresponding zfp actual bits/value are measured from compressed bytes and match the target rates to within6×10−56\\times 10^\{\-5\}bits/value, so the gains come from cache geometry rather than hidden rate differences\. Cache reconstruction follows the same trend: learned gauges reduce zfp KV NRMSE by18\.3%18\.3\\%over the same operating points\.

The same learned gauge also improves scalar low\-bit paths\. For block\-uniform quantization, learned gauges reduce KL divergence, logit MSE, and top\-1 flip rate by44\.2%44\.2\\%,42\.1%42\.1\\%, and26\.1%26\.1\\%, respectively\. For KIVI\-style quantization, the corresponding reductions are27\.2%27\.2\\%,31\.3%31\.3\\%, and15\.2%15\.2\\%\. Both quantizers improve all three output metrics in every one of the1818model–rate pairs\. Thus, a gauge trained only from cache geometry can improve quality preservation for common coordinate\-sensitive quantizers without language\-modeling, logit, or task\-output losses during gauge training\.

### Coordinate Controls

Figure[3](https://arxiv.org/html/2607.20538#Sx5.F3)evaluates a structured control set under the same zfp backend\. Random orthogonal gauges reduce KL by8\.5%8\.5\\%on average, Hadamard by18\.3%18\.3\\%, and DCT by15\.5%15\.5\\%\. PCA/KLT improves KV NRMSE but worsens the output metrics on average, showing that ordering directions by second\-moment reconstruction does not determine where decoding error affects attention logits\. Learned gauges reduce KL by44\.0%44\.0\\%and improve all1818model–rate pairs relative to both identity coordinates and the random\-gauge average\. Among the evaluated controls, the frequency\-trained gauge therefore provides the largest and most consistent improvement under matched backend settings and measured rates\.

### Mechanism

The training objective directly targets frequency concentration in the codec\-facing token–channel layout\. Figure[4](https://arxiv.org/html/2607.20538#Sx5.F4)confirms that the objective transfers to held\-out evaluation windows: identity and random gauges average approximately0\.5700\.570spectral concentration, while learned gauges reach0\.7090\.709\. The total training loss decreases by 30\.1%, and the frequency term by 33\.4%\. These measurements connect the training signal to the observed zfp rate\-distortion behavior: the gauge creates a more spectrally concentrated storage\-facing channel layout for the numerical codec, and actual compression/decompression then yields lower cache and output perturbation\.

For scalar quantization, the mechanism also includes range redistribution and outlier smoothing, effects already known to aid low\-bit LLM inference \(\(Ashkbooset al\.[2024](https://arxiv.org/html/2607.20538#bib.bib12); Liuet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib13); Suet al\.[2025](https://arxiv.org/html/2607.20538#bib.bib15)\)\)\. Learned gauges still reduce output perturbation beyond random gauges in the aggregate quantizer rows, indicating that frequency\-shaped geometry contributes useful structure even when the backend’s error mechanism differs from zfp’s\.

### Scale, Task, and Storage Checks

![Refer to caption](https://arxiv.org/html/2607.20538v1/x5.png)Figure 5:Validation beyond the six\-model matrix\. Top left: zfp KL on Gemma\-3\-27B\. Top right: correct\-answer KL averaged over33,44, and66bits/value for each task family; error bars are standard errors across the three rate\-specific model means\. Bottom: persistent KV\-allocation growth and decode\-plus\-model time for the learned\-coordinate paths in the serial context\-length run; timing includes explicit cache recovery\.Figure[5](https://arxiv.org/html/2607.20538#Sx5.F5)adds scale, task\-prompt, and storage/timing evidence\. On Gemma\-3\-27B, learned gauges reduce zfp KL by32\.1%32\.1\\%,39\.5%39\.5\\%, and49\.3%49\.3\\%at33,44, and66bits/value, following the same direction as the six\-model matrix\.

Task\-prompt scoring shows the same paired\-distribution trend\. Averaged across tasks, models, and rates, learned gauges lower correct\-answer KL from0\.7720\.772to0\.5370\.537and correct\-answer delta\-NLL from0\.8190\.819to0\.5710\.571relative to the full\-cache reference\. Accuracy is also recorded, but these moderate\-compression settings rarely move the full\-cache answer across a decision boundary; likelihood and ranking metrics therefore provide higher\-resolution evidence of quality preservation\.

The serial context\-length run measures persistent KV allocation growth and recovery cost in the implemented path\. Allocation is reported as growth over a one\-token origin, isolating context\-dependent KV memory from fixed model weights and persistent allocator state\. At the longest measured context, zfp\-4 uses4\.0004\.000actual bits/value, and its allocation growth is essentially aligned with the 4\-bit block\-uniform path, while KIVI\-4 uses a larger effective payload\. Within this serial implementation, zfp\-4 has lower measured decode overhead than the quantization paths\. These measurements validate that fixed\-rate numerical compression attains its intended persistent KV\-storage reduction while including explicit cache recovery and model decoding in the measured path\.

## Discussion and Limitations

Codec\-Gauge improves KV\-cache compression quality by training the gauge exposed to a backend\. Figure[3](https://arxiv.org/html/2607.20538#Sx5.F3)shows that the improvement is not explained by generic rotation, fixed transform coding, or a data\-derived basis alone; Figure[4](https://arxiv.org/html/2607.20538#Sx5.F4)shows the intended storage\-facing spectral mechanism\. The gauge shapes the codec\-facing frequency distribution in a fixed token/channel layout, then is tested through actual compression, decompression, inverse recovery, and rolling\-cache scoring\.

The quantization rows broaden the finding\. Orthogonal rotations and incoherence processing are already strong tools for low\-bit quantization, and random gauges capture part of that effect\. The learned frequency\-distribution objective still reduces output perturbation on top of random gauges for block\-uniform and KIVI\-style paths, without quantization\-specific losses, language\-modeling loss, or logit supervision\.

The training objective remains outside both codec and model\-output loops: it differentiates through neither zfp nor a quantizer and uses no reconstruction or output loss\. Its transfer to held\-out zfp compression and two scalar quantizers therefore supports a representation\-level geometric effect rather than adaptation to one decoder’s artifacts\.

The practical consequence is fidelity headroom at a fixed cache budget: the same backend configuration and measured rate produce a recovered cache closer to the full\-cache reference\. Because gauge parameters are fixed per checkpoint and do not scale with context length, their storage cost is amortized as the retained context grows\.

The paired rolling metrics measure cache\-induced distributional change under fixed tokens and reference logits\. KL, logit MSE, delta\-NLL, and top\-1 flips quantify how much the compressed\-cache model deviates from the full\-cache model before a discrete answer changes\. Prompts from these three task families add likelihood\-based checks in long\-context task formats; thresholded accuracy is also reported, but is a lower\-resolution view when moderate compression rarely changes the selected answer\.

The evaluated checkpoints span multiple providers, attention variants, and cache implementations\. The 27B model check follows the same trend at a larger scale, while the six\-model matrix tests architectural diversity\. Results are also stable acrossg=8g=8,1616,3232, and full\-head gauges;g=16g=16is used as a representative middle setting\.

Modern KV\-cache systems jointly choose precision, packing, paging, eviction, and scheduling\. Codec\-Gauge acts at the retained\-tensor interface: quantizers and cache managers retain these backend decisions, while the gauge changes only the coordinates presented to them\. Holding the remaining system choices fixed therefore isolates the observed rate\-distortion improvement as a representation\-side effect\.

Timing includes explicit key/value recovery rather than assuming fusion\. Deployment\-oriented implementations can specialize this path through kernel fusion, backend\-specific packing, and paged\-KV integration without changing the learned coordinate objective\. The coordinate controls, two quantization paths, task prompts, 27B extension, and measured storage/timing paths together establish the effect beyond a single model, transform, or evaluation metric\.

## Conclusion

We introduced Codec\-Gauge, a post\-training coordinate\-shaping method for Transformer KV caches\. Learned gauges improve zfp fixed\-rate compression beyond raw and control coordinates under matched measured bits/value, and also improve output preservation for block\-uniform and KIVI\-style quantization paths\. The method keeps the model and backend fixed, learns only small orthogonal KV\-coordinate maps from frozen\-model cache tensors, and evaluates the result through actual compression, decompression, and rolling\-cache scoring\.

The broader conclusion is that KV\-cache compressibility is not only a property of the backend or the checkpoint in its original basis: the coordinates exposed to a codec or quantizer are an actionable variable\. Training this gauge with a frequency\-distribution objective improves cache reconstruction and output\-distribution stability at storage rates realized by actual compressed\-cache paths, without changing attention semantics\. Cache\-coordinate shaping therefore complements existing KV compression, quantization, and serving pipelines while separating a reusable representation\-side optimization from backend\-specific engineering choices\.

#### Generative AI use\.

Generative AI tools assisted with manuscript preparation, code, figures, and reference checks; the authors verified all outputs and take full responsibility\.

## References

- N\. Ahmed, T\. Natarajan, and K\. R\. Rao \(1974\)Discrete cosine transform\.IEEE Transactions on ComputersC\-23\(1\),pp\. 90–93\.External Links:[Document](https://dx.doi.org/10.1109/T-C.1974.223784)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Transform and numerical KV compression\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px3.p1.1)\.
- J\. Ainslie, J\. Lee\-Thorp, M\. de Jong, Y\. Zemlyanskiy, F\. Lebrón, and S\. Sanghai \(2023\)GQA: training generalized multi\-query transformer models from multi\-head checkpoints\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 4895–4901\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.298),[Link](https://aclanthology.org/2023.emnlp-main.298/)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p1.1)\.
- S\. Ashkboos, A\. Mohtashami, M\. L\. Croci, B\. Li, P\. Cameron, M\. Jaggi, D\. Alistarh, T\. Hoefler, and J\. Hensman \(2024\)QuaRot: outlier\-free 4\-bit inference in rotated LLMs\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=dfqsW38v1X)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Introduction](https://arxiv.org/html/2607.20538#Sx1.p7.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1),[Compression Conditions](https://arxiv.org/html/2607.20538#Sx4.SSx3.p1.1),[Mechanism](https://arxiv.org/html/2607.20538#Sx5.SSx3.p2.1)\.
- Y\. Bai, S\. Tu, J\. Zhang, H\. Peng, X\. Wang, X\. Lv, S\. Cao, J\. Xu, L\. Hou, Y\. Dong, J\. Tang, and J\. Li \(2025\)LongBench v2: towards deeper understanding and reasoning on realistic long\-context multitasks\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 3639–3664\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.183),[Link](https://aclanthology.org/2025.acl-long.183/)Cited by:[Evaluation Design](https://arxiv.org/html/2607.20538#Sx4.SSx4.p6.1)\.
- P\. Behnam, Y\. Fu, R\. Zhao, P\. Tsai, Z\. Yu, and A\. Tumanov \(2025\)RocketKV: accelerating long\-context LLM inference via two\-stage KV cache compression\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 3358–3392\.External Links:[Link](https://proceedings.mlr.press/v267/behnam25a.html)Cited by:[Token, page, and dynamic cache selection\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px2.p1.1)\.
- Z\. Cai, Y\. Zhang, B\. Gao, Y\. Liu, Y\. Li, T\. Liu, K\. Lu, W\. Xiong, Y\. Dong, J\. Hu, and W\. Xiao \(2025\)PyramidKV: dynamic KV cache compression based on pyramidal information funneling\.InConference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=ayi7qezU87)Cited by:[Token, page, and dynamic cache selection\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px2.p1.1)\.
- C\. Chang, W\. Lin, C\. Lin, C\. Chen, Y\. Hu, P\. Wang, N\. Huang, L\. Ceze, M\. S\. Abdelfattah, and K\. Wu \(2025\)Palu: KV\-cache compression with low\-rank projection\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=LWMS4pk2vK)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Architectural, low\-rank, and training\-aware KV representations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px4.p1.1)\.
- J\. Chee, Y\. Cai, V\. Kuleshov, and C\. De Sa \(2023\)QuIP: 2\-bit quantization of large language models with guarantees\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://openreview.net/forum?id=xrk9g5vcXR)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Introduction](https://arxiv.org/html/2607.20538#Sx1.p7.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1),[Compression Conditions](https://arxiv.org/html/2607.20538#Sx4.SSx3.p1.1)\.
- T\. Dao, D\. Y\. Fu, S\. Ermon, A\. Rudra, and C\. Ré \(2022\)FlashAttention: fast and memory\-efficient exact attention with IO\-awareness\.InAdvances in Neural Information Processing Systems,Vol\.35\.Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p1.1)\.
- DeepSeek\-AI \(2024\)DeepSeek\-V2: a strong, economical, and efficient mixture\-of\-experts language model\.External Links:2405\.04434Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Architectural, low\-rank, and training\-aware KV representations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px4.p1.1)\.
- S\. Ge, Y\. Zhang, L\. Liu, M\. Zhang, J\. Han, and J\. Gao \(2024\)Model tells you what to discard: adaptive KV cache compression for LLMs\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=uNrFpDPMyo)Cited by:[Token, page, and dynamic cache selection\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px2.p1.1)\.
- Y\. Gelberg, Y\. Eitan, M\. Bronstein, Y\. Gal, and H\. Maron \(2026\)Training transformers for KV cache compressibility\.External Links:2605\.05971Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Architectural, low\-rank, and training\-aware KV representations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px4.p1.1)\.
- C\. R\. C\. Hooper, S\. Kim, H\. Mohammadzadeh, M\. W\. Mahoney, Y\. S\. Shao, K\. Keutzer, and A\. Gholami \(2024\)KVQuant: towards 10 million context length LLM inference with KV cache quantization\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=0LXotew9Du)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1)\.
- C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, and B\. Ginsburg \(2024\)RULER: what’s the real context size of your long\-context language models?\.InConference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=kIoBbc76Sy)Cited by:[Evaluation Design](https://arxiv.org/html/2607.20538#Sx4.SSx4.p6.1)\.
- J\. Kai, Y\. Wang, B\. Zeng, H\. Bai, B\. Jiang, Z\. He, and Z\. Lin \(2026\)FreqKV: key\-value compression in frequency domain for context window extension\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=wFSOtyvQ9d)Cited by:[Transform and numerical KV compression\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px3.p1.1)\.
- H\. Kang, Q\. Zhang, S\. Kundu, G\. Jeong, Z\. Liu, T\. Krishna, and T\. Zhao \(2024\)GEAR: an efficient error reduction framework for KV cache compression in LLM inference\.InProceedings of The 4th NeurIPS Efficient Natural Language and Speech Processing Workshop,Proceedings of Machine Learning Research, Vol\.262,pp\. 305–321\.External Links:[Link](https://proceedings.mlr.press/v262/kang24a.html)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with PagedAttention\.InProceedings of the 29th Symposium on Operating Systems Principles,pp\. 611–626\.External Links:[Document](https://dx.doi.org/10.1145/3600006.3613165)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p1.1)\.
- W\. Lee, J\. Lee, J\. Seo, and J\. Sim \(2024\)InfiniGen: efficient generative inference of large language models with dynamic KV cache management\.InProceedings of the 18th USENIX Symposium on Operating Systems Design and Implementation,pp\. 155–172\.External Links:[Link](https://www.usenix.org/conference/osdi24/presentation/lee)Cited by:[Token, page, and dynamic cache selection\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px2.p1.1)\.
- R\. Li, Y\. Fu, M\. Sheng, X\. Long, H\. Yu, and P\. Li \(2025\)FAEDKV: infinite\-window fourier transform for unbiased KV cache compression\.InFindings of the Association for Computational Linguistics: EMNLP 2025,pp\. 16856–16866\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.914),[Link](https://aclanthology.org/2025.findings-emnlp.914/)Cited by:[Transform and numerical KV compression\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px3.p1.1)\.
- Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. Chen \(2024\)SnapKV: LLM knows what you are looking for before generation\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=poE54GOq2l)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Token, page, and dynamic cache selection\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px2.p1.1)\.
- B\. Lin, Z\. Zeng, Z\. Xiao, S\. Kou, T\. Hou, X\. Gao, H\. Zhang, and Z\. Deng \(2025a\)MatryoshkaKV: adaptive KV compression via trainable orthogonal projection\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=BQwsRy1h3U)Cited by:[Architectural, low\-rank, and training\-aware KV representations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px4.p1.1)\.
- Y\. Lin, H\. Tang, S\. Yang, Z\. Zhang, G\. Xiao, C\. Gan, and S\. Han \(2025b\)QServe: W4A8KV4 quantization and system co\-design for efficient LLM serving\.InProceedings of Machine Learning and Systems,Vol\.7\.External Links:2405\.04532,[Link](https://proceedings.mlsys.org/paper_files/paper/2025/hash/fbe2b2f74a2ece8070d8fb073717bda6-Abstract-Conference.html)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1)\.
- P\. Lindstrom \(2014\)Fixed\-rate compressed floating\-point arrays\.IEEE Transactions on Visualization and Computer Graphics20\(12\),pp\. 2674–2683\.External Links:[Document](https://dx.doi.org/10.1109/TVCG.2014.2346458)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Introduction](https://arxiv.org/html/2607.20538#Sx1.p5.1),[Transform and numerical KV compression\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px3.p1.1)\.
- A\. Liu, J\. Liu, Z\. Pan, Y\. He, G\. Haffari, and B\. Zhuang \(2024a\)MiniCache: KV cache compression in depth dimension for large language models\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Document](https://dx.doi.org/10.52202/079017-4443),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/fd0705710bf01b88a60a3d479ea341d9-Abstract-Conference.html)Cited by:[Architectural, low\-rank, and training\-aware KV representations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px4.p1.1)\.
- Y\. Liu, H\. Li, Y\. Cheng, S\. Ray, Y\. Huang, Q\. Zhang, K\. Du, J\. Yao, S\. Lu, G\. Ananthanarayanan, M\. Maire, H\. Hoffmann, A\. Holtzman, and J\. Jiang \(2024b\)CacheGen: KV cache compression and streaming for fast large language model serving\.InProceedings of the ACM SIGCOMM Conference,pp\. 38–56\.External Links:[Document](https://dx.doi.org/10.1145/3651890.3672274)Cited by:[Token, page, and dynamic cache selection\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px2.p1.1)\.
- Z\. Liu, C\. Zhao, I\. Fedorov, B\. Soran, D\. Choudhary, R\. Krishnamoorthi, V\. Chandra, Y\. Tian, and T\. Blankevoort \(2025\)SpinQuant: LLM quantization with learned rotations\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ogO6DGE6FZ)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Introduction](https://arxiv.org/html/2607.20538#Sx1.p7.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1),[Compression Conditions](https://arxiv.org/html/2607.20538#Sx4.SSx3.p1.1),[Mechanism](https://arxiv.org/html/2607.20538#Sx5.SSx3.p2.1)\.
- Z\. Liu, J\. Yuan, H\. Jin, S\. Zhong, Z\. Xu, V\. Braverman, B\. Chen, and X\. Hu \(2024c\)KIVI: a tuning\-free asymmetric 2bit quantization for KV cache\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 32332–32344\.External Links:[Link](https://proceedings.mlr.press/v235/liu24bz.html)Cited by:[Backend compatibility\.](https://arxiv.org/html/2607.20538#Sx1.SS0.SSS0.Px1.p2.4),[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1),[Compression Conditions](https://arxiv.org/html/2607.20538#Sx4.SSx3.p2.8)\.
- P\. Nawrot, A\. Łańcucki, M\. Chochowski, D\. Tarjan, and E\. Ponti \(2024\)Dynamic memory compression: retrofitting LLMs for accelerated inference\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 37396–37412\.External Links:[Link](https://proceedings.mlr.press/v235/nawrot24a.html)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Architectural, low\-rank, and training\-aware KV representations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px4.p1.1)\.
- G\. Penedo, H\. Kydlíček, L\. Ben Allal, A\. Lozhkov, M\. Mitchell, C\. Raffel, L\. Von Werra, and T\. Wolf \(2024\)The FineWeb datasets: decanting the web for the finest text data at scale\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 30811–30849\.External Links:[Document](https://dx.doi.org/10.52202/079017-0970),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/370df50ccfdf8bde18f8f9c2d9151bda-Abstract-Datasets_and_Benchmarks_Track.html)Cited by:[Gauge Parameterization and Training](https://arxiv.org/html/2607.20538#Sx4.SSx1.p3.6)\.
- U\. Saxena and K\. Roy \(2025\)KVLinC: KV cache quantization with hadamard rotation and linear correction\.External Links:2510\.05373,[Document](https://dx.doi.org/10.48550/arXiv.2510.05373),[Link](https://arxiv.org/abs/2510.05373)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Introduction](https://arxiv.org/html/2607.20538#Sx1.p7.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1),[Compression Conditions](https://arxiv.org/html/2607.20538#Sx4.SSx3.p1.1)\.
- N\. Shazeer \(2019\)Fast transformer decoding: one write\-head is all you need\.External Links:1911\.02150Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p1.1)\.
- K\. Staniszewski and A\. Łańcucki \(2026\)KV cache transform coding for compact storage in LLM inference\.InInternational Conference on Learning Representations,External Links:2511\.01815,[Link](https://openreview.net/forum?id=aNVKROYpLB)Cited by:[Transform and numerical KV compression\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px3.p1.1)\.
- Z\. Su, H\. Wei, Z\. Chen, W\. Shen, L\. Li, H\. Yu, and K\. Yuan \(2025\)RotateKV: accurate and robust 2\-bit KV cache quantization for LLMs via outlier\-aware adaptive rotations\.InProceedings of the Thirty\-Fourth International Joint Conference on Artificial Intelligence,pp\. 6200–6208\.External Links:[Document](https://dx.doi.org/10.24963/ijcai.2025/690)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Introduction](https://arxiv.org/html/2607.20538#Sx1.p7.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1),[Compression Conditions](https://arxiv.org/html/2607.20538#Sx4.SSx3.p1.1),[Mechanism](https://arxiv.org/html/2607.20538#Sx5.SSx3.p2.1)\.
- Y\. Sun, R\. Liu, H\. Bai, H\. Bao, K\. Zhao, Y\. Li, J\. Hu, X\. Yu, L\. Hou, C\. Yuan, X\. Jiang, W\. Liu, and J\. Yao \(2025\)FlatQuant: flatness matters for LLM quantization\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 57587–57613\.External Links:[Link](https://proceedings.mlr.press/v267/sun25l.html)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p7.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1)\.
- J\. Tang, Y\. Zhao, K\. Zhu, G\. Xiao, B\. Kasikci, and S\. Han \(2024\)QUEST: query\-aware sparsity for efficient long\-context LLM inference\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 47901–47911\.External Links:[Link](https://proceedings.mlr.press/v235/tang24l.html)Cited by:[Token, page, and dynamic cache selection\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px2.p1.1)\.
- A\. Tseng, J\. Chee, Q\. Sun, V\. Kuleshov, and C\. De Sa \(2024\)QuIP\#: even better LLM quantization with hadamard incoherence and lattice codebooks\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 48630–48656\.External Links:[Link](https://proceedings.mlr.press/v235/tseng24a.html)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Introduction](https://arxiv.org/html/2607.20538#Sx1.p7.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1)\.
- G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. Lewis \(2024\)Efficient streaming language models with attention sinks\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=NG7sS51zVF)Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Token, page, and dynamic cache selection\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px2.p1.1)\.
- J\. Y\. Yang, B\. Kim, J\. Bae, B\. Kwon, G\. Park, E\. Yang, S\. J\. Kwon, and D\. Lee \(2024\)No token left behind: reliable KV cache compression via importance\-aware mixed precision quantization\.External Links:2402\.18096Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[KV\-cache quantization and rotations\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px1.p1.1)\.
- Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. Ré, C\. Barrett, Z\. Wang, and B\. Chen \(2023\)H2​OH\_\{2\}O: heavy\-hitter oracle for efficient generative inference of large language models\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[Introduction](https://arxiv.org/html/2607.20538#Sx1.p2.1),[Token, page, and dynamic cache selection\.](https://arxiv.org/html/2607.20538#Sx2.SS0.SSS0.Px2.p1.1)\.

## Appendix

The complete code, configurations, checkpoints, and experiment outputs are available athttps://huggingface\.co/cccat6/Codec\-Gauge\. The repository contains raw outputs for the gauge\-size sweep, coordinate controls, 27B model check, task scoring, memory diagnostics, reproduction scripts, configurations, checkpoints, generated experiment results, CUDA/zfp build notes, and hardware/software environment records\. The released configurations record the exact checkpoint identifiers and experiment settings used for all reported runs\.

Appendix Table 1:Formal\-run environment and experiment scale used for the released results\. The complete sanitized hardware and software record is included in the released artifact\.
Appendix Table 2:Metric definitions used in the full\-matrix, task\-prompt, and storage analyses\. The cache metrics measure reconstruction of internal KV tensors, while the output metrics measure how much the recovered\-cache model deviates from the full\-cache reference distribution\.
Appendix Table 3:Exact aggregate learned\-gauge reductions for the maing=16g=16matrix\. Each cell reports the mean reduction relative to identity coordinates over six models and the33,44, and66bits/value operating points, followed by the number of model–rate pairs improved out of1818\. The table complements the main figures by making the magnitude and consistency of the effect explicit: zfp and block\-uniform improve on every paired condition across all four metrics, while KIVI\-style quantization remains positive on average and wins most KV\-NRMSE pairs despite its stronger backend\-specific quantization structure\.
Appendix Table 4:Exact zfp coordinate\-control summary at the intermediate operating points\. All controls use the same backend, token stream, measured rates, and evaluator; only the cache\-coordinate map changes\. The fixed and data\-derived transforms often help, confirming that cache coordinates matter, but their improvements are uneven across output and reconstruction metrics\. The learned gauge is the only control that improves every model–rate pair for all four metrics, supporting the frequency\-distribution objective as more than a generic rotation effect\.![Refer to caption](https://arxiv.org/html/2607.20538v1/x6.png)Appendix Figure 1:Per\-model KL divergence for the maing=16g=16matrix across all evaluated zfp, block\-uniform, and KIVI settings\. Each subplot keeps the model fixed and compares identity, random, and learned coordinates under matched backend settings and measured rates\. The learned gauge improves the main trend across model families rather than relying on one checkpoint or one backend setting, while the random\-gauge rows show the strength of generic orthogonal controls\.![Refer to caption](https://arxiv.org/html/2607.20538v1/x7.png)Appendix Figure 2:Gauge\-size sweep overg=8g=8,1616,3232, and full\-head gauges at the intermediate operating points\. The same learned\-coordinate effect appears across group sizes, showing that the conclusion is not tied to a single matrix dimension\. The main text usesg=16g=16as a representative middle setting: it is compact enough to keep the gauge local and inexpensive, while preserving the trend observed in both smaller and larger gauges\.![Refer to caption](https://arxiv.org/html/2607.20538v1/x8.png)Appendix Figure 3:Coordinate\-control breakdown for zfp by rate, with the identity zero baseline omitted to expose the relative spread among non\-identity controls\. Random, Hadamard, DCT, PCA/KLT, and learned gauges are all evaluated through the same zfp compression/decompression path\. The learned gauge provides the most consistent positive reduction across output perturbation and KV reconstruction metrics, while PCA/KLT illustrates why a reconstruction\-oriented data basis alone is not sufficient for the codec\-facing objective\.![Refer to caption](https://arxiv.org/html/2607.20538v1/x9.png)Appendix Figure 4:Gemma\-3\-27B metrics under identity, random, and learned gauges\. This run uses the same zfp backend and intermediate operating points as the main coordinate study, but on a substantially larger checkpoint\. The larger\-model check follows the same direction as the six\-model matrix, supporting the interpretation that the learned cache\-coordinate effect is not restricted to the small\-model sweep\.![Refer to caption](https://arxiv.org/html/2607.20538v1/x10.png)Appendix Figure 5:Task\-prompt scoring breakdown by task family\. The prompts cover needle\-style retrieval, variable tracking, and LongBench\-v2\-style multiple\-choice checks under the same compressed\-cache conditions\. Accuracy is included, but the likelihood\-based columns are the intended diagnostic signal here: when moderate compression leaves many discrete decisions unchanged, correct\-answer likelihood and perturbation metrics reveal the quality margin that accuracy alone can hide\.![Refer to caption](https://arxiv.org/html/2607.20538v1/x11.png)Appendix Figure 6:Gauge training and spectral concentration across gauge sizes\. The loss curves verify that each gauge size learns the intended frequency\-distribution objective, and the concentration bars show that the final gauges consistently reshape the codec\-facing spectrum\. This supports the main mechanism claim: the learned coordinate system changes the numerical geometry seen by the backend rather than only exploiting a single rate or a single model\.![Refer to caption](https://arxiv.org/html/2607.20538v1/x12.png)Appendix Figure 7:No\-compression sanity checks for the full cache, cache clone, random\-gauge inverse, and learned\-gauge inverse paths\. These rows isolate implementation and numerical round\-trip error without lossy compression or quantization\. Perturbations remain near numerical noise, which verifies that the losses reported in the main matrix arise from backend compression or quantization after coordinate shaping, not from the gauge transform itself\.