RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

Hugging Face Daily Papers Papers
kv-cache cache-eviction inference-efficiency long-context compression self-distillation

Summary

RestoreKV introduces a learned restoration mechanism that complements query-agnostic KV cache eviction, recovering full-cache behavior under aggressive budgets by generating a compact context-conditioned restore cache via a single LoRA-adapted pass, improving performance on four long-context benchmarks.

Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets. Existing methods primarily improve which original KV pairs are retained. We introduce RestoreKV, which complements this selection-based formulation with learned restoration under the same total KV budget. Our key insight is that, although the information lost through eviction is context-specific, the mechanism for generating its compact complement can be shared across contexts. After context prefill, a few restore tokens attend to the full KV cache in a single LoRA-adapted pass, generating a compact, context-conditioned restore cache. The base importance scorer and eviction rule remain unchanged, and the adapters are disabled for all subsequent queries and decoding. RestoreKV is trained through parameter-efficient self-distillation from the frozen full-cache model, optimizing only 0.4% of the parameters and requiring no task-specific tuning. Across four backbones and four long-context benchmarks, RestoreKV substantially reduces compression-induced degradation. On Qwen3-4B, it improves 59 of 60 paired, budget-matched settings across five base eviction methods; at a 5% budget, it raises KVzip from 38.2 to 73.2 on RULER-4K. Applied to KVzip+, RestoreKV reaches 86.4 RULER accuracy at 16times compression on the KVPress Benchmark, while adding less than 0.5% one-time cache-construction overhead in a 32K-context evaluation. Our project page is available at https://paper.pnu-cvsp.com/RestoreKV/
Original Article
View Cached Full Text

Cached at: 08/05/26, 05:46 PM

Paper page - RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

Source: https://huggingface.co/papers/2608.01247

Abstract

Query-agnosticKVcacheevictioncompressesacontextonceandreusestheresultingcacheforarbitraryfuturequeries,butperformancecancollapseundertightbudgets.ExistingmethodsprimarilyimprovewhichoriginalKVpairsareretained.WeintroduceRestoreKV,whichcomplementsthisselection-basedformulationwithlearnedrestorationunderthesametotalKVbudget.Ourkeyinsightisthat,althoughtheinformationlostthroughevictioniscontext-specific,themechanismforgeneratingitscompactcomplementcanbesharedacrosscontexts.Aftercontextprefill,afewrestoretokensattendtothefullKVcacheinasingleLoRA-adaptedpass,generatingacompact,context-conditionedrestorecache.Thebaseimportancescorerandevictionruleremainunchanged,andtheadaptersaredisabledforallsubsequentqueriesanddecoding.RestoreKVistrainedthroughparameter-efficientself-distillationfromthefrozenfull-cachemodel,optimizingonly0.4%oftheparametersandrequiringnotask-specifictuning.Acrossfourbackbonesandfourlong-contextbenchmarks,RestoreKVsubstantiallyreducescompression-induceddegradation.OnQwen3-4B,itimproves59of60paired,budget-matchedsettingsacrossfivebaseevictionmethods;ata5%budget,itraisesKVzipfrom38.2to73.2onRULER-4K.AppliedtoKVzip+,RestoreKVreaches86.4RULERaccuracyat16timescompressionontheKVPressBenchmark,whileaddinglessthan0.5%one-timecache-constructionoverheadina32K-contextevaluation.Ourprojectpageisavailableathttps://paper.pnu-cvsp.com/RestoreKV/

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2608\.01247

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### higokri/RestoreKV Text Generation• Updated1 day ago • 1

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.01247 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.01247 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

KVFetch: Temporal Prefetching for the Missing Half of KV Cache Compression

arXiv cs.LG

The paper introduces KVFetch, a training-free framework that adds a temporal recall channel to score-based KV cache compressors to fix "sequential forgetting" (verbatim copying breaking under compression), recovering copying performance from 0.8 to 78.4 and boosting the RULER-16K average by +8.4 at iso-budget.