TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning

# Query-Agnostic KV Cache Compression via Task-Guided Pruning
Source: [https://arxiv.org/html/2608.03276](https://arxiv.org/html/2608.03276)
###### Abstract

Long\-context inference with large language models \(LLMs\) is constrained by the linear growth of the key–value \(KV\) cache to sequence length\. While pruning offers mitigation, prevailing methods determine query\-specific token importance that cannot be reused across unseen queries\. In contrast, we introduce TaskPress, a framework for task\-guided, query\-agnostic KV cache eviction\. Instead of optimizing the cache for a single query, TaskPress constructs a reusable memory representation conditioned on a high\-level task guide\. The guide functions as a "meta\-query" during prefill to filter irrelevant tokens before downstream queries are issued\. In addition, TaskPress leverages quantization scale factors as a zero\-cost signal for detecting influential representation outliers, providing an efficient proxy for token importance\. Experiments conducted on various tasks with long context input demonstrate that TaskPress efficiently creates a compact, reusable cache across diverse queries\.

TaskPress: Query\-Agnostic KV Cache Compression via Task\-Guided Pruning

Wonpyo Park1,2Seung\-won Hwang11Seoul National University2Google

## 1Introduction

Large language models \(LLMs\) increasingly operate over long contexts\. enabling novel applications reaching up to 2M tokensComaniciet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib2)\)\. During inference, transformers cache intermediate key and value \(KV\) representations, which incurs significant memory overhead due to its linear growth with respect to sequence length\. As a result, KV cache often exceeds model weights, saturating memory and degrading throughput\.

Query\-A:What did Lady Eleanor hide?Query\-B:Did Doctor Black argue near the library?Task Guide \(Meta\-Query\):Track physical evidence, suspicious behaviors, and character alibis\.Context:The rain lashed violently against the manor windows, drowning out the sound of the wind\. It was a cold, miserable Tuesday\.Lady Eleanor quietly slipped the arsenic bottle intoher velvet purse\.The fireplace crackled warmly in the corner, casting long shadows across the Persian rug\. The butler served Earl Grey tea to the guests\.Doctor Black was seen arguing with the victim nearthe library just before midnight\.Figure 1:Comparison of attended context between specific queries and task guide\. Highlighted colors show that conditioning on specific queries restricts attention to unique answers, creating an ‘overfitted’ context when KV cache is pruned in query\-dependent manner,e\.g\.,Query\-Afails to generalize toQuery\-Bor vice versa\. In contrast,Task Guideacts as a meta\-query with a broader semantic scope\. Task\-level information preserved as compressed KV cache, is optimized to cover both queries and remain robust for future inputs\.![Refer to caption](https://arxiv.org/html/2608.03276v1/x1.png)Figure 2:Comparison of KV cache eviction processes\. The upper and lower panels illustrates the attention maps for importance scoring and the resulting cross\-query adaptability, respectively\. While query\-dependent methodsLiet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib5)\); Tanget al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib24)\); Caiet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib23)\)lack adaptability and context reconstructionKimet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib6)\)incurs significant overhead, TaskPress compresses the context \(CTX\) using a task guide \(TSK\) rather than a specific query \(Q\)\. This ensures a query\-agnostic compression with negligible cost\.Although quantizationLiuet al\.\([2024a](https://arxiv.org/html/2608.03276#bib.bib26)\)and KV cache pruningZhanget al\.\([2023](https://arxiv.org/html/2608.03276#bib.bib7)\)reduce memory costs, existing approachesLiet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib5)\); Kimet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib6)\)typically rely on decoding attention to dynamically evict less relevant tokens \(Fig[2](https://arxiv.org/html/2608.03276#S1.F2)a\)\. This makes them inherently*query\-dependent*: because token importance relies on a specific query, the compressed cache cannot be reused for new queriesLiet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib5)\)\. Alternatively, context reconstruction methodsKimet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib6)\)support multiple queries but introduce significant overhead, doubling the prompt length \(Fig[2](https://arxiv.org/html/2608.03276#S1.F2)b\)\. These limitations underscore the need for efficient,*query\-agnostic*compression strategies\.

However, many practical workloads build on tasks, serving multiple queries\. For example, Fig 1 shows that each query attends to different context therefore constructing query\-specific KV caches leads to redundant computation and prevents efficient reuse of the compressed context\.

In this work, we propose a different perspective: instead of optimizing KV pruning for a single query, we aim to construct a*query\-agnostic compressed memory*that can serve future queries drawn from the same task\. By doing so, we move KV pruning from query\-time to task\-time\. This decoupling allows the compressed cache to remain highly reusable across multiple turns and robust even when subsequent user queries exhibit moderate task drifts from the original intent\.

To this end, we realize this conceptual shift throughTaskPress, a framework for task\-guided, query\-agnostic KV cache eviction\. To construct this guide, we propose a dual approach depending on the deployment scenario\. For*known tasks*, we find that simply prompting an off\-the\-shelf LLM to generate a zero\-shot task description already achieves a strong baseline performance\. However, for*unknown or dynamic tasks*where the specific objective cannot be predefined, we introduce an evolutionary prompt optimization method\. By analyzing a small calibration set of queries, this method automatically infers the underlying user intention and iteratively refines the task guide\.

Furthermore, we repurpose scale factors from quantizationLiuet al\.\([2024b](https://arxiv.org/html/2608.03276#bib.bib17)\)as a zero\-cost proxy to identify influential outlier tokens within Value\. Incorporating this outlier detection, TaskPress achieves highly efficient compression, which we empirically validate on LongBenchBaiet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib8)\)and RULERHsiehet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib9)\)over both query\-agnostic and query\-dependent baselines\.

## 2Related Work

##### KV Cache Pruning and Eviction\.

Early approaches rely on static or heuristic\-based token eviction: StreamingLLMXiaoet al\.\([2023](https://arxiv.org/html/2608.03276#bib.bib22)\)retains initial attention sinks to stabilize generation for infinite\-length inputs, while H2OZhanget al\.\([2023](https://arxiv.org/html/2608.03276#bib.bib7)\)drops tokens with low accumulated attention scores\. Building on these foundations, more recent work shifts towardquery\-awarepruning\. Approaches such as SnapKVLiet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib5)\), PyramidKVCaiet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib23)\), and QuestTanget al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib24)\)compute token importance dynamically based on the attention allocation from a specific user query\. Although these query\-dependent methods achieve high compression rates for single\-turn inference, compressed memory cannot be safely reused for new queries over the same context, enforcing redundant and expensive prefill computations in multi\-turn or multi\-query workloads\.

##### Query\-Agnostic Compression\.

To overcome the limitations of query\-dependent pruning, query\-agnostic KV cache management approaches include Expected AttentionDevotoet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib25)\), estimating token importance based on the anticipated distribution of future queries\. Alternatively, KVZipKimet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib6)\)addresses cache reusability by introducing a context reconstruction to adapt compressed cache to novel queries\. However, such reconstruction doubles the prompt length during the prefill stage, incurring a severe quadratic computational overhead∼𝒪​\(Lc2\)\\sim\\mathcal\{O\}\(L\_\{c\}^\{2\}\)\.

##### Our Distinction\.

Ours is a query\-agnostic approach with distinction of using task guide as a comprehensive query encapsulating anticipated future queries\. Another distinction of TaskPress is repurposing pre\-computed quantization scale factorsLiuet al\.\([2024b](https://arxiv.org/html/2608.03276#bib.bib17),[a](https://arxiv.org/html/2608.03276#bib.bib26)\)readily available in serving pipelines, as a zero\-cost proxy to detect high\-entropy Value outliers\. Lastly, TaskPress employs an evolutionary search to discover the optimal task guide for unknown task inspired by prompt optimizer like GEPAAgrawalet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib28)\)\.

## 3Method

We introduceTaskPress, a framework driven by four core mechanisms\. First, we establishkeyimportance guided by a meta\-query that estimates the expected distribution of downstream queries \(Section[3\.2](https://arxiv.org/html/2608.03276#S3.SS2)\)\. Second, we determinevalueimportance derived from the quantization scale \(Section[3\.3](https://arxiv.org/html/2608.03276#S3.SS3)\)\. Third, we integrate these metrics through a multiplicative scoring function \(Section[3\.4](https://arxiv.org/html/2608.03276#S3.SS4)\)\. Finally, we employ an evolutionary search algorithm for task guide discovery \(Section[3\.5\.2](https://arxiv.org/html/2608.03276#S3.SS5.SSS2)\)\.

### 3\.1Problem Statement

Utilizing a task guideTTas a stable meta\-query, we decouple compression from specific query, enabling the creation of a query\-agnostic subset that can be reused by future user queries aligned with the task’s objective \(We detail the evolutionary optimization of this optimal guideTTin Section[3\.5\.2](https://arxiv.org/html/2608.03276#S3.SS5.SSS2)\.\)

Given a KV cache of lengthLcL\_\{c\}\. our goal is to select a subset of indices of contextℐ⊂\{1,…,Lc\}\\mathcal\{I\}\\subset\\\{1,\\dots,L\_\{c\}\\\}for each head with cardinality\|ℐ\|≪Lc\|\\mathcal\{I\}\|\\ll L\_\{c\}, such that the compressed KV cache preserves performance on subsequent queries governed by its task\.

### 3\.2Task\-Guided Key Importance

During the prefill phase, the task description of lengthLtL\_\{t\}is appended to the context of lengthLcL\_\{c\}; subsequently, the KV cache corresponding to the task description is evicted immediately after prefill\.

Let𝐀∈ℝLt×Lc\\mathbf\{A\}\\in\\mathbb\{R\}^\{L\_\{t\}\\times L\_\{c\}\}denote the attention map between the task description and input context, sliced from the entire attention map of sizeLc\+t×Lc\+tL\_\{c\+t\}\\times L\_\{c\+t\}\.𝐀j,i\\mathbf\{A\}\_\{j,i\}represents the weight assigned to context tokeniiby task tokenjj\. By aggregating scores across the task dimension and applying 1D average pooling with kernel sizeKKto capture local dependencies, the task\-guided score for theii\-th context token is:

sikey=1K​∑k=−⌊K/2⌋⌊K/2⌋∑j=1Lt𝐀j,i\+ks^\{\\text\{key\}\}\_\{i\}=\\frac\{1\}\{K\}\\sum\_\{k=\-\\lfloor K/2\\rfloor\}^\{\\lfloor K/2\\rfloor\}\\sum\_\{j=1\}^\{L\_\{t\}\}\\mathbf\{A\}\_\{j,i\+k\}\(1\)
Since this derives strictly from query\-key interactions,sikeys^\{\\text\{key\}\}\_\{i\}exclusively measures Key importance\.

### 3\.3Quantization Scale as Value Importance

However, the task\-guided Key score does not measure the density of the retrieved value\. Since self\-attention operates as a weighted sum of attention weights and value, capturing the importance of value is vital as well\. To address this, we propose to identify activation outliers, which act as effective indicators of high\-entropy, critical features\.

Our distinction is reusing quantization scale factors as such measure\. Specifically, assuming a per\-token quantizationLiuet al\.\([2024b](https://arxiv.org/html/2608.03276#bib.bib17)\)scheme, the scale factorγi\\gamma\_\{i\}of a value𝐯i\\mathbf\{v\}\_\{i\}is calculated as:

γi=max⁡\(\|𝐯i\|\)2b−1−1\\gamma\_\{i\}=\\frac\{\\max\(\|\\mathbf\{v\}\_\{i\}\|\)\}\{2^\{b\-1\}\-1\}\(2\)
Since the bit\-widthbbis constant across all tokens,γi\\gamma\_\{i\}is directly proportional to the maximum magnitude of𝐯i\\mathbf\{v\}\_\{i\}\. We therefore define our value outlier score assivalue=γis^\{\\text\{value\}\}\_\{i\}=\\gamma\_\{i\}\. A key advantage of this approach is its zero\-overhead: because quantization is the de facto standard for efficient LLM serving,γi\\gamma\_\{i\}is already computed and readily available in memory, requiring no additional operations\. We’ve found thatγi\\gamma\_\{i\}is an efficient proxy for the norm and discussed further on Section[4\.3\.2](https://arxiv.org/html/2608.03276#S4.SS3.SSS2)\.

### 3\.4KV Importance Scoring

To construct the final task\-conditioned memory, the importance score for cache retention is computed as the element\-wise product of the task\-guided key importance and the value outlier score:

si=sikey⋅sivalues\_\{i\}=s^\{\\text\{key\}\}\_\{i\}\\cdot s^\{\\text\{value\}\}\_\{i\}\(3\)
By employing a multiplicative interaction, it must concurrently satisfy two conditions: it must bestructurally retrievablevia the task\-aligned attention space \(high Key score\) andintrinsically richin semantic information \(high Value outlier score\)\. Finally, the indicesℐ\\mathcal\{I\}corresponding to the tokens with the highest combined scoressis\_\{i\}are selected\.

Table 1:LongBench results with zero\-shot task guide\. Eviction Ratio denotes the percentage of the KV cache that has been pruned\. Boldface and underline denote the best and the second\-best accuracy, respectively\.⋅int4\\cdot^\{\\texttt\{int4\}\}denotes quantized KV cache\. By default, all methods use quantized KV cache\.
### 3\.5Task Guide Formulation

In this section, we propose a systematic, end\-to\-end framework to construct an optimal task guideT∗T^\{\*\}\. A task guide encapsulates where future queries will focus\. Therefore, the task guide should define semantic scope or entity to focus on, such as character interactions, dates, or names\.

#### 3\.5\.1Zero\-shot task guide initialization

We utilize existing conversational LLMs to generate a strong, knowledge\-grounded baseline guide for initialization\. We leverage an off\-the\-shelf Gemini app to generate a concise summary of the task objective\. Specifically, for each task we used this prompt template:‘‘Generate a task guidance for \{task\} in 2\-\-3 sentences\.’’\. The resulting natural language is used directly as the task guide for the subsequent attention computation with minimal modification\.

In our experiment, the initial task guides generated by off\-the\-shelf LLMs already serve as a strong static baseline \(Section[4\.1\.1](https://arxiv.org/html/2608.03276#S4.SS1.SSS1)\)\.

#### 3\.5\.2Evolutionary task guide discovery

For unknown tasks, zero\-shot prompts are inapplicable because LLM does not aware of task’s objective\. To address this, we propose generating a task guide by inferring intent from a small collection of queries, consolidating their shared informational needs into a single unifying meta\-query\.

Because directly optimizing natural language via gradient descent is intractable, we adopt an evolutionary Algorithm[1](https://arxiv.org/html/2608.03276#alg1)to iteratively refine candidate guides\. Operating on a small calibration set \(𝒟cal\\mathcal\{D\}\_\{\\text\{cal\}\}\), the algorithm evaluates each generation, selects top\-performing candidates based on downstream task accuracy, and converges on an optimized meta\-queryT∗T^\{\*\}\. Furthermore, this framework can be repurposed for zero\-shot task guide refinement\. Our evolutionary refinement yields accuracy gains, as demonstrated in Table[3](https://arxiv.org/html/2608.03276#S4.T3)\.

Algorithm 1Task Guide Discovery0:Calibration set

𝒟cal\\mathcal\{D\}\_\{\\text\{cal\}\}, LLM mutator

ℳ\\mathcal\{M\}, Total generations

GG, Selection size

KK
1:Initialize Candidate Pool𝒯\\mathcal\{T\}:

2:\# Sample calibration queries

Qcal∼𝒟calQ\_\{\\text\{cal\}\}\\sim\\mathcal\{D\}\_\{\\text\{cal\}\}
3:\# Generate

NNinitial candidates:

4:

𝒯0←ℳ​\(Qcal;N\)\\mathcal\{T\}\_\{0\}\\leftarrow\\mathcal\{M\}\(Q\_\{\\text\{cal\}\};N\)
5:for

g=1g=1to

GGdo

6:Evaluate & Select:

7:

𝒯best←Top−⁡K​\(𝒯g−1,Score​\(⋅,𝒟cal\)\)\\mathcal\{T\}\_\{\\text\{best\}\}\\leftarrow\\operatorname\{Top\-\}K\(\\mathcal\{T\}\_\{g\-1\},\\text\{Score\}\(\\cdot,\\mathcal\{D\}\_\{\\text\{cal\}\}\)\)
8:Mutate

9:\# Add

NNnew candidates:

10:

𝒯g←ℳ​\(𝒞best,Qcal;N\)∪𝒯best\\mathcal\{T\}\_\{g\}\\leftarrow\\mathcal\{M\}\(\\mathcal\{C\}\_\{\\text\{best\}\},Q\_\{\\text\{cal\}\};N\)\\cup\\mathcal\{T\}\_\{\\text\{best\}\}
11:endfor

12:\# Return the best task guide:

13:return

T∗=arg⁡maxT∈𝒯G⁡Score​\(T,𝒟cal\)T^\{\*\}=\\arg\\max\_\{T\\in\\mathcal\{T\}\_\{G\}\}\\text\{Score\}\(T,\\mathcal\{D\}\_\{\\text\{cal\}\}\)

## 4Experiments

Baselines\.We compare ours against recent state\-of\-the\-art methods SnapKVLiet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib5)\), PyramidKVCaiet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib23)\), StreamingLLMXiaoet al\.\([2023](https://arxiv.org/html/2608.03276#bib.bib22)\), ExpectedAttentionDevotoet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib25)\), ThinKXuet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib29)\)and KVzipKimet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib6)\)using LLaMADubeyet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib3)\)and Qwen3Yanget al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib15)\)models\.

Quantization\.To simulate realistic deployment, we applyint4KV cache quantizationLiuet al\.\([2024a](https://arxiv.org/html/2608.03276#bib.bib26)\)\. There were negligible impacts on quality\. This yields 4×\\timescompression by default; combined with 75% eviction total compression reaches 16×\\times\.

Eviction\.We assign uniform budget for each KV head and indicesℐ\\mathcal\{I\}are selected per head independently\. Eviction was performed after the prefill optimizing the decoding process\. The task guide prompt is utilized exclusively during the eviction phase and is discarded immediately thereafter\.

Evolution\.We retain the topK=2K=2and generate newN=3N=3candidates\. The evolutionary process is conducted forG=4G=4generations\.

Code\.We conducted all of our experiments on top of kvpress github repositoryDevotoet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib25)\)with official implementations of other methods\.

### 4\.1Evaluation on Long\-Context Benchmarks

#### 4\.1\.1Results of zero\-shot task guide

We assess our method’s query\-agnostic long\-context capabilities using the LongBench\. For this specific evaluation, TaskPress employs only the initial, zero\-shot task guides generated directly by an off\-the\-shelf LLM to demonstrate that task guide serves as a strong baseline even without optimizations\. Following established conventionsKimet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib6)\); Senguptaet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib27)\), baselines such as SnapKV and PyramidKV utilize the last context window instead of an actual query\.

As shown in Tables[1](https://arxiv.org/html/2608.03276#S3.T1)and[2](https://arxiv.org/html/2608.03276#S4.T2), TaskPress outperforms baselines in most scenarios, achieving the highest overall average\. Under aggressive 75% compression, standalone channel\-wise pruning like ThinKXuet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib29)\)suffers a severe performance drop\. However, combining TaskPress with ThinK \(each evicting 50%, resulting in 75% total compression\) yields the best performance\. This demonstrates that a simple zero\-shot LLM guide serves as a highly effective “meta\-query”\.

Table 2:LongBench results of a large model Qwen3\-32B on Single and Multi\-Document QA benchmarks\.
#### 4\.1\.2Results of evolved task guide

Table 3:Impact of evolutionary task guide optimization on QA tasks\.∗refers to an experiment with an evolved task guide\.Unknownrefers to task guides discovered solely from query sets\.Knownrefers to task guides with zero\-shot initialization \(Sec\.[3\.5\.1](https://arxiv.org/html/2608.03276#S3.SS5.SSS1)\)\. Values in parentheses highlight the absolute score improvement from evolution\.To simulate a scenario where TaskPress is deployed on unknown tasks, we employ our evolutionary discovery flow, allowing the model to infer the task guide directly from queries\. Lacking a dedicated LongBench validation split, we optimize using 50 calibration samples repurposed from the test set, strictly excluding them from the final evaluation\. At a 50% eviction rate, Table[3](https://arxiv.org/html/2608.03276#S4.T3)shows this pipeline successfully discovers effective guides for unknown QA tasks\. Furthermore, applying this pipeline to the zero\-shot task guides \(Section[3\.5\.1](https://arxiv.org/html/2608.03276#S3.SS5.SSS1)\) effectively improves overall accuracy\.

#### 4\.1\.3Results on retrieval capability

We evaluate our method on RULERHsiehet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib9)\)needle\-in\-a\-haystack subtasks\. We formed a unified task guide by concatenating short instructions for each subtasks, thereby challenging the method to handle multiple subtasks simultaneously\.

As shown in Table[4](https://arxiv.org/html/2608.03276#S4.T4), we achieve best accuracy at a 75% eviction rate\. For 50% eviction rate, we achieved the second best accuracy against self\-reconstruction counterpart which is more costly\. While TaskPress outperforms others on most of subtasks, but it gained relatively low accuracy on MK\-2/3\. We attribute this degradation to attention dilution caused by an excessive number of target needles, which we analyze in detail in Appendix[B](https://arxiv.org/html/2608.03276#A2)\.

Table 4:Results on RULER\-16K with LLaMA\-3\.1\-8B\-Instruct\. Boldface and underline denotes the best score among 75% and 50% eviction rates respectively\.AllSnapKVPyramidKVExpectedAttentionKVZipTaskPressSub\-Task100%50% / 75%50% / 75%50% / 75%50% / 75%50% / 75%CWE88\.175\.7 / 20\.540\.2 / 5\.5279\.3 / 61\.864\.3 / 23\.586\.3/71\.1FWE91\.789\.7 / 79\.086\.2 / 74\.985\.1 / 78\.489\.9/ 82\.588\.9 /87\.0MK\-199\.492\.4 / 49\.291\.0 / 48\.891\.0 / 35\.090\.4 / 66\.299\.4/99\.6MK\-210043\.8 / 17\.839\.6 / 12\.268\.2 / 17\.699\.8/82\.450\.0 / 10\.6MK\-399\.616\.8 / 3\.615\.4 / 3\.635\.0 / 2\.078\.4/5\.27\.2 / 1\.0MQ99\.183\.9 / 34\.684\.8 / 31\.474\.5 / 24\.093\.7 / 76\.499\.3/98\.6MV98\.979\.6 / 30\.384\.0 / 32\.071\.2 / 15\.391\.4 / 70\.098\.9/98\.1S\-110098\.0 / 91\.692\.2 / 87\.699\.8 / 99\.2100/100100/100S\-210097\.6 / 81\.299\.6 / 82\.493\.4 / 48\.890\.0 / 62\.4100/100S\-310014\.8 / 3\.49\.6 / 2\.618\.0 /6\.084\.6 / 29\.292\.2/44\.0QA\-181\.651\.6 / 32\.442\.6 / 24\.876\.0/ 63\.474\.8 /64\.654\.8 / 37\.6QA\-256\.844\.6 / 34\.432\.4 / 26\.252\.6/ 44\.452\.4 /46\.650\.4 / 42\.2VT99\.893\.8 / 78\.688\.64 / 77\.3697\.8 / 77\.799\.8 /99\.6100/ 99\.4\[2pt/1pt\] Avg93\.567\.9 / 42\.862\.0 / 39\.272\.4 / 44\.185\.3/ 62\.279\.0 /68\.4
#### 4\.1\.4Comparison with query\-aware method

We investigate whether task\-guided KV cache eviction can still compete with query\-dependent baselines in their native settings\. We evaluate this on two QA datasets: NarrativeQAKočiskỳet al\.\([2018](https://arxiv.org/html/2608.03276#bib.bib18)\)and QMSumZhonget al\.\([2021](https://arxiv.org/html/2608.03276#bib.bib19)\)\. These benchmarks were selected because they pair a single document with multiple questions, a setting where query\-dependent methods typically excel\. In Fig[3](https://arxiv.org/html/2608.03276#S4.F3), the green line labeled ‘query’ represents the native performance of query\-dependent baselines; SnapKV\. Our results demonstrate that TaskPress \(red line\) achieves performance comparable to or better than these baselines\. Specifically, even at a highly aggressive eviction rate of 75%\.

![Refer to caption](https://arxiv.org/html/2608.03276v1/assets/fig3.png)Figure 3:Accuracy comparison on the NarrativeQA and QMSum datasets with LLaMA\-3\.1\-8B\-Instruct\. We evaluate four compression strategies: \(1\)query\-dependent, where the specific query guides eviction; \(2\)query\-reuse, where a cache compressed for one query is applied to others; \(3\)lastcontext, where the final context segment serves as a proxy for query; and \(4\) Task guide \(Ours\), which utilizes a high\-level task guide\.

### 4\.2Impact on End\-to\-End Efficiency

#### 4\.2\.1Cost of KV cache eviction

We analyzed the computational overhead of the eviction process\. Our method utilize attention map between context and task guides and eviction process hasO​\(Lt​Lc\)O\(L\_\{t\}L\_\{c\}\)complexity\. In contrast, reconstruction\-based baselinesKimet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib6)\)scale quadraticallyO​\(Lc2\)O\(L\_\{c\}^\{2\}\)\. SinceLt≪LcL\_\{t\}\\ll L\_\{c\}, our approach significantly lowers overhead, achieving an 80×\\timeslatency improvement \(see Fig\.[4](https://arxiv.org/html/2608.03276#S4.F4)\)\.

![Refer to caption](https://arxiv.org/html/2608.03276v1/assets/output.png)Figure 4:\(Left\) the eviction latency of each method\. \(Right\) panel depicts the memory savings achieved during decoding after eviction\. The experiments were conducted in a context length of 32K on LLaMA\-3\.1\-8B\.
#### 4\.2\.2Cost of task guide evolution

Using the setup in Section[4](https://arxiv.org/html/2608.03276#S4)with 50 calibration samples, the entire procedure took∼\\sim25\.2 minutes\. This comprised 41\.4 seconds for 5 Gemini\-2\.0\-Flash API calls \(guide initialization and mutation; 5,980 input and 486 output tokens\) and 1,459\.7 seconds for local calibration set evaluation on a single A100 40GB GPU\. Crucially, this∼\\sim25\-minute runtime is a strictly one\-time, offline setup cost\.

#### 4\.2\.3Decoding and prefill performance

Table 5:Performance impact during decoding\.Table 6:Performance impact during the prefill phase\. The value after the slash \(/\) indicates the total prompt length, including the task guide\.We evaluated performance impacts during the prefill and decoding phases—specifically time\-to\-first\-token \(TTFT\), time\-per\-output\-token \(TPOT\), throughput, and peak memory\. Tests were conducted in a long context \(64K\) setup with LLaMA\-3\.1\-8B on single A100 80GB\. We adopted flash attentionDao \([2023](https://arxiv.org/html/2608.03276#bib.bib21)\)\. Reducing the KV cache size directly accelerates decoding speed\. While task\-guided pruning often raises concerns about prefill overhead, our results show it is negligible\. At a 75% eviction rate with a 64\-token task guide, TTFT increases by only 1\.2%\. This is because TaskPress maintains linear complexity by computing attention strictly between the short task guide and the context\. This initial overhead is easily offset by the resulting gains in decoding speed\.

#### 4\.2\.4Efficiency gain in multi\-query setting

Lastly, we analyzed the efficiency gains of KV cache reuse in a multi\-query setting\. Query\-dependent methods inherently face amemory\-latency trade\-off: because their eviction mechanisms rely on the specific query, systems are forced into one of two suboptimal strategies \(see Figure[5](https://arxiv.org/html/2608.03276#S4.F5)\)\.

Query\-Dependent: Optimize Latency \(Sacrifice Memory\)

full\_kv=prefill\(context\)

forqinqueries:

comp\_kv=evict\(full\_kv,q\)

generate\(comp\_kv,q\)

Query\-Dependent: Optimize Memory \(Sacrifice Latency\)

forqinqueries:

full\_kv=prefill\(context\)

comp\_kv=evict\(full\_kv,q\)

generate\(comp\_kv,q\)

TaskPress: Query\-Agnostic \(Ours\)

full\_kv=prefill\(context\)

comp\_kv=evict\(full\_kv,task\)

forqinqueries:

generate\(comp\_kv,q\)

Figure 5:Pseudo\-code comparison of KV cache eviction strategies between query dependent and TaskPress\.The first strategy prioritizes latency by retaining the full KV cache, which severely inflates memory overhead and negates compression \(Figure[5](https://arxiv.org/html/2608.03276#S4.F5), top\)\. Conversely, the second strategy prioritizes memory by discarding the cache, but suffers high latency since every new query forces a redundant full\-context prefill \(Figure[5](https://arxiv.org/html/2608.03276#S4.F5), middle\)\.

TaskPress breaks this trade\-off by decoupling compression from the query \(Figure[5](https://arxiv.org/html/2608.03276#S4.F5), bottom\), incurring only a one\-time task guide prefill and eviction step\. We validated this by benchmarking end\-to\-end latency and peak memory across varying query counts on an 8K\-token context with 75% eviction in TableLABEL:tab:multi\_query\_latency\. TaskPress outperforms both strategies\. Note that query\-dependent’s latency optimized strategy consumes 1,184\.4 \(MB\) for KV cache which is5 times biggerthan TaskPress \(236\.88 MB\)\. This is because it holds both compressed and full KV cache during generate\.

Table 7:Comparison of end\-to\-end latency \(sec\) in a multi\-query setting\. Lat\-Opt refers to latency optimized whereas Mem\-Opt refers to memory optimized strategy\.

### 4\.3Analysis on Importance Scoring

#### 4\.3\.1Impacts of Key and Value score

We analyzed impacts of the task guide score and value scale score using the single\-document QA datasets\. As shown in Table[8](https://arxiv.org/html/2608.03276#S4.T8), utilizing the task guide score alone yields better performance than the query\-agnostic SnapKV baseline\. Furthermore, combining the task guide score with value scale scoring results in the highest overall accuracy\.

Table 8:Key and Value importance score ablation study\.
#### 4\.3\.2Scale factor as a proxy for Value norm

Self\-attention computes a weighted sum of value vectors, value magnitude is a critical indicator of token importanceGuoet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib30)\); Devotoet al\.\([2025](https://arxiv.org/html/2608.03276#bib.bib25)\)\. Building on this, our key insight is to utilize the scale factor as an efficient proxy for the value norm\. This is supported by a strong Pearson correlation of0\.79±0\.100\.79\\pm 0\.10between the two metrics\. Furthermore, the scale factor shows near\-zero correlation \(0\.06±0\.230\.06\\pm 0\.23\) with the attention map, indicating it captures orthogonal information\. As detailed in Section[4\.3\.1](https://arxiv.org/html/2608.03276#S4.SS3.SSS1), combining these complementary signals ultimately improves accuracy\.

### 4\.4Score aggregation strategy

We compared three strategies to blendskeys^\{\\text\{key\}\}andsvalues^\{\\text\{value\}\}: multiplication, addition with normalization, and the harmonic mean\. Multiplication achieves the highest accuracy on both datasets \(see Table[9](https://arxiv.org/html/2608.03276#S4.T9)\)\.

Table 9:Key and Value score unification ablation study \(Eviction ratio: 75%\)\.
### 4\.5Analysis on Task Guide

#### 4\.5\.1Robustness of underlying model

To evaluate the influence of the source model, we prompted different models to generate a task guide using the same instruction\. Table[10](https://arxiv.org/html/2608.03276#S4.T10)indicate that all evaluated models produced effective task guides, highlighting the robustness of underlying model\.

Table 10:NarrativeQA accuracy of source models\.
#### 4\.5\.2Impact of granularity or length

To control the specificity of the instructions, we vary the length of the task guide across three prompt variations\. As shown in Table[11](https://arxiv.org/html/2608.03276#S4.T11), task guide with moderate length was optimal\.

Table 11:Impacts of task guide granularity on NarrativeQA accuracy\.
#### 4\.5\.3Impact of paraphrasing

We evaluated the impact of wording or paraphrasing\. Firstly, we replaced five words in the task guide with synonyms and combined them to generate five distinct prompt \(P\) sets\. Secondly, to measure the impact of sentence\-level variations, we paraphrased the task guide into several alternative sentences\. The overall robustness is consistently maintained with low \(1%<\) standard deviation\.

Table 12:Impacts of guide wording variations\. ‘W’ denotes word\-level \(synonym replacements\) and ‘S’ denotes sentence\-level \(paraphrasing\)\.
#### 4\.5\.4Task drift during conversation

To assess the impact of misaligned task guides on model performance, we conducted experiments under two conditions: using guides from \(1\) similar but non\-target QA tasks with different semantic focuses, and \(2\) entirely disparate tasks \(e\.g\., NarrativeQA and RULER’s needle\-in\-a\-haystack\)\. As shown in Table[13](https://arxiv.org/html/2608.03276#S4.T13), the original task\-aligned guide yields the highest accuracy\. Notably, performance degradation becomes significant with highly dissimilar guides and higher eviction rates\.

Table 13:Impacts of misaligned task guide on accuracy\.

## 5Conclusion

We present TaskPress, a framework that achieves efficient, query\-agnostic KV cache compression by utilizing task guides as semantic anchors and repurposing quantization scale factors for outlier detection, thereby significantly outperforming existing baselines on standard long\-context benchmarks\.

## 6Limitations

Despite the effectiveness of TaskPress in balancing efficiency and flexibility, few limitations remain\.

Dependency on Task Scope\.While our approach is query\-agnostic within a specific domain, it remains task\-dependent\. The compressed KV cache is optimized strictly for the semantic scope defined by the task guide\. Consequently, if user queries drift significantly from the anticipated task \(e\.g\., asking a coding question during a document summarization session\), the retrieved context may be insufficient, potentially degrading performance compared to full\-cache baselines\.

Extremely Dense Information\.A limitation of TaskPress is its vulnerability to severe attention dilution in artificially dense environments, such as the "needle inside needles" scenario in RULER MK\-2 and MK\-3\. Because we use softmax\-normalized cross\-attention to score token importance, a context entirely saturated with target\-like entities scatters the attention mass\. This causes individual target scores to drop below background noise, leading to unintended cache eviction\. However, such extreme density rarely reflects real\-world workloads\. As detailed in Appendix[B](https://arxiv.org/html/2608.03276#A2), TaskPress maintains robust retrieval performance provided the targets appear at a moderate, realistic frequency\.

Reliance on Quantization Statistics\.Our outlier detection leverages scale factors inherent to quantized models as a zero\-cost proxy for importance\. In full\-precision settings \(e\.g\., FP16 or BF16\) where these pre\-computed artifacts are absent, the requisite statistics must be derived explicitly\. However, this imposes negligible overhead, as the equivalent metric—the maximum absolute value per token—can be computed efficiently on\-the\-fly\. For instance, processing a 64K token context using LLaMA\-3\.1\-8B in full precision requires 9\.66 seconds for the overall prefill, whereas dynamically calculating the absolute min\-max adds merely 0\.26 seconds\. This represents a marginal overhead of roughly 2\.7

## References

- L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang,et al\.\(2025\)GEPA: reflective prompt evolution can outperform reinforcement learning\.arXiv preprint arXiv:2507\.19457\.Cited by:[§2](https://arxiv.org/html/2608.03276#S2.SS0.SSS0.Px3.p1.1)\.
- Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou,et al\.\(2024\)Longbench: a bilingual, multitask benchmark for long context understanding\.InProceedings of the 62nd annual meeting of the association for computational linguistics \(volume 1: Long papers\),pp\. 3119–3137\.Cited by:[Appendix A](https://arxiv.org/html/2608.03276#A1.p1.1),[§1](https://arxiv.org/html/2608.03276#S1.p6.1)\.
- Z\. Cai, Y\. Zhang, B\. Gao, Y\. Liu, Y\. Li, T\. Liu, K\. Lu, W\. Xiong, Y\. Dong, J\. Hu,et al\.\(2024\)PyramidKV: dynamic kv cache compression based on pyramidal information funneling\.arXiv preprint arXiv:2406\.02069\.Cited by:[Figure 2](https://arxiv.org/html/2608.03276#S1.F2),[§2](https://arxiv.org/html/2608.03276#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2608.03276#S4.p1.1)\.
- G\. Comanici, E\. Bieber, M\. Schaekermann, I\. Pasupat, N\. Sachdeva, I\. Dhillon, M\. Blistein, O\. Ram, D\. Zhang, E\. Rosen,et al\.\(2025\)Gemini 2\.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.arXiv preprint arXiv:2507\.06261\.Cited by:[§1](https://arxiv.org/html/2608.03276#S1.p1.1)\.
- T\. Dao \(2023\)Flashattention\-2: faster attention with better parallelism and work partitioning\.arXiv preprint arXiv:2307\.08691\.Cited by:[§4\.2\.3](https://arxiv.org/html/2608.03276#S4.SS2.SSS3.p1.1)\.
- A\. Devoto, M\. Jeblick, and S\. Jégou \(2025\)Expected attention: kv cache compression by estimating attention from future queries distribution\.arXiv preprint arXiv:2510\.00636\.Cited by:[§2](https://arxiv.org/html/2608.03276#S2.SS0.SSS0.Px2.p1.1),[§4\.3\.2](https://arxiv.org/html/2608.03276#S4.SS3.SSS2.p1.2),[§4](https://arxiv.org/html/2608.03276#S4.p1.1),[§4](https://arxiv.org/html/2608.03276#S4.p5.1)\.
- A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Yang, A\. Fan,et al\.\(2024\)The llama 3 herd of models\.arXiv e\-prints,pp\. arXiv–2407\.Cited by:[§4](https://arxiv.org/html/2608.03276#S4.p1.1)\.
- Z\. Guo, H\. Kamigaito, and T\. Watanabe \(2024\)Attention score is not all you need for token importance indicator in kv cache reduction: value also matters\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 21158–21166\.Cited by:[§4\.3\.2](https://arxiv.org/html/2608.03276#S4.SS3.SSS2.p1.2)\.
- C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. Ginsburg \(2024\)RULER: what’s the real context size of your long\-context language models?\.arXiv preprint arXiv:2404\.06654\.Cited by:[Appendix A](https://arxiv.org/html/2608.03276#A1.p2.1),[§1](https://arxiv.org/html/2608.03276#S1.p6.1),[§4\.1\.3](https://arxiv.org/html/2608.03276#S4.SS1.SSS3.p1.1)\.
- J\. Kim, J\. Kim, S\. Kwon, J\. W\. Lee, S\. Yun, and H\. O\. Song \(2025\)KVzip: query\-agnostic kv cache compression with context reconstruction\.arXiv preprint arXiv:2505\.23416\.Cited by:[Figure 2](https://arxiv.org/html/2608.03276#S1.F2),[§1](https://arxiv.org/html/2608.03276#S1.p2.1),[§2](https://arxiv.org/html/2608.03276#S2.SS0.SSS0.Px2.p1.1),[§4\.1\.1](https://arxiv.org/html/2608.03276#S4.SS1.SSS1.p1.1),[§4\.2\.1](https://arxiv.org/html/2608.03276#S4.SS2.SSS1.p1.4),[§4](https://arxiv.org/html/2608.03276#S4.p1.1)\.
- T\. Kočiskỳ, J\. Schwarz, P\. Blunsom, C\. Dyer, K\. M\. Hermann, G\. Melis, and E\. Grefenstette \(2018\)The narrativeqa reading comprehension challenge\.Transactions of the Association for Computational Linguistics6,pp\. 317–328\.Cited by:[§4\.1\.4](https://arxiv.org/html/2608.03276#S4.SS1.SSS4.p1.1)\.
- Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. Chen \(2024\)Snapkv: llm knows what you are looking for before generation\.Advances in Neural Information Processing Systems37,pp\. 22947–22970\.Cited by:[Figure 2](https://arxiv.org/html/2608.03276#S1.F2),[§1](https://arxiv.org/html/2608.03276#S1.p2.1),[§2](https://arxiv.org/html/2608.03276#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2608.03276#S4.p1.1)\.
- Z\. Liu, C\. Zhao, I\. Fedorov, B\. Soran, D\. Choudhary, R\. Krishnamoorthi, V\. Chandra, Y\. Tian, and T\. Blankevoort \(2024a\)SpinQuant: llm quantization with learned rotations\.arXiv preprint arXiv:2405\.16406\.Cited by:[§1](https://arxiv.org/html/2608.03276#S1.p2.1),[§2](https://arxiv.org/html/2608.03276#S2.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2608.03276#S4.p2.2)\.
- Z\. Liu, J\. Yuan, H\. Jin, S\. Zhong, Z\. Xu, V\. Braverman, B\. Chen, and X\. Hu \(2024b\)Kivi: a tuning\-free asymmetric 2bit quantization for kv cache\.arXiv preprint arXiv:2402\.02750\.Cited by:[§1](https://arxiv.org/html/2608.03276#S1.p6.1),[§2](https://arxiv.org/html/2608.03276#S2.SS0.SSS0.Px3.p1.1),[§3\.3](https://arxiv.org/html/2608.03276#S3.SS3.p2.2)\.
- A\. Sengupta, S\. Chaudhary, and T\. Chakraborty \(2025\)Value\-guided kv compression for llms via approximated cur decomposition\.arXiv preprint arXiv:2509\.15038\.Cited by:[§4\.1\.1](https://arxiv.org/html/2608.03276#S4.SS1.SSS1.p1.1)\.
- J\. Tang, Y\. Zhao, K\. Zhu, G\. Xiao, B\. Kasikci, and S\. Han \(2024\)Quest: query\-aware sparsity for efficient long\-context llm inference\.arXiv preprint arXiv:2406\.10774\.Cited by:[Figure 2](https://arxiv.org/html/2608.03276#S1.F2),[§2](https://arxiv.org/html/2608.03276#S2.SS0.SSS0.Px1.p1.1)\.
- G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. Lewis \(2023\)Efficient streaming language models with attention sinks\.arXiv preprint arXiv:2309\.17453\.Cited by:[§2](https://arxiv.org/html/2608.03276#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2608.03276#S4.p1.1)\.
- Y\. Xu, Z\. Jie, H\. Dong, L\. Wang, X\. Lu, A\. Zhou, A\. Saha, C\. Xiong, and D\. Sahoo \(2025\)Think: thinner key cache by query\-driven pruning\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 56691–56709\.Cited by:[§4\.1\.1](https://arxiv.org/html/2608.03276#S4.SS1.SSS1.p2.1),[§4](https://arxiv.org/html/2608.03276#S4.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§4](https://arxiv.org/html/2608.03276#S4.p1.1)\.
- Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. Ré, C\. Barrett,et al\.\(2023\)H2o: heavy\-hitter oracle for efficient generative inference of large language models\.Advances in Neural Information Processing Systems36,pp\. 34661–34710\.Cited by:[§1](https://arxiv.org/html/2608.03276#S1.p2.1),[§2](https://arxiv.org/html/2608.03276#S2.SS0.SSS0.Px1.p1.1)\.
- M\. Zhong, D\. Yin, T\. Yu, A\. Zaidi, M\. Mutuma, R\. Jha, A\. Hassan, A\. Celikyilmaz, Y\. Liu, X\. Qiu,et al\.\(2021\)QMSum: a new benchmark for query\-based multi\-domain meeting summarization\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 5905–5921\.Cited by:[§4\.1\.4](https://arxiv.org/html/2608.03276#S4.SS1.SSS4.p1.1)\.

## Appendix ATask Guides used in our experiments

Table[15](https://arxiv.org/html/2608.03276#A2.T15)details the task guides employed in our LongBenchBaiet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib8)\)experiments for single\- and multi\-document question answering\. These guides were primarily generated by Gemini, with minimal human refinement\.

Table[16](https://arxiv.org/html/2608.03276#A2.T16)details the task guide employed for our evaluation on the RULER benchmarkHsiehet al\.\([2024](https://arxiv.org/html/2608.03276#bib.bib9)\)\. RULER comprises a diverse set of sub\-tasks, including long\-context retrieval \(Needle\-in\-a\-Haystack\), multi\-hop tracing, aggregation, and question answering\. Notably, we utilize a single, unified task guide across all sub\-tasks, demonstrating that our method effectively generalizes across heterogeneous tasks without requiring task\-specific customization\.

The task guides used in other tasks are provided in our attached source code\.

## Appendix BExplanation of the MK\-2/3 lower accuracy in RULER

In this section, we explain why TaskPress has relatively lower accuracy on MK\-2 and MK\-3 tasks of RULER\. MK\-2/3 are the tasks that require finding “a needle inside needles”\. Below is an example of context and question drawn from MK\-2\. MK\-3 shares the same schematic except the key and value are UUIDs\.

``` Context: A special magic number is hidden within the following text. Make sure to memorize it. ... One of the special magic numbers for confused-prince is: 4446065. One of the special magic numbers for innocent-clause is: 5774152. .... One of the special magic numbers for woebegone-emission is: 4123902. Question: What is the special magic number for innocent-clause mentioned in the provided text? ```

Our task guide instructed the model to capture a “needle" with the magic numbers or UUIDs, e\.g\., a snippet of task guide is"…long\-context retrieval by hiding specific facts, such as arbitrary magic numbers or UUIDs in key\-value forms…"\.

TaskPress calculates KV cache importance by taking the mean cross\-attention scores between the task guide and the entire context\. However, because the context is full of needles \(\>820\>820\), attention scores are spread thin, causing attention dilution\.

We conducted an empirical measurement of attention mass of RULER\-16K\. In MK\-1, the task guide sharply focuses on a single needle, assigning the target key an attention score of3\.01×10−43\.01\\times 10^\{\-4\}, which is roughly six times higher than the context mean\. However, MK\-2 and MK\-3 introduce multiple needles, forcing the attention mass to scatter\. Consequently, the attention score for any single target key drastically plummets to1\.2×10−51\.2\\times 10^\{\-5\}—falling significantly below the mean background noise of5\.0×10−55\.0\\times 10^\{\-5\}\.

Since it is highly unrealistic for real\-world data to consist exclusively of target "needles," we conducted an experiment by incrementally increasing the number of needles inserted into the context\. Using the official repository, we re\-generated the RULER MK\-2 dataset with varying needle counts\. Our results demonstrate that as long as the context contains a moderate, realistic number of needles—rather than an overwhelming density—TaskPress maintains robust retrieval performance and effectively captures the targets under presence of multiple needles\.

Table 14:RULER\-16K MK\-2 results varying the number of needlesTable 15:Task guides used in our LongBench QA experiments and randomly sampled example queries for each task\.Table 16:Task guide used in our RULER experiments and randomly sampled example queries\.
## Appendix CMutation & Crossover Prompt

The evolutionary task\-guide search inevolve\_task\_guide\.pyuses a single large\-language\-model prompt to drive the variation step of each generation\. After the top\-kkparent guides are selected by fitness on the calibration set, the prompt in “Mutation & Crossover Prompt Template” is sent to the LLM \(Gemini in our setup\) together with the parents and a batch of real calibration queries\. The LLM is asked to returnNoffspringN\_\{\\text\{offspring\}\}new candidate guides, each produced by*either*mutation\(query\-driven rewriting of a single parent\)*or*crossover\(structure\-preserving blending of two parents\)\. The output guides are then evaluated on the calibration set, and the cycle repeats for the next generation\.

The template is intentionally short and constrained \(2–3 sentences,<64\{<\}64tokens per guide, no commentary\) so that the LLM produces drop\-in replacements rather than free\-form explanations\. The slot legend below lists which substrings are filled in at runtime and which fragments are optional when the downstream task name is not known a priori\.

Mutation & Crossover Prompt TemplateYou are an expert at refining task guides for AI systems\. // Optional task\-name header — omit the entire line when the task identity is unknown a priori\. Current top\-performing task guides\[ for ‘\{task\_name\}’ \]: \{parents\_text\} Here are real sample queries users might ask: \{queries\_text\} Your goal is to produce\{n\_offspring\}improved task guides\. For each guide, apply EITHER mutation OR crossover: Mutation \(query\-driven exploration & refinement\):•Deeply analyze the sample queries to extract specific clues, constraints, and implicit intents required for retrieval \(e\.g\. temporal limits, exact formatting, key entities\)\.•Explicitly embed these extracted query traits into the parent guides\.•While you can make targeted semantic edits, you must also occasionally make BOLD, SIGNIFICANT structural changes or entirely fresh rephrasings to explore new instruction spaces\.Crossover \(structure\-preserving blending\):•When combining parent guides, preserve the core phrasing and structure of each parent\. Blend their complementary ideas by splicing or interleaving specific clauses, rather than rewriting from scratch\.Output exactly\{n\_offspring\}distinct task guides, one per line, numbered 1\. to\{n\_offspring\}\. Each guide must be 2–3 sentences and less than 64 tokens\. No extra commentary\.

##### Slot legend\.

\{red\}entries are runtime substitutions:\{parents\_text\}is the newline\-joined list of parent guides,\{queries\_text\}is the newline\-joined calibration queries, and\{n\_offspring\}is the requested offspring count\.Blue italicfragments mark the*optional*task\-identity hint: when the downstream task isunknown a priori\(e\.g\. open\-domain deployment, ablation studies, or settings where revealing the task name would bias the search\), both the comment line and the “for ‘\{task\_name\}’ ” clause should be removed so the LLM mutates the guides purely from the calibration queries\.