Workload-Aware Caching for Multi-Agent Systems

# Workload-Aware Caching for Multi-Agent Systems
Source: [https://arxiv.org/html/2607.20495](https://arxiv.org/html/2607.20495)
###### Abstract

Multi\-agent systems decompose complex tasks into directed acyclic graphs \(DAGs\) of specialized agent executions, creating natural opportunities for caching intermediate results across queries\. However, existing cache eviction policies treat all cached entries uniformly based on access history, ignoring structural and workload signals uniquely available in agentic execution environments\. We present a workload\-aware eviction policy that combines three signals, namely recomputation cost, DAG dependency count, and agent invocation frequency, into a unified scoring function that retains the most valuable entries under memory constraints\. Evaluated across three multi\-agent benchmarks spanning diverse reuse regimes, our policy reduces latency by up to 64\.7% relative to the uncached baseline and achieves on average a 31\.1% latency reduction over the next best finite\-capacity baseline, while approaching the performance of an unbounded cache and maintaining accuracy on par with or exceeding all competing finite\-capacity methods\. We further show that workload\-aware content caching is complementary to other agentic system optimization methods, including plan\-level caching and parallel agent execution, with each technique targeting a distinct efficiency bottleneck in multi\-agent pipelines\.

## IIntroduction

AI systems are moving from single model architectures towards compound systems that use and coordinate different specialized agents, tools, and reasoning steps to solve queries\. Multi agent systems \(MAS\) have shown capabilities across domains from software engineering to scientific research to autonomous decision making\[[9](https://arxiv.org/html/2607.20495#bib.bib29),[3](https://arxiv.org/html/2607.20495#bib.bib37),[15](https://arxiv.org/html/2607.20495#bib.bib44),[31](https://arxiv.org/html/2607.20495#bib.bib35),[33](https://arxiv.org/html/2607.20495#bib.bib30),[30](https://arxiv.org/html/2607.20495#bib.bib38)\]\. However, this architectural shift has also introduced efficiency challenges, and as agent interactions become more complex with more elaborate plans, computational costs can limit practical deployment in the real world\. The inference costs of modern reasoning models combined with the need for multiple agent invocations per query makes every redundant computation quite expensive\[[14](https://arxiv.org/html/2607.20495#bib.bib43)\], not just in latency but in resource utilization as well\.

Multi agent workloads can show substantial computational overlap across queries\. This redundancy comes up when different queries share common computational patterns\. To give an example, in financial analysis, queries like ”What was Company X’s revenue?” and ”What was Company X’s profit margin?” both share identical initial steps like retrieving financial statements, parsing documents, and extracting standard fields, differing only in the final calculations\. Similarly, document analysis queries asking for summaries versus dataset descriptions might share the same parsing, section identification, and entity extraction operations\. The pattern extends beyond exact repetition as well\. Structurally similar queries like ”Analyze Q1 2023 performance” and ”Analyze Q2 2023 performance” might differ only in date filters but share data aggregation, normalization, and visualization logic\.

Many modern multi agent systems use DAG\-based planning where specialized planning agents decompose tasks into structured execution plans \(Figure[1](https://arxiv.org/html/2607.20495#S1.F1)\)\. These plans are represented as directed acyclic graphs where nodes correspond to computational tasks \(LLM calls, tool executions, data processing\) and edges encode dependencies between those tasks\. An executor then dispatches agents to complete the task while respecting dependencies\. This decomposition into reusable computational units leads to a natural caching opportunity, where if we cache node results, we can reuse them when similar queries require the same operations\. However, unbounded caching quickly finishes available memory, which necessitates eviction policies that decide which cached results to retain\.

![Refer to caption](https://arxiv.org/html/2607.20495v1/figures/dag.png)Figure 1:Multi\-agent workflow for network monitoring represented as a DAG\. Nodes represent steps completed by specialized agents with edges encoding dependencies\. A\) Telemetry Collection, B\) Data Parsing and Normalization, C\) Metric Aggregation and Calculation, D\) Capacity/Usage Analysis, E\) Anomaly Detection, F\) SLA Policy Evaluation, G\) Dashboard and Alert Updates\. Expensive operations benefit significantly from caching\.Traditional eviction strategies like Least Recently Used \(LRU\) or Least Frequently Used \(LFU\) make decisions based purely on access history, with metrics like recency or frequency, treating all cached items uniformly\. This approach is functional, but it ignores several structural and workload signals that are available in multi agent planning systems\. For instance, consider a cache at capacity containing two nodes\. Node A was accessed 30 seconds ago, feeds into 1 downstream calculation \(so it has one outgoing edge\), and takes 0\.3 seconds to recompute\. Node B was accessed 5 minutes ago, feeds into 4 downstream nodes, and takes 8 seconds to recompute due to expensive database queries and document parsing\. LRU evicts Node B based on older access time\. When the next query arrives requiring Node B’s outputs but not Node A’s, the system must recompute it \(8 seconds\)\. Evicting Node A would have cost only 0\.3 seconds\. This is just one example of uniform treatment that fails to recognize that Node B’s structural importance \(feeding multiple dependents\) and high recomputation cost make it more valuable to retain\. Beyond structural considerations, traditional policies also do not exploit workload patterns\. When users explore related questions \(e\.g\. analyzing different metrics for the same company, or testing code variations against the same test suite\), certain DAG subgraphs become temporarily more valuable based on query similarity\. A policy aware of these semantic patterns could prioritize retaining nodes likely to be reused in future queries\.

This paper introduces a workload aware, structure informed cache eviction policy that is specifically designed for multi agent systems\. Our approach considers three dimensions that most traditional policies ignore\. \(1\) Topological importance of nodes within DAG structures, \(2\) computational cost of recomputing evicted nodes, and \(3\) likelihood of future reuse based on observed workload patterns\. By combining these signals into a unified scoring function, our policy reduces latency by up to 64\.7% relative to the uncached baseline, while approaching the performance of an unbounded cache across diverse multi\-agent workloads\. We also analyze how our approach interacts with complementary multi\-agent efficiency techniques, finding that workload\-aware content caching, plan\-level caching, and parallel agent execution are mutually reinforcing\. Overall, our contributions are:

- C1Workload\-Aware Eviction Policy\.A cache eviction policy for multi\-agent systems that combines recomputation cost, DAG dependency count, and agent invocation frequency into a unified scoring function, enabling principled eviction decisions\.
- C2Empirical Evaluation\.A comprehensive evaluation across three multi\-agent benchmarks spanning different reuse regimes, demonstrating consistent improvements in latency, throughput, and accuracy relative to standard eviction baselines\.
- C3Eviction Quality Analysis\.An analysis showing that eviction quality, not just hit rate, is also a primary driver of latency gains in heterogeneous agentic workloads, with our policy consistently evicting cheaper entries than all competing methods\.
- C4Agentic Optimization Study\.An investigation of how workload\-aware caching interacts with complementary techniques including plan\-level caching and parallel agent execution, revealing synergies\.

## IIBackground and Related Work

We organize related work into three areas\. Caching techniques for LLM based agents, workflow abstractions using DAGs, and efficient multi\-agent system orchestration\.

### II\-ACaching in LLM Agents

As LLM\-based agents have grown in complexity and deployment scale, caching has emerged as a major optimization technique at multiple different levels of abstraction\. We organize approaches by the granularity at which they cache computational artifacts\.

##### KV Cache

KV\-caching refers to caching key\-value tensors from transformer attention computations\[[29](https://arxiv.org/html/2607.20495#bib.bib41)\]\. Modern serving systems like vLLM\[[12](https://arxiv.org/html/2607.20495#bib.bib5)\]and SGLang\[[40](https://arxiv.org/html/2607.20495#bib.bib6)\]introduced paged storage and radix\-tree organization for efficient KV\-cache management, usually employing LRU\-based eviction when GPU memory becomes constrained\. Recent work extends this specifically for multi\-agent systems\. KVFlow\[[20](https://arxiv.org/html/2607.20495#bib.bib4)\]introduces workflow\-aware eviction using an Agent Step Graph to predict execution order, assigning each agent a steps\-to\-execution value estimating temporal proximity to future activation and evicting caches of agents unlikely to run soon\. KVCOMM\[[34](https://arxiv.org/html/2607.20495#bib.bib7)\]addresses a complementary challenge, enabling KV cache sharing across agents with different prefix contexts by maintaining an anchor pool storing observed KV\-cache offset deviations under varying prefixes, enabling cache reuse\.

##### Semantic and Tool\-Level Caching

Higher level systems cache LLM outputs based on query similarity rather than exact matches\. GPTCache\[[1](https://arxiv.org/html/2607.20495#bib.bib8)\]pioneered embedding\-based semantic caching for LLM applications, using similarity search\[[10](https://arxiv.org/html/2607.20495#bib.bib40)\]to reuse responses for semantically related queries\. Building on this foundation, Cortex\[[23](https://arxiv.org/html/2607.20495#bib.bib9)\]extends semantic caching to tool outputs in agent workloads, addressing cross\-region data access latency through semantic\-aware knowledge caching\. These methods, and similar ones, target individual LLM or tool invocations, reducing redundant computation when similar queries recur\.

##### Plan and Reasoning\-Level Caching

The highest level targets planning and reasoning stages of agent execution\. Agentic Plan Caching\[[39](https://arxiv.org/html/2607.20495#bib.bib10)\]extracts structured plan templates from completed executions, storing generalized workflows that can be adapted to new queries with similar task intents through keyword\-based matching and lightweight template adaptation\. Approaches such as those seen in SemanticALLI\[[4](https://arxiv.org/html/2607.20495#bib.bib11)\]complement this by caching intermediate reasoning representations at multiple checkpoints within analytics pipelines, treating structured reasoning states as first\-class cacheable artifacts\. These approaches operate at fundamentally different granularities\. KV cache methods optimize inference latency, semantic caching targets individual calls, and plan level systems reduce planning overhead\. Our work introduces caching at the level of task execution results\.

### II\-BWorkflows Represented as DAGs

Directed acyclic graphs naturally express computational workflows with explicit dependencies, enabling dependency\-aware optimizations across multiple domains\.

##### Data Analytics Systems\.

DAG\-aware caching emerged in analytics frameworks where batch jobs decompose into stages with data dependencies\. LRC\[[35](https://arxiv.org/html/2607.20495#bib.bib12)\]introduced reference\-count\-based eviction for Apache Spark\[[37](https://arxiv.org/html/2607.20495#bib.bib39)\], prioritizing cached data blocks with the highest number of uncomputed children \(an indicator of future utility that outperforms recency\) or frequency\-based policies\. LERC\[[36](https://arxiv.org/html/2607.20495#bib.bib14)\]extended this with coordinated eviction respecting the all\-or\-nothing property, saying that a task only benefits from caching if all input dependencies remain in memory\. Similar approaches optimize materialized views and distributed query execution\[[13](https://arxiv.org/html/2607.20495#bib.bib13),[24](https://arxiv.org/html/2607.20495#bib.bib15)\]\. These systems target batch workloads with static, data\-centric DAGs\.

##### Multi\-Agent Systems with DAG Planning\.

Agent systems adopt DAGs to structure task execution, where nodes represent agent actions \(LLM calls, tool invocations, computations\) and edges encode execution dependencies\. Recent work demonstrates how agents coordinate through DAG based task graphs for subject specific reasoning and decentralized architectures\[[5](https://arxiv.org/html/2607.20495#bib.bib17),[32](https://arxiv.org/html/2607.20495#bib.bib16)\], while benchmarks characterize common DAG patterns in multi agent workflows\[[21](https://arxiv.org/html/2607.20495#bib.bib18)\]\.

### II\-CMulti\-Agent System Orchestration and Optimization

Beyond just caching, there is also research that explores complementary approaches to improving multi\-agent efficiency through orchestration, scheduling, and workflow design\.

##### Scheduling and Orchestration\.

There are several systems that optimize agent execution through workflow aware resource management\. Kairos\[[2](https://arxiv.org/html/2607.20495#bib.bib19)\]introduces priority scheduling that accounts for agent execution characteristics and memory constraints\. Ayo\[[25](https://arxiv.org/html/2607.20495#bib.bib20)\]is another example that proposes fine grained dataflow orchestration, which can enable cross module optimizations\. These approaches improve efficiency by optimizing execution order and resource allocation and often operate orthogonally to caching\.

##### Workflow Generation and Automation\.

Recent work has also explored automated workflow construction\. AFlow\[[38](https://arxiv.org/html/2607.20495#bib.bib21)\]and ADAS\[[8](https://arxiv.org/html/2607.20495#bib.bib22)\]learn effective collaboration patterns and agents by searching over and learning about workflow architectures\. DynaSaur\[[18](https://arxiv.org/html/2607.20495#bib.bib24)\]enables dynamic action composition, allowing agents to adaptively modify workflows based on intermediate results\. These orchestration and optimization techniques address concerns orthogonal to caching\. Scheduling focuses on execution ordering, workflow generation improves task decomposition, while caching eliminates redundant computation\. The techniques can be combined\. Better scheduling may improve cache locality, while effective caching reduces computational load\.

## IIIDesign

Multi\-agent workflows decompose complex tasks into sequences of specialized operations, such as retrieval, reasoning, synthesis, and more\. These operations can form directed acyclic graphs \(DAGs\) where each node represents an agent execution and edges encode dependencies between tasks\. Caching expensive operations from this DAG such as LLM reasoning, visual encoding, and OCR can dramatically reduce latency by avoiding redundant computation across queries\. However, existing eviction policies miss three signals unique to multi\-agent workloads; DAG topology, which reveals which cached results feed downstream tasks, execution costs which can vary by orders of magnitude across agent types, and agent invocation frequency which can also reflect how likely a cached result is to be reused\. We present a workload\-aware eviction policy that combines these three signals into a unified scoring function to maximize cache utility for multi agent systems under memory constraints\.

### III\-AProblem Formulation

Consider a workflow executor processing queries with a multi agent system by decomposing each into a task DAG\. Each task executes using a specialized agent and produces a result consumed by dependent tasks downstream\.

##### Task DAG

For queryqq, the workflow is a DAG,G=\(V,E\)G=\(V,E\)whereV=\{t1,…,tn\}V=\\\{t\_\{1\},\\ldots,t\_\{n\}\\\}are tasks and\(ti,tj\)∈E\(t\_\{i\},t\_\{j\}\)\\in Emeanstjt\_\{j\}depends ontit\_\{i\}’s result\. Each task has an associated agent typea​\(ti\)a\(t\_\{i\}\), such as retrieval, OCR, or synthesis\.

##### Cache Model

Task execution results are stored as keyed mappings under a fixed capacityCC\. Cache entries are keyed by a tuple of the task description and subject identifier \(e\.g\., the source document or video\) explicitly outputted by the planning agent\. When capacity is exceeded, the eviction policy selects a victim entry to remove\.

##### Execution Costs\.

Each tasktit\_\{i\}incurs recomputation timeτ​\(ti\)\\tau\(t\_\{i\}\)on a cache miss, varying by agent type and operation\. A cache hit savesτ​\(ti\)\\tau\(t\_\{i\}\), and a miss pays it\.

### III\-BWorkload Aware Cache Management

LRU and LFU rely exclusively on access history \(recency or frequency\), blind to future reuse and cost heterogeneity\. Our approach combines complementary signals\.

#### III\-B1Dependency Count

DAG topology allows for forward\-looking reuse estimation\. For a cached tasktit\_\{i\}, we trackD​\(ti\)D\(t\_\{i\}\), the set of downstream tasks that consume its result

D​\(ti\)=\{tj:\(ti,tj\)∈E\}D\(t\_\{i\}\)=\\\{t\_\{j\}:\(t\_\{i\},t\_\{j\}\)\\in E\\\}
The dependent count\|D​\(ti\)\|\|D\(t\_\{i\}\)\|serves as a structural proxy for generality and future value\. An entry with zero dependents feeds nothing downstream and is generally safe to evict\. Entries with many dependents, by contrast, are consumed by a larger number of distinct child tasks, which shows that their result may be general enough to be useful across many different similar computational contexts\. Nodes that accumulate substantial upstream context through prior steps \(previous steps acting as something of an informational filter\) tend to produce results that are less generalizable, as their output is increasingly specific to the particular query path taken\. Frame extraction in a video analysis workload is a natural example, where a single extraction result feeds different downstream analysis agents independently\. By contrast, a final answering node at the bottom of the same DAG has zero dependents and is maximally query specific\. While this relationship is not absolute, dependent count provides an intuitive and lightweight structural signal for how broadly reusable a cached entry is likely to be, complementing the recomputation cost and frequency signals in the combined scoring function\.

#### III\-B2Recomputation Cost

Not all cache misses carry equal penalties\. The cost of a miss is determined entirely by how long the evicted entry takes to recompute, and in heterogeneous agentic workloads this variation can be substantial\. Evicting an agentic operation taking on the order of 8 to 12 seconds, is far more damaging than evicting a lightweight answering step that completes in under a second\. Yet traditional policies like LRU treat these entries identically, making eviction decisions purely on access patterns regardless of the computational investment each entry represents\. We record the observed execution timeτ​\(ti\)\\tau\(t\_\{i\}\)for each cached entry and use it directly as a scoring signal, so that the policy is explicitly aware of what it would cost to recompute each entry on a miss\. This directly addresses a shortcoming of cost\-blind policies where an entry accessed slightly less recently but requiring much more computation to recompute should not be evicted simply because another entry was touched more recently\.

#### III\-B3Agent Invocation Frequency

Beyond the structure of individual DAGs, the distribution of agent invocations across the query stream carries its own signal about future reuse\. An agent type that has been invoked frequently up to the current point in the workload is likely to remain active\. If agent A has been called 120 times while an agent B has been called only 20 times, the workload is clearly oriented toward agent A’s content, and entries produced by frame extraction are correspondingly more likely to be needed again\. We track the cumulative invocation countf​\(a\)f\(a\)for each agent typeaaacross all queries processed so far and use it as a proxy for expected future reuse\. Entries from high\-frequency agents receive higher keep scores, while entries from rarely invoked agents are deprioritized\. This allows the policy to naturally be more inclined towards keeping agent types that are central to the current workload focus, such as a database query agent in a financial analysis workload where many queries retrieve information from the same set of reports, without requiring any explicit workload specification from the user\.

#### III\-B4Combined Score

The three signals combine into a single keep score:

score​\(ti\)=wdep⋅D​\(ti\)\+wcost⋅τ​\(ti\)\+wfreq⋅f​\(a​\(ti\)\)\\text\{score\}\(t\_\{i\}\)=w\_\{\\text\{dep\}\}\\cdot D\(t\_\{i\}\)\+w\_\{\\text\{cost\}\}\\cdot\\tau\(t\_\{i\}\)\+w\_\{\\text\{freq\}\}\\cdot f\(a\(t\_\{i\}\)\)wherewdepw\_\{\\text\{dep\}\},wcostw\_\{\\text\{cost\}\}, andwfreqw\_\{\\text\{freq\}\}are non\-negative weights\. The eviction policy removes the entry minimizing this score, which is the entry with fewest dependents, lowest recomputation cost, and least\-invoked agent type\. Section[IV](https://arxiv.org/html/2607.20495#S4)examines weight sensitivity with ablation experiments\.

![Refer to caption](https://arxiv.org/html/2607.20495v1/figures/dageviction.png)Figure 2:Workload aware eviction decision\. When cache capacity is exceeded, our policy takes into account workload characteristics such as dependency count, execution cost, and agent frequency when making decisions\. Node D is correctly identified as lower priority despite being recently accessed, avoiding premature eviction of higher value nodes such as Node C\.

### III\-CEviction Policy

We maintain cached entries in a min\-heap ordered by keep score\. Algorithm[1](https://arxiv.org/html/2607.20495#alg1)describes the procedure\. When a task result is cached, we compute its score based on current workload state with the signals of dependency count, execution cost, and agent frequency, and insert it into the heap\. When capacity is exceeded, eviction is done inO​\(log⁡n\)O\(\\log n\)time\.

Algorithm 1Workload\-Aware Cache Eviction1:Cache capacity

CC, min\-heap

HH, weights

wdep,wcost,wfreqw\_\{\\text\{dep\}\},w\_\{\\text\{cost\}\},w\_\{\\text\{freq\}\}
2:

3:functionCacheInsert\(task

tit\_\{i\}, result, DAG

GG\)

4:

score←wdep⋅D​\(ti\)\+wcost⋅τ​\(ti\)\+wfreq⋅f​\(a​\(ti\)\)\\text\{score\}\\leftarrow w\_\{\\text\{dep\}\}\\cdot D\(t\_\{i\}\)\+w\_\{\\text\{cost\}\}\\cdot\\tau\(t\_\{i\}\)\+w\_\{\\text\{freq\}\}\\cdot f\(a\(t\_\{i\}\)\)
5:

H\.insert​\(ti,score\)H\.\\text\{insert\}\(t\_\{i\},\\text\{score\}\)⊳\\trianglerightO​\(log⁡n\)O\(\\log n\)

6:whilecache size

\>C\>Cdo

7:

victim←H\.extract\-min​\(\)\\text\{victim\}\\leftarrow H\.\\text\{extract\-min\}\(\)⊳\\trianglerightO​\(log⁡n\)O\(\\log n\)

8:Evict victim

9:endwhile

10:endfunction

## IVEvaluation

We evaluate our workload aware eviction policy against standard cache replacement baselines across a diverse set of multi agent workloads\. Our key findings are:

- •Comparison to Unbounded CachingOur workload aware policy consistently approaches the performance of the unbounded cache across benchmarks, achieving latency within a small margin of the theoretical upper bound attainable under unbounded memory \(§[IV\-B](https://arxiv.org/html/2607.20495#S4.SS2)\)\.
- •Superior efficiency over all other finite\-capacity baselines\.Workload aware caching achieves on average a 31\.1% reduction in latency over the next best finite capacity baseline, and reduces latency by up to 64\.7% relative to the uncached baseline across benchmarks \(§[IV\-B](https://arxiv.org/html/2607.20495#S4.SS2)\)\.
- •Substantial throughput gains\.Our policy significantly increases system throughput across all benchmarks, achieving up to a 2\.84×\\timesimprovement over the uncached baseline \(§[IV\-B](https://arxiv.org/html/2607.20495#S4.SS2)\)\.
- •Accuracy\.Workload aware caching retains on average 94\.7% of the uncached baseline’s accuracy across benchmarks, while matching or surpassing the accuracy of all competing finite\-capacity methods\. \(§[IV\-B](https://arxiv.org/html/2607.20495#S4.SS2)\)\.

### IV\-AExperimental Setup

#### IV\-A1Benchmarks

We evaluate on three benchmarks including document, presentation, and video understanding tasks\.

##### SlideVQA

A presentation question answering benchmark\[[26](https://arxiv.org/html/2607.20495#bib.bib32)\]where queries target slides within presentation decks\. Workloads include 1\-13 queries per deck\. This representsmoderate reuse, where we have multiple queries with access to the same deck\. This means creating opportunities for caching different results from the DAGs\. However, the final answers remain query specific, even if evidence from previous steps are similar\.

##### MP\-DocVQA

A multi page document QA benchmark\[[28](https://arxiv.org/html/2607.20495#bib.bib34)\]where queries are about information in PDF documents with evidence found throughout pages\. Each document contains 1\-20 pages, with 1\-85 structurally similar queries per document\. This representshighly variable reuse, where queries have extensive reuse of operations as users explore documents in depth across many different queries in some cases\. There are many documents for which there are little to no reusable operations, and also many for which there is an extensive amount of reuse\.

##### VideoMME

A video understanding benchmark\[[7](https://arxiv.org/html/2607.20495#bib.bib33)\]where queries are about content in video files\. Each video receives 3 queries about content \(e\.g temporal, visual\)\. This representslow reuse, where limited reuse on similar videos results in higher cache churn\. Sub\-tasks are highly, highly variable in terms of agent costs\.

#### IV\-A2Multi\-Agent Workflows

All benchmarks execute using a multi agent system, implemented as a Plan\-Act architecture\[[6](https://arxiv.org/html/2607.20495#bib.bib36)\]\. Given a query, a planning agent \(using Qwen2\.5:14b as a model backbone\) decomposes it into a DAG of tasks needed to answer it, each assigned to a specialized agent\. For example, a frame extraction agent, an OCR agent, a visual analysis agent, etc\. These agents are quite heterogeneous in terms of their execution times and complexity\. Specialized agents utilize the Qwen2\.5:7b model\. The final answers to queries are synthesized and reasoned about with the Qwen2\.5:14b model\.

#### IV\-A3Baseline Eviction Policies

We compare against standard cache replacement policies, including:

- •No Cache:Execute all tasks without caching \(upper bound on execution time\)
- •LRU \(Least Recently Used\):Evicts the entry accessed least recently
- •LFU \(Least Frequently Used\):Evicts the entry accessed least frequently
- •FIFO \(First\-In\-First\-Out\):Evicts the oldest entry
- •ARC \(Adaptive Replacement Cache\[[16](https://arxiv.org/html/2607.20495#bib.bib31)\]\):Dynamically balances recency and frequency using two lists
- •Workload Aware \(Ours\):Utilizes workload characteristics for eviction decisions as described in Section[III](https://arxiv.org/html/2607.20495#S3)

##### Agentic Optimization Baselines

Beyond standard eviction policies, we also compare against complementary agentic optimization techniques in Section[IV\-C](https://arxiv.org/html/2607.20495#S4.SS3), including caching via GPTCache\[[1](https://arxiv.org/html/2607.20495#bib.bib8)\], plan\-level caching via Agentic Plan Caching\[[39](https://arxiv.org/html/2607.20495#bib.bib10)\], and parallel agent execution\[[11](https://arxiv.org/html/2607.20495#bib.bib25)\]\. These methods target different bottlenecks in the agentic pipeline\.

#### IV\-A4Hardware and Experimental Parameters

We ran experiments in an HPC environment with a single A100 GPU, using the Qwen2\.5:14b and Qwen2\.5:7b models\[[27](https://arxiv.org/html/2607.20495#bib.bib27)\]for our agents through use of Ollama\[[19](https://arxiv.org/html/2607.20495#bib.bib26)\]\. Scoring weights were set towdep=1w\_\{\\text\{dep\}\}=1,wcost=2w\_\{\\text\{cost\}\}=2, andwfreq=0\.01w\_\{\\text\{freq\}\}=0\.01, held constant across all benchmarks for fairness\. Execution cost is weighted most heavily to account for the high heterogeneity in agent execution times, while the smaller frequency weight allows cumulative invocation counts to increase the signal’s contribution as agents are used more across queries\. The optimal weights may differ depending on observed workload characteristics, and adaptive weight selection is an interesting direction for future work\. Cache capacity is set to 50% of a working set round for each benchmark, where a round represents the minimum query volume before the cache begins seeing repeated source/subject material\. This maps to 5,000MB for VideoMME and 500MB for SlideVQA and MP\-DocVQA\. To validate this choice, we sweep across capacity levels on VideoMME, comparing our policy against LRU\. As shown in Figure[3](https://arxiv.org/html/2607.20495#S4.F3), the largest differences between policies occurs at moderate capacity, where eviction decisions have meaningful impact\. At lower capacities, the eviction pressure is too high for policies to get useful hits\. At higher capacities, policies converge as eviction becomes rare/the eviction pressure is much too low\. The 50% capacity choice is a good midpoint which allows for a reasonable amount of eviction pressure where policy choices actually matter, and we can see the differences between them more clearly\.

![Refer to caption](https://arxiv.org/html/2607.20495v1/figures/capacity_sweep_hitrate.png)

![Refer to caption](https://arxiv.org/html/2607.20495v1/figures/capacity_sweep_latency.png)

Figure 3:Cache hit rate \(top\) and latency \(bottom\) across capacity levels on VideoMME, comparing Workload\-Aware and LRU eviction policies\. The largest performance gap is seen at a midpoint capacity \( 50% of working set\), where eviction decisions have the most impact\. Policies converge as capacity approaches the full working set\.

### IV\-BCache Management

#### IV\-B1Main Results

TABLE I:Performance comparison across datasets\. Note that MP\-DocVQA and SlideVQA evaluate plain English answers using the Average Normalized Levenshtein Similarity \(ANLS\) metric, whereas VideoMME uses accuracy percentages for multiple\-choice questions\.Table[I](https://arxiv.org/html/2607.20495#S4.T1)presents our main results across 3 benchmarks\. Our workload aware policy consistently achieves the highest hit rates and lowest latency while maintaining accuracy in the realm of the other caching eviction baselines\. Under a fixed memory budget, it often approaches the performance of an infinite memory/unbounded cache\. Other methods often come with both a lower hit rate and lower accuracy, with the exception of the SlideVQA dataset, where the differences in hit rate and latency are most stark in favor of workload aware caching\. The scatter plots in Figure[4](https://arxiv.org/html/2607.20495#S4.F4)further demonstrate the benefits of Workload Aware caching\.

![Refer to caption](https://arxiv.org/html/2607.20495v1/figures/scatter.png)Figure 4:Latency vs Hit Rate\. Demonstrates that Workload Aware caching consistently outperforms other baselines in both metrics, and often approaches the performance of an unbounded cache\.On workloads with moderate to high reuse opportunity, we see the biggest differences in terms of hit rates and latency, as the cache has more flexibility to achieve hits\. On SlideVQA and MP\-DocVQA, we achieve hit rates of 45\.1% and 40\.6% respectively, reducing per query latency by 64\.7% and 56\.6% relative to the uncached baseline, and improving throughput by up to 2\.84×\\times\. It is also important to note that these gains are at a minimal distance from the theoretical upper bound of a cache with infinite storage capacity\. Our policy achieves latency within 3\.4% and 1\.8% of an infinite cache on SlideVQA and MP\-DocVQA respectively, despite having a fixed memory budget\. The next best finite capacity approach, LFU, is behind our approach by 45\.8% and 32\.2% on the same benchmarks\. ARC, LFU, and FIFO perform progressively worse\. For instance, LRU and FIFO on MP\-DocVQA achieve hit rates of just 21\.8% and 19\.2% respectively, with latency even approaching that of the uncached baseline\.

On datasets with lower reuse, such as VideoMME, Workload Aware caching continues to outperform other finite capacity baselines\. With only approximately three queries per video, the cache has limited opportunity to accumulate reusable results before transitioning to a different video with a distinct profile for reusing agent steps\. This can be seen with the universally lower hit rates across all policies\. Our workload aware policy achieves the highest hit rate at 17\.9%, as well as the lowest latency of 27\.38 seconds per query amongst finite capacity methods\. This represents a 31\.6% latency reduction when compared to the uncached baseline\.

In summary, across all benchmarks, our workload aware policy maintains or surpasses the accuracy of all competing finite capacity methods, and in several cases closely approaches the uncached baseline among all caching methods\.

#### IV\-B2Eviction Cost Analysis

![Refer to caption](https://arxiv.org/html/2607.20495v1/figures/evicted_cost.png)Figure 5:Average Evicted Cost of Entries for Each MethodTABLE II:Execution time saved by cache hits as a percentage of the time saved by Workload\-Aware caching, which saves the most cost out of all methods\. Higher is better\.Cache hit rate, while an important metric, does not always tell the full story of performance\[[22](https://arxiv.org/html/2607.20495#bib.bib23)\]\. Figure[5](https://arxiv.org/html/2607.20495#S4.F5)and Table[II](https://arxiv.org/html/2607.20495#S4.T2)demonstrate one reason for our method’s increase in serving performance\. Workload aware caching has the capability to identify costly entries in the cache and preserve them, which means the evicted items are dramatically less costly than what the other policies evict on all three benchmarks\. Cost blind policies such as LRU, FIFO, LFU, and ARC evict purely based on access history, making them just as likely to evict an expensive frame extraction or OCR result as a cheap lightweight lookup operation\. With recomputation cost added as a scoring signal as in our policy, we are able to ensure expensive items remain in the cache\.

The practical consequences of this are also very significant\. When a cache miss does occur with our policy, it tends to be for a cheaper entry, while misses on other policies have a higher probability of being for an expensive node/agentic step\. This asymmetry compounds over time, and as shown in Table[I](https://arxiv.org/html/2607.20495#S4.T1), our policy ends up with a lower query latency amongst all benchmarks compared to the uncached baseline and all finite capacity methods, despite the absolute difference in hit rate not always being large\. This indicates that the quality of what is kept in the cache matters just as much as how often the cache is hit, especially with heterogeneous workflows such as agentic ones\.

#### IV\-B3Cross\-Model Results

![Refer to caption](https://arxiv.org/html/2607.20495v1/figures/latency_reduction_vs_model.png)Figure 6:Latency reduction relative to the uncached baseline across all methods and benchmarks, comparing Qwen\-2\.5 and Mistral\-small3\.2 model families\. Workload\-Aware caching consistently achieves the highest latency reduction among finite\-capacity methods across both model families, demonstrating that the policy’s advantage is not specific to a particular model\. Absolute latency reductions differ between model families due to differences in per\-query inference speed and planning behavior, but the relative ordering of policies is preserved\.We additionally evaluate our policy’s performance by replicating the main experiments using Mistral\-small3\.2:24b\[[17](https://arxiv.org/html/2607.20495#bib.bib28)\]as the agent backbone across all three benchmarks\. Figure[6](https://arxiv.org/html/2607.20495#S4.F6)shows latency reduction relative to the uncached baseline for both model families\. Workload\-Aware caching consistently achieves the highest latency reduction among finite\-capacity methods across all three benchmarks and both model families, with the relative ordering of policies preserved in all cases\. Absolute latency reductions differ between model families due to differences in per\-query inference speed and planning behavior, but the advantage of Workload\-Aware caching over competing baselines is robust across both settings\.

### IV\-CAgentic Optimization

TABLE III:Performance comparison of agentic baselines, highlighting latency and accuracy trade\-offs\. Accuracy is reported in ANLS for SlideVQA and standard percentage for VideoMME\.We evaluate how our workload\-aware caching interacts with two complementary multi\-agent efficiency techniques, each targeting different bottlenecks in multi\-agent systems\. First, parallel execution of agents similar to LLMCompiler\[[11](https://arxiv.org/html/2607.20495#bib.bib25)\], executing independent DAG nodes concurrently instead of sequentially to reduce latency\. For plan\-level caching, we compare with Agentic Plan Caching \(APC\)\[[39](https://arxiv.org/html/2607.20495#bib.bib10)\], which reuses structured plan templates across tasks to reduce planning overhead\. Our technique, in comparison, targets the bottleneck of redundant node level recomputation across queries\. We evaluate each technique in isolation and in combination with our approach\. For APC, we report both cold start results, where the cache starts empty, and warm results\. Results are shown in Table[III](https://arxiv.org/html/2607.20495#S4.T3)\. Results are reported for benchmarks with moderate reuse \(SlideVQA\) and low reuse \(VideoMME\) to demonstrate the performance of each method in different settings\.

Our workload aware caching achieves the best latency as a stand alone method, outperforming other individual techniques\. Alone, parallel execution provides notable latency benefits depending on the workload\. For example, on VideoMME, where the opportunities for overlap are high \(e\.g\. captioning frames and extracting audio are independent of each other, yet both computationally expensive\) parallelism has an obvious impact relative to the uncached baseline \(see Table[I](https://arxiv.org/html/2607.20495#S4.T1)\)\. In comparison, on SlideVQA, the benefits of parallelism can be seen but are smaller relative to the uncached baseline, as the dependencies between steps differ, and the latency attributed to them may be more heterogeneous \(thereby reducing the impact of parallelism\)\.

APC alone shows limited impact under cold start conditions, where the overhead of cache generation offsets much of the potential planning savings in our workloads\. With a lower number of repeated/heavily related queries, the benefits of caching plans are not seen immediately, resulting in cold start latency that is equal to or slightly worse than the uncached baseline\. As the template cache warms up, APC begins to demonstrate more meaningful improvement\. Warm APC results show reduced latency relative to cold on both benchmarks, confirming plan level reuse is a useful signal to use alongside others, especially with a populated cache\.

GPTCache style semantic caching shows a double edged tradeoff, leading to lower latency on both benchmarks, but at a consistent accuracy cost across both benchmarks as well\. This occurs because approximate matches return cached intermediate results that are incorrect for the current query, propagating errors through downstream agent steps and correspondingly lowering the accuracy\. This reflects a characteristic of agentic settings, as unlike end\-to\-end query caching where a semantically similar response may be acceptable, intermediate DAG node results must be precise since they feed into dependent computations\. Exact match caching avoids this tradeoff, prioritizing correctness over the raw hit rate\.

The combination of APC, parallel execution, and workload\-aware content caching was also evaluated, and shows clear synergy\. Each technique targets a distinct stage of the agentic pipeline, and together they can collectively reduce latency across planning, execution, and node\-level recomputation\. On VideoMME, APC \+ Parallel \+ Workload\-Aware achieves competitive latency with our standalone approach in cold conditions, and warm results show further improvement as plan templates accumulate\. These results suggest that the three techniques are genuinely complementary, with each contributing independently to overall efficiency\. On SlideVQA, however, the combined approach shows higher latency than our standalone workload\-aware caching\. This reflects a practical deployment consideration in resource\-constrained environments\. In our case, running parallel agent execution \(so potentially multiple GPU\-bound tools concurrently\) alongside language model inference and cached items \(e\.g\. tensors\) on a single GPU introduces resource contention that can degrade throughput\.

Overall, these results highlight that workload\-aware content caching, plan\-level caching, and parallel execution are complementary techniques that each address a distinct efficiency bottleneck in multi\-agent systems\. Our approach performs strongly as a standalone method and provides a solid foundation that other techniques can build upon\.

### IV\-DAblation Study

TABLE IV:Ablation study of the Workload\-Aware scoring function components\. Accuracy is reported in ANLS for SlideVQA and standard percentage for VideoMME\.To evaluate the choice of parameters in our workload aware policy, we ablate the three scoring signals\. We evaluate three variants, leaving one of the scoring signals out each time\. Results are shown in Table[IV](https://arxiv.org/html/2607.20495#S4.T4)for SlideVQA and VideoMME\.

Across both benchmarks, removing any single scoring signal degrades hit rate and latency relative to utilizing all three, confirming that the combination of signals is robustly beneficial across different workloads\.

The relative importance of each signal differs across workloads\. On SlideVQA, removing the execution time term has by far the largest impact, collapsing hit rate and degrading latency to near the uncached baseline\. This dramatic change reflects the important role of execution cost as a protective signal in this workload, indicating that items that were expensive were also more likely to be reused there\. Removing frequency or dependency count produces a more moderate, yet still noticeable degradation in comparison to the full policy\.

On VideoMME, all three ablations degrade performance by comparable margins, with hit rates falling from 17\.9% to between 14\.7 and 15\.6%, and average query latency increasing from 27\.38s to around 31\-34 seconds\. Here, dependency count carried the largest independent impact, with No\-Dep having the lowest hit rate\. The execution time term, while still important, does not dominate to the same degree as it did in SlideVQA, indicating a difference in the costs of agents, which ones are reused most, and the way they are used in the planning DAGs\.

Taken all together, the ablation results validate our choices of parameters\. Each signal addresses a distinct aspect of eviction quality\. Execution cost protects expensive entries, while dependency count and agent frequency reflect workload alignment \(e\.g\. agents used quite frequently or entries that have many dependencies are more likely to be reused in the workload\)\. The full combination consistently outperforms other subsets, and the workload dependent variation in signal importance motivates the use of all three rather than a single one\.

## VLimitations and Future Work

Our current evaluation uses fixed weights for the three scoring signals,wdepw\_\{\\text\{dep\}\},wcostw\_\{\\text\{cost\}\}, andwfreqw\_\{\\text\{freq\}\}, which are held constant across the query stream\. While our ablation study shows that all three signals contribute meaningfully across diverse workloads, the relative importance of each signal varies by workload characteristics\. For instance, a workload with fairly consistent agent\-calling frequencies but heavily variable execution times between steps might work better with a lower frequency signal\. An interesting direction for future work is to learn or adapt these weights dynamically based on similar observed query patterns, allowing the policy to specialize to the workload distribution it encounters in deployment\. A second interesting direction is a broader and tighter integration with complementary efficiency techniques\. Our agentic optimization study shows that workload\-aware caching, plan\-level caching, and parallel execution are complementary and mutually reinforcing\. However, these techniques currently operate independently\. Jointly optimizing across caching, scheduling, and execution ordering could be interesting\. For instance, a scheduling\-aware cache that prioritizes retaining entries for agents scheduled to run soon, similar in spirit to KVFlow’s approach at the KV cache level, could yield compounding efficiency gains beyond what is achievable by combining them independently\. Another interesting future direction is cache\-aware planning, where the planning agent is informed of the current cache state when generating execution plans\. Currently, planning and caching operate as independent stages, where the planner takes queries and generates DAG represented plans without knowledge of what is already cached\. A planner aware of cached entries could decompose queries in ways that explicitly maximize reuse, representing a tighter and potentially more powerful integration of planning and caching in multi\-agent systems\.

## VIConclusion

In conclusion, we presented a workload\-aware cache eviction policy for multi\-agent systems that exploits three signals unique to DAG\-structured agentic workloads, namely recomputation cost, dependency count, and agent invocation frequency\. Unlike traditional eviction policies that treat all cached entries uniformly based on access history, our approach preserves entries that are expensive to recompute, structurally central to the DAG, and aligned with the current workload focus\. Evaluated across three diverse benchmarks spanning different reuse regimes, our policy consistently outperforms all finite\-capacity baselines in latency and throughput while maintaining accuracy, and approaches the performance of an unbounded cache despite operating under a fixed memory budget\.

Our results demonstrate that the quality of eviction decisions matters as much as cache hit rate in heterogeneous agentic workloads\. By selectively retaining expensive and structurally important nodes, our policy converts a moderate hit rate advantage into large latency savings, a property that cost\-blind policies cannot replicate\. Our analysis of complementary techniques further shows that workload\-aware content caching, plan\-level caching, and parallel execution target distinct bottlenecks in multi\-agent pipelines and are very effective in combination, with the right mix depending on workload structure\.

As multi\-agent systems grow in complexity and scale, efficient resource utilization at every level of the execution stack becomes more and more important\. We hope this work motivates further exploration of workload\-aware optimization techniques for agentic systems, and highlights the value of using signals that are uniquely available in DAG\-based multi\-agent execution environments\.

## References

- \[1\]F\. Bang\(2023\-12\)GPTCache: an open\-source semantic cache for LLM applications enabling faster answers and cost savings\.InProceedings of the 3rd Workshop for Natural Language Processing Open Source Software \(NLP\-OSS 2023\),L\. Tan, D\. Milajevs, G\. Chauhan, J\. Gwinnup, and E\. Rippeth \(Eds\.\),Singapore,pp\. 212–218\.External Links:[Link](https://aclanthology.org/2023.nlposs-1.24/),[Document](https://dx.doi.org/10.18653/v1/2023.nlposs-1.24)Cited by:[§II\-A](https://arxiv.org/html/2607.20495#S2.SS1.SSS0.Px2.p1.1),[§IV\-A3](https://arxiv.org/html/2607.20495#S4.SS1.SSS3.Px1.p1.1)\.
- \[2\]J\. Chen, J\. Shi, Q\. Chen, and M\. Guo\(2025\)Kairos: low\-latency multi\-agent serving with shared llms and excessive loads in the public cloud\.External Links:2508\.06948,[Link](https://arxiv.org/abs/2508.06948)Cited by:[§II\-C](https://arxiv.org/html/2607.20495#S2.SS3.SSS0.Px1.p1.1)\.
- \[3\]Y\. Chen, M\. Shetty, G\. Somashekar, M\. Ma, Y\. Simmhan, J\. Mace, C\. Bansal, R\. Wang, and S\. Rajmohan\(2025\-05\)AIOpsLab: a holistic framework for evaluating ai agents for enabling autonomous cloud\.InMLSys ’25,Cited by:[§I](https://arxiv.org/html/2607.20495#S1.p1.1)\.
- \[4\]V\. Chillara, D\. Kline, C\. Alvares, E\. Wooten, H\. Yang, S\. Khetan, C\. Bauer, T\. Guillory, T\. Shah, Y\. Dhariwal, V\. Pavlov, and G\. Popstefanov\(2026\)SemanticALLI: caching reasoning, not just responses, in agentic systems\.External Links:2601\.16286,[Link](https://arxiv.org/abs/2601.16286)Cited by:[§II\-A](https://arxiv.org/html/2607.20495#S2.SS1.SSS0.Px3.p1.1)\.
- \[5\]J\. Dong, Z\. Lin, W\. Lin, and M\. Zhang\(2025\)S\-dag: a subject\-based directed acyclic graph for multi\-agent heterogeneous reasoning\.External Links:2511\.06727,[Link](https://arxiv.org/abs/2511.06727)Cited by:[§II\-B](https://arxiv.org/html/2607.20495#S2.SS2.SSS0.Px2.p1.1)\.
- \[6\]L\. E\. Erdogan, H\. Furuta, S\. Kim, N\. Lee, S\. Moon, G\. Anumanchipalli, K\. Keutzer, and A\. Gholami\(2025\)Plan\-and\-act: improving planning of agents for long\-horizon tasks\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=ybA4EcMmUZ)Cited by:[§IV\-A2](https://arxiv.org/html/2607.20495#S4.SS1.SSS2.p1.1)\.
- \[7\]C\. Fu, Y\. Dai, Y\. Luo, L\. Li, S\. Ren, R\. Zhang, Z\. Wang, C\. Zhou, Y\. Shen, M\. Zhang,et al\.\(2025\)Video\-mme: the first\-ever comprehensive evaluation benchmark of multi\-modal llms in video analysis\.InCVPR,Cited by:[§IV\-A1](https://arxiv.org/html/2607.20495#S4.SS1.SSS1.Px3.p1.1)\.
- \[8\]S\. Hu, C\. Lu, and J\. Clune\(2025\)Automated design of agentic systems\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=t9U3LW7JVX)Cited by:[§II\-C](https://arxiv.org/html/2607.20495#S2.SS3.SSS0.Px2.p1.1)\.
- \[9\]C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. R\. Narasimhan\(2024\)SWE\-bench: can language models resolve real\-world github issues?\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=VTF8yNQM66)Cited by:[§I](https://arxiv.org/html/2607.20495#S1.p1.1)\.
- \[10\]J\. Johnson, M\. Douze, and H\. Jégou\(2019\)Billion\-scale similarity search with GPUs\.IEEE Transactions on Big Data7\(3\),pp\. 535–547\.Cited by:[§II\-A](https://arxiv.org/html/2607.20495#S2.SS1.SSS0.Px2.p1.1)\.
- \[11\]S\. Kim, S\. Moon, R\. Tabrizi, N\. Lee, M\. W\. Mahoney, K\. Keutzer, and A\. Gholami\(2024\)An llm compiler for parallel function calling\.InProceedings of the 41st International Conference on Machine Learning,ICML’24\.Cited by:[§IV\-A3](https://arxiv.org/html/2607.20495#S4.SS1.SSS3.Px1.p1.1),[§IV\-C](https://arxiv.org/html/2607.20495#S4.SS3.p1.1)\.
- \[12\]W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica\(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles,Cited by:[§II\-A](https://arxiv.org/html/2607.20495#S2.SS1.SSS0.Px1.p1.1)\.
- \[13\]Z\. Li, X\. Pi, and Y\. Park\(2023\)S/c: speeding up data materialization with bounded memory\.In2023 IEEE 39th International Conference on Data Engineering \(ICDE\),Vol\.,pp\. 1981–1994\.External Links:[Document](https://dx.doi.org/10.1109/ICDE55515.2023.00393)Cited by:[§II\-B](https://arxiv.org/html/2607.20495#S2.SS2.SSS0.Px1.p1.1)\.
- \[14\]Y\. Liuet al\.\(2025\)Efficient inference for large reasoning models: a survey\.arXiv preprint arXiv:2503\.23077\.Cited by:[§I](https://arxiv.org/html/2607.20495#S1.p1.1)\.
- \[15\]C\. Lu, C\. Lu, R\. T\. Lange, J\. Foerster, J\. Clune, and D\. Ha\(2024\)The AI scientist: towards fully automated open\-ended scientific discovery\.arXiv preprint arXiv:2408\.06292\.Cited by:[§I](https://arxiv.org/html/2607.20495#S1.p1.1)\.
- \[16\]N\. Megiddo and D\. S\. Modha\(2003\-03\)ARC: a Self\-Tuning, low overhead replacement cache\.In2nd USENIX Conference on File and Storage Technologies \(FAST 03\),San Francisco, CA\.External Links:[Link](https://www.usenix.org/conference/fast-03/arc-self-tuning-low-overhead-replacement-cache)Cited by:[5th item](https://arxiv.org/html/2607.20495#S4.I2.i5.p1.1.1)\.
- \[17\]Mistral AI\(2025\)Mistral Small 3\.2\.Note:https://huggingface\.co/mistralai/Mistral\-Small\-3\.2\-24B\-Instruct\-2506Cited by:[§IV\-B3](https://arxiv.org/html/2607.20495#S4.SS2.SSS3.p1.1)\.
- \[18\]D\. Nguyen, V\. D\. Lai, S\. Yoon, R\. A\. Rossi, H\. Zhao, R\. Zhang, P\. Mathur, N\. Lipka, Y\. Wang, T\. Bui, F\. Dernoncourt, and T\. Zhou\(2024\)DynaSaur: large language agents beyond predefined actions\.arXiv preprint arXiv:2411\.01747\.Cited by:[§II\-C](https://arxiv.org/html/2607.20495#S2.SS3.SSS0.Px2.p1.1)\.
- \[19\]OllamaOllama: run AI models locally\.Note:Accessed: February 2025External Links:[Link](https://ollama.com/)Cited by:[§IV\-A4](https://arxiv.org/html/2607.20495#S4.SS1.SSS4.p1.3)\.
- \[20\]Z\. Pan, A\. Patel, Z\. Hu, Y\. Shen, Y\. Guan, W\. Li, L\. Qin, Y\. Wang, and Y\. Ding\(2025\)KVFlow: efficient prefix caching for accelerating llm\-based multi\-agent workflows\.External Links:2507\.07400,[Link](https://arxiv.org/abs/2507.07400)Cited by:[§II\-A](https://arxiv.org/html/2607.20495#S2.SS1.SSS0.Px1.p1.1)\.
- \[21\]S\. Qiao, R\. Fang, Z\. Qiu, X\. Wang, N\. Zhang, Y\. Jiang, P\. Xie, F\. Huang, and H\. Chen\(2025\)Benchmarking agentic workflow generation\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=vunPXOFmoi)Cited by:[§II\-B](https://arxiv.org/html/2607.20495#S2.SS2.SSS0.Px2.p1.1)\.
- \[22\]Z\. Qiu, J\. Yang, and M\. Harchol\-Balter\(2024\)Can increasing the hit ratio hurt cache throughput? \(long version\)\.External Links:2404\.16219,[Link](https://arxiv.org/abs/2404.16219)Cited by:[§IV\-B2](https://arxiv.org/html/2607.20495#S4.SS2.SSS2.p1.1)\.
- \[23\]C\. Ruan, C\. Bi, K\. Zheng, Z\. Shi, X\. Wan, and J\. Li\(2026\)Cortex: achieving low\-latency, cost\-efficient remote data access for llm via semantic\-aware knowledge caching\.External Links:2509\.17360,[Link](https://arxiv.org/abs/2509.17360)Cited by:[§II\-A](https://arxiv.org/html/2607.20495#S2.SS1.SSS0.Px2.p1.1)\.
- \[24\]P\. Sioulas, I\. Mytilinis, and A\. Ailamaki\(2023\)Real\-time analytics by coordinating reuse and work sharing\.External Links:2307\.08018,[Link](https://arxiv.org/abs/2307.08018)Cited by:[§II\-B](https://arxiv.org/html/2607.20495#S2.SS2.SSS0.Px1.p1.1)\.
- \[25\]X\. Tan, Y\. Jiang, Y\. Yang, and H\. Xu\(2025\)Towards end\-to\-end optimization of llm\-based applications with ayo\.InProceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2,External Links:[Link](https://doi.org/10.1145/3676641.3716278),[Document](https://dx.doi.org/10.1145/3676641.3716278)Cited by:[§II\-C](https://arxiv.org/html/2607.20495#S2.SS3.SSS0.Px1.p1.1)\.
- \[26\]R\. Tanaka, K\. Nishida, K\. Nishida, T\. Hasegawa, I\. Saito, and K\. Saito\(2023\)SlideVQA: a dataset for document visual question answering on multiple images\.InAAAI,Cited by:[§IV\-A1](https://arxiv.org/html/2607.20495#S4.SS1.SSS1.Px1.p1.1)\.
- \[27\]Q\. Team\(2024\)Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§IV\-A4](https://arxiv.org/html/2607.20495#S4.SS1.SSS4.p1.3)\.
- \[28\]R\. Tito, D\. Karatzas, and E\. Valveny\(2023\)Hierarchical multimodal transformers for multi\-page docvqa\.External Links:2212\.05935,[Link](https://arxiv.org/abs/2212.05935)Cited by:[§IV\-A1](https://arxiv.org/html/2607.20495#S4.SS1.SSS1.Px2.p1.1)\.
- \[29\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,pp\. 5998–6008\.Cited by:[§II\-A](https://arxiv.org/html/2607.20495#S2.SS1.SSS0.Px1.p1.1)\.
- \[30\]L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin, W\. X\. Zhao, Z\. Wei, and J\. Wen\(2023\)A survey on large language model based autonomous agents\.Frontiers of Computer Science18\.External Links:[Link](https://api.semanticscholar.org/CorpusID:261064713)Cited by:[§I](https://arxiv.org/html/2607.20495#S1.p1.1)\.
- \[31\]Y\. Yamada, R\. T\. Lange, C\. Lu, S\. Hu, C\. Lu, J\. Foerster, J\. Clune, and D\. Ha\(2025\)The AI scientist\-v2: workshop\-level automated scientific discovery via agentic tree search\.arXiv preprint arXiv:2504\.08066\.Cited by:[§I](https://arxiv.org/html/2607.20495#S1.p1.1)\.
- \[32\]Y\. Yang, H\. Chai, S\. Shao, Y\. Song, S\. Qi, R\. Rui, and W\. Zhang\(2025\)AgentNet: decentralized evolutionary coordination for LLM\-based multi\-agent systems\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=tXqLxHlb8Z)Cited by:[§II\-B](https://arxiv.org/html/2607.20495#S2.SS2.SSS0.Px2.p1.1)\.
- \[33\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. Cao\(2023\)ReAct: synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations,Cited by:[§I](https://arxiv.org/html/2607.20495#S1.p1.1)\.
- \[34\]H\. Ye, Z\. Gao, M\. Ma, Q\. Wang, Y\. Fu, M\. Chung, Y\. Lin, Z\. Liu, J\. Zhang, D\. Zhuo, and Y\. Chen\(2025\)KVCOMM: online cross\-context KV\-cache communication for efficient LLM\-based multi\-agent systems\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=yGOytgjurF)Cited by:[§II\-A](https://arxiv.org/html/2607.20495#S2.SS1.SSS0.Px1.p1.1)\.
- \[35\]Y\. Yu, W\. Wang, J\. Zhang, and K\. Ben Letaief\(2017\)LRC: dependency\-aware cache management for data analytics clusters\.InIEEE INFOCOM 2017 \- IEEE Conference on Computer Communications,Vol\.,pp\. 1–9\.External Links:[Document](https://dx.doi.org/10.1109/INFOCOM.2017.8057007)Cited by:[§II\-B](https://arxiv.org/html/2607.20495#S2.SS2.SSS0.Px1.p1.1)\.
- \[36\]Y\. Yu, W\. Wang, J\. Zhang, and K\. B\. Letaief\(2017\)LERC: coordinated cache management for data\-parallel systems\.InGLOBECOM 2017 \- 2017 IEEE Global Communications Conference,Vol\.,pp\. 1–6\.External Links:[Document](https://dx.doi.org/10.1109/GLOCOM.2017.8254999)Cited by:[§II\-B](https://arxiv.org/html/2607.20495#S2.SS2.SSS0.Px1.p1.1)\.
- \[37\]M\. Zaharia, M\. Chowdhury, T\. Das, A\. Dave, J\. Ma, M\. McCauly, M\. J\. Franklin, S\. Shenker, and I\. Stoica\(2012\)Resilient distributed datasets: a \{Fault\-Tolerant\} abstraction for \{In\-Memory\} cluster computing\.In9th USENIX symposium on networked systems design and implementation \(NSDI 12\),pp\. 15–28\.Cited by:[§II\-B](https://arxiv.org/html/2607.20495#S2.SS2.SSS0.Px1.p1.1)\.
- \[38\]J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang, B\. Zheng, B\. Liu, Y\. Luo, and C\. Wu\(2025\)AFlow: automating agentic workflow generation\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=z5uVAKwmjf)Cited by:[§II\-C](https://arxiv.org/html/2607.20495#S2.SS3.SSS0.Px2.p1.1)\.
- \[39\]Q\. Zhang, M\. Wornow, and K\. Olukotun\(2025\)Agentic plan caching: test\-time memory for fast and cost\-efficient LLM agents\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=n4V3MSqK77)Cited by:[§II\-A](https://arxiv.org/html/2607.20495#S2.SS1.SSS0.Px3.p1.1),[§IV\-A3](https://arxiv.org/html/2607.20495#S4.SS1.SSS3.Px1.p1.1),[§IV\-C](https://arxiv.org/html/2607.20495#S4.SS3.p1.1)\.
- \[40\]L\. Zheng, L\. Yin, Z\. Xie, C\. Sun, J\. Huang, C\. H\. Yu, S\. Cao, C\. Kozyrakis, I\. Stoica, J\. E\. Gonzalez, C\. Barrett, and Y\. Sheng\(2024\)SGLang: efficient execution of structured language model programs\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=VqkAKQibpq)Cited by:[§II\-A](https://arxiv.org/html/2607.20495#S2.SS1.SSS0.Px1.p1.1)\.