FinCacheServe: Dependency-Consistent Answer Reuse for Cost-Efficient RAG Serving over Mutable Enterprise Documents
摘要
FinCacheServe is a system for dependency-consistent answer reuse in RAG serving over mutable enterprise documents, using document versions, evidence fingerprints, and tool fingerprints to invalidate caches. Evaluations show it skips over 53% of LLM calls with zero stale outputs, reducing GPU cost compared to versioned semantic caching.
查看缓存全文
缓存时间: 2026/07/31 04:02
# FinCacheServe: Dependency-Consistent Answer Reuse for Cost-Efficient RAG Serving over Mutable Enterprise Documents
Source: [https://arxiv.org/html/2607.26076](https://arxiv.org/html/2607.26076)
\[1\]\\fnmLingteng\\surZeng\\orcidhttps://orcid\.org/0009\-0006\-0362\-630X
\[1\]\\orgdivFaculty of Engineering,\\orgnameThe Chinese University of Hong Kong,\\orgaddress\\cityHong Kong SAR,\\countryChina
2\]\\orgdivSchool of Computer Science and Technology,\\orgnameBeijing Institute of Technology,\\orgaddress\\cityBeijing,\\countryChina
###### Abstract
Retrieval\-augmented generation services over mutable enterprise documents repeatedly execute semantically equivalent analysis requests\. Answer reuse can remove GPU\-bound generation work, yet response caches require dependency consistency when filings, evidence chunks, and tool outputs change\. FinCacheServe treats each generated answer as a serving object indexed by enterprise intent and guarded by document versions, evidence fingerprints, tool fingerprints, model identity, and decoding configuration\. A vLLM implementation evaluates SEC\-derived financial\-document workloads with Qwen2\.5 models\. On a 2,230\-request hosted 7B trace, FinCacheServe skips 53\.27% of LLM calls with zero observed dependency\-stale outputs\. Across three hosted 32B operator\-suite seeds, it skips 53\.31% of 544 requests, compared with 38\.97% for versioned semantic caching and 22\.43% for grounded\-style reuse\. Capacity, backend, and SLO replays show oracle\-bounded cache management, 100k\-entry transactional metadata behavior, and 44\.30% lower estimated Wh per dependency\-fresh 2s\-SLO success than versioned semantic caching\.
###### keywords:
retrieval\-augmented generation, large language model serving, answer caching, mutable enterprise documents, freshness consistency, dependency invalidation, GPU cost
## 1Introduction
Cloud retrieval\-augmented generation \(RAG\) services increasingly serve analytical workloads over corporate reports, filings, policies, tickets, and logs\. A typical request retrieves evidence, builds an augmented prompt, calls a large language model \(LLM\), and returns an answer with cited context\. This execution path spends GPU memory, prefill time, and decoding tokens even when the request repeats a previous analytical intent\. The repeated work becomes significant in dashboards, compliance workflows, financial analysis assistants, and agentic systems that issue recurring questions over slowly changing corpora\.
Caching is the natural performance mechanism for repeated serving work\. LLM\-serving systems have reduced GPU pressure through KV\-cache management, prefix reuse, prefill reuse, and context scheduling\[[1](https://arxiv.org/html/2607.26076#bib.bib1),[2](https://arxiv.org/html/2607.26076#bib.bib2),[3](https://arxiv.org/html/2607.26076#bib.bib3),[4](https://arxiv.org/html/2607.26076#bib.bib4),[5](https://arxiv.org/html/2607.26076#bib.bib5),[6](https://arxiv.org/html/2607.26076#bib.bib6)\]\. Those mechanisms accelerate execution after the request reaches the model\. Answer reuse attacks a different bottleneck\. A valid answer\-cache hit can bypass the model invocation, removing both prefill and decoding for that request\.
Mutable enterprise documents complicate answer reuse\. A semantically similar query can require a fresh answer when a filing amendment changes a source table, when an extraction pipeline refreshes evidence chunks, when a ratio computation changes, or when serving moves to a different model/configuration\. A cache hit based on text similarity or a fixed time\-to\-live window can therefore expose stale evidence\. Financial filings make this dependency problem concrete and reproducible: each answer is tied to a company, period, filing scope, evidence set, tool output, and filing version\. The workload contains repeated analytical intent, yet the validity of reuse depends on a fine\-grained dependency contract\.
FinCacheServe is a serving\-layer system for dependency\-consistent answer reuse\. The system caches generated answers as materialized serving objects\. Each entry records normalized financial identity, evidence fingerprints, tool\-output fingerprints, source\-document versions, model identity, and decoding configuration\. The reuse gate admits an answer when the incoming request matches the recorded task identity and every recorded dependency remains compatible with the current metadata plane\. Filing updates propagate through a document\-to\-answer reverse dependency index before later cache reads observe the updated version store\.
This design targets a supercomputing\-relevant performance point: reducing GPU\-bound LLM invocations in hosted RAG serving while keeping a precise consistency contract for mutable evidence\. The implementation integrates an answer cache, a retrieval cache, a FAISS\-backed retriever\[[7](https://arxiv.org/html/2607.26076#bib.bib7)\], and vLLM\-hosted Qwen2\.5 models\[[1](https://arxiv.org/html/2607.26076#bib.bib1),[8](https://arxiv.org/html/2607.26076#bib.bib8)\]\. The evaluation uses SEC\-derived workloads as a public high\-value enterprise\-document benchmark, with hosted 7B/14B/32B runs, strong answer\-cache baselines, request\-level provider\-call accounting, capacity\-limited cache\-management replay, online policy sensitivity, metadata\-backend validation, energy sensitivity, and interleaved query/update stress\.
The paper makes four contributions\.
1. 1\.Dependency\-consistent answer reuse\.FinCacheServe defines an answer\-cache contract that binds reuse to financial intent, source\-document versions, evidence fingerprints, tool\-output fingerprints, model identity, and decoding configuration\.
2. 2\.GPU\-serving implementation\.The implementation integrates answer\-level reuse with a vLLM\-backed RAG serving path and records LLM invocations, skipped calls, token estimates, freshness events, and stale\-origin categories at request granularity\.
3. 3\.Performance and safety evaluation\.A 2,230\-request hosted 7B trace shows 53\.27% skipped LLM calls with zero observed dependency\-stale outputs\. Three hosted 32B operator\-suite seeds show 53\.31% skipped calls, exceeding versioned semantic, grounded\-style, entity\-period semantic, and exact\-invalidation baselines under the same dependency\-stale criterion\.
4. 4\.Metadata\-plane validation\.Field\-level ablations, capacity\-limited admission and eviction replay, policy\-history sensitivity, adversarial near\-collision probes, SLO and energy replay, human metadata audit, reverse\-index scaling, transactional\-backend validation, and interleaved query/update stress characterize the mechanism behind the performance gain and the consistency behavior of the metadata path\.
## 2Related work
### 2\.1RAG over financial documents
Financial question answering requires evidence selection, numerical reasoning, and period\-aware interpretation\. FinQA studies numerical reasoning over financial reports\[[9](https://arxiv.org/html/2607.26076#bib.bib9)\], and FinanceBench evaluates financial question answering against professional\-style evidence requirements\[[10](https://arxiv.org/html/2607.26076#bib.bib10)\]\. SEC EDGAR provides public APIs for filing metadata and submissions\[[11](https://arxiv.org/html/2607.26076#bib.bib11)\]\. These sources motivate a reproducible mutable\-document benchmark for RAG serving: requests are repetitive, answers are valuable, and source versions matter\.
### 2\.2LLM serving caches
vLLM introduced PagedAttention to improve KV\-cache memory management for high\-throughput LLM serving\[[1](https://arxiv.org/html/2607.26076#bib.bib1)\]\. SGLang exploits prefix sharing in structured language\-model programs\[[2](https://arxiv.org/html/2607.26076#bib.bib2)\]\. RAGCache and CacheBlend reuse knowledge or cached states in RAG execution\[[3](https://arxiv.org/html/2607.26076#bib.bib3),[4](https://arxiv.org/html/2607.26076#bib.bib4)\]\. ContextPilot and AdaptCache further optimize long\-context or KV\-cache reuse\[[5](https://arxiv.org/html/2607.26076#bib.bib5),[6](https://arxiv.org/html/2607.26076#bib.bib6)\]\. These systems reduce the cost of model execution\. FinCacheServe complements them by moving reuse to the final answer layer, where a valid hit removes the model invocation from the request path\.
### 2\.3Semantic response caching
Semantic caching reuses responses for similar inputs and can reduce LLM serving cost\[[12](https://arxiv.org/html/2607.26076#bib.bib12)\]\. Krites adds asynchronous verification for tiered semantic\-cache promotion\[[13](https://arxiv.org/html/2607.26076#bib.bib13)\]\. GroundedCache\-style routing adds evidence overlap and source\-validity gates to RAG answer reuse\[[14](https://arxiv.org/html/2607.26076#bib.bib14)\]\. MinCache uses hierarchical exact, resemblance, and semantic matching for efficient chatbot caching\[[15](https://arxiv.org/html/2607.26076#bib.bib15)\]\. FinCacheServe adds a domain\-specific dependency contract for mutable enterprise documents: answer reuse depends on financial identity, period, document scope, evidence fingerprints, tool outputs, model identity, generation parameters, and filing versions\.
### 2\.4Cache consistency and provenance
HTTP caching separates reuse from validation by requiring freshness metadata and revalidation semantics\[[16](https://arxiv.org/html/2607.26076#bib.bib16)\]\. Materialized\-view maintenance propagates base\-data updates to derived results\[[17](https://arxiv.org/html/2607.26076#bib.bib17),[18](https://arxiv.org/html/2607.26076#bib.bib18)\]\. Data provenance records why and where derived data were produced\[[19](https://arxiv.org/html/2607.26076#bib.bib19)\]\. Large\-scale data serving systems expose the performance\-consistency tradeoff under updates\[[20](https://arxiv.org/html/2607.26076#bib.bib20),[21](https://arxiv.org/html/2607.26076#bib.bib21)\]\. FinCacheServe applies these principles to RAG serving, where the cached object is a natural\-language answer backed by mutable evidence and serving\-time tools\.
## 3System model and consistency target
The service receives a requestr=\(q,m,θ\)r=\(q,m,\\theta\), whereqqis the user query,mmis the model identifier, andθ\\thetais the decoding configuration\. Retrieval returns evidence chunksErE\_\{r\}from documentsDrD\_\{r\}\. Each documentddhas a versionv\(d\)v\(d\)in the metadata plane\. Tool execution produces structured outputsTrT\_\{r\}such as ratio values, table slices, or extracted filing facts\.
An answer\-cache entrycccontains an answeraa, a normalized request signatures\(c\)s\(c\), source\-document versionsV\(c\)V\(c\), evidence fingerprinthE\(c\)h\_\{E\}\(c\), tool fingerprinthT\(c\)h\_\{T\}\(c\), model identitym\(c\)m\(c\), decoding configurationθ\(c\)\\theta\(c\), and reverse\-index memberships from source documents to answer entries\.
The primary performance target is LLM skip rate:
skiprate=requestsservedfromanswercachemeasuredrequests\.\\mathrm\{skip\\ rate\}=\\frac\{\\mathrm\{requests\\ served\\ from\\ answer\\ cache\}\}\{\\mathrm\{measured\\ requests\}\}\.\(1\)
The primary consistency target is dependency freshness\. A served output is dependency\-stale when the answer or prompt uses a cached dependency whose recorded source\-document version, evidence fingerprint, or tool fingerprint conflicts with the current metadata state for the request\. This definition covers stale answer\-cache hits and stale retrieval\-evidence reuse\. It evaluates consistency with recorded dependencies; factual answer accuracy is evaluated separately through answer audit and source review\.
The reuse gate has a linearization pointτ\\tau\. A cached answer can be served atτ\\tauwhen all predicates below hold:
s\(r\)\\displaystyle s\(r\)≃s\(c\),\\displaystyle\\simeq s\(c\),\(2\)∀d∈Dc,vτ\(d\)\\displaystyle\\forall d\\in D\_\{c\},\\quad v\_\{\\tau\}\(d\)=V\(c,d\),\\displaystyle=V\(c,d\),\(3\)hE\(r\)\\displaystyle h\_\{E\}\(r\)=hE\(c\),\\displaystyle=h\_\{E\}\(c\),\(4\)hT\(r\)\\displaystyle h\_\{T\}\(r\)=hT\(c\),\\displaystyle=h\_\{T\}\(c\),\(5\)m\\displaystyle m=m\(c\),θ=θ\(c\)\.\\displaystyle=m\(c\),\\quad\\theta=\\theta\(c\)\.\(6\)Filing updates linearize by updating the version store and invalidating dependent answer entries before later reuse\-gate reads\. Concurrent queries and updates are ordered by this metadata\-plane linearization\.
## 4FinCacheServe design
Figure 1:FinCacheServe serving architecture\. A request reaches the answer\-reuse router after retrieval\-cache and vector\-search resolution\. The router admits reuse through financial signature, evidence, version, and tool/model gates\. Filing updates update the version store and invalidate dependent answer entries before later cache reads\.### 4\.1Request signature
The request signature contains company identity, reporting period, query family, document scope, tool requirement, model identity, and decoding configuration\. Query family captures analytical intent, including liquidity, revenue change, debt, margins, risk\-factor change, and cash\-flow movement\. Period identity prevents cross\-quarter or cross\-year reuse\. Document scope separates annual reports, quarterly reports, current reports, and extracted XBRL facts\. Tool requirement separates generated textual explanations from requests that depend on computed financial quantities\.
### 4\.2Answer\-cache entry
Each answer\-cache entry stores answer text, cited document identifiers, chunk identifiers, chunk hashes, document versions, evidence fingerprint, tool fingerprint, model identity, decoding configuration, and request signature fields\. The evidence fingerprint is an order\-stable digest of cited chunk identifiers, chunk hashes, and document versions\. The tool fingerprint hashes the tool name, tool version, normalized inputs, and returned structured payload\. These hashes turn reusable answers into dependency\-bound serving objects\.
### 4\.3Reuse gate
The reuse gate first looks up exact and signature\-compatible candidates\. Candidate entries pass through four gate families\. The signature gate checks company, query family, period, document scope, tool requirement, model identity, and decoding configuration\. The evidence gate compares retrieved\-support fingerprints\. The version gate compares stored source\-document versions against the current version store\. The tool/model gate checks tool fingerprints and model configuration\. A rejected candidate routes to retrieval, tool execution, model generation, and promotion into the answer cache\.
### 4\.4Dependency invalidation
The metadata plane maintains a document\-to\-answer reverse index\. When a filing update changes a document version, the system updates the version store and uses the reverse index to invalidate dependent answer entries\. The lookup path isO\(1\)O\(1\)for exact or bucketed signature candidates plus gate evaluation over a small candidate set\. Invalidation isO\(k\)O\(k\)forkkanswer entries that depend on the updated document\. The memory footprint isO\(N\+E\)O\(N\+E\)forNNanswer entries andEEdocument\-to\-answer edges\.
### 4\.5Admission and eviction under bounded capacity
Hosted deployments bound answer\-cache capacity by memory, privacy retention, and operational isolation\. FinCacheServe therefore separates the correctness gate from the cache\-management policy\. The gate decides whether a candidate answer is dependency\-consistent\. The admission and eviction policy decides which generated answers deserve scarce cache space\.
For an answer entrycc, the default utility score is
U\(c\)=H^\(s\(c\)\)C\(c\)−λmM\(c\)−λfF\(c\)−λuR\(c\),U\(c\)=\\hat\{H\}\(s\(c\)\)\\,C\(c\)\-\\lambda\_\{m\}M\(c\)\-\\lambda\_\{f\}F\(c\)\-\\lambda\_\{u\}R\(c\),\(7\)whereH^\(s\(c\)\)\\hat\{H\}\(s\(c\)\)estimates reuse demand for the normalized signature,C\(c\)C\(c\)estimates avoided generation cost,M\(c\)M\(c\)is entry size,F\(c\)F\(c\)is document fan\-out, andR\(c\)R\(c\)is update hazard for the dependent documents\. Entries with positive utility are admitted\. When capacity is exhausted, FinCacheServe evicts the entry with the lowest utility densityU\(c\)/M\(c\)U\(c\)/M\(c\)\. The policy runs in the metadata plane: it uses routing fields, document dependencies, and historical request counts already collected by the serving layer, and it leaves the dependency\-consistency gate unchanged\.
The offline CacheOpt policy estimatesH^\\hat\{H\},FF, andRRfrom the replayed trace to establish an upper\-bound policy target short of Belady\. The online CacheOpt policy estimates these terms from a sliding policy\-history window of recent requests and filing updates\. This separation makes the cache\-management result auditable: global trace statistics define a policy ceiling, while the online variant measures deployable adaptation from serving history\.
### 4\.6Relationship to baseline cache policies
Table[1](https://arxiv.org/html/2607.26076#S4.T1)positions FinCacheServe against cache families used in the evaluation\. The central mechanism is the reuse contract: a cached answer remains eligible while its financial identity, evidence support, tool state, model state, and source versions remain compatible with the incoming request\.
Table 1:Mechanism boundary relative to cache families evaluated in this paper\.
## 5Implementation
The implementation contains five components: a request normalizer, retrieval cache, answer cache, metadata plane, and hosted\-generation connector\. Retrieval uses cached embeddings and FAISS search\[[7](https://arxiv.org/html/2607.26076#bib.bib7)\]\. Generation uses vLLM\-hosted Qwen2\.5\-Instruct models\[[1](https://arxiv.org/html/2607.26076#bib.bib1),[8](https://arxiv.org/html/2607.26076#bib.bib8)\]\. The cache path records request\-level routing, LLM invocation flags, skipped\-call counters, token estimates, evidence hashes, tool hashes, version checks, stale\-origin categories, and provider\-call closure\.
The answer cache maintains an exact index for repeated requests, a signature bucket index for financial\-intent reuse, and a document\-to\-answer reverse index for invalidation\. Gate decisions are logged per request\. A cache hit records whether it is exact, signature, or semantic within a signature bucket\. A cache miss records the rejected candidate family and the fallback generation path\. Filing updates enter the metadata plane as version\-store updates, followed by invalidation of all reverse\-indexed answer entries\.
The concurrency implementation uses a service\-level metadata lock around version\-store updates and answer\-entry mutation\. Hosted LLM calls execute outside the metadata critical section\. The design preserves a single linearization order for cache reads and filing updates in the evaluated implementation\.
## 6Experimental methodology
### 6\.1Workloads
The evaluation uses SEC\-derived financial\-document workloads\. Public filing content and metadata provide company, period, document scope, and source\-version structure\. Requests cover liquidity, revenue, margin, debt, risk, cash\-flow, filing\-change, and ratio\-analysis families\. Table[2](https://arxiv.org/html/2607.26076#S6.T2)summarizes the workloads\.
Table 2:Evaluation workloads\. Hosted workloads invoke vLLM; metadata stress runs the cache metadata path with deterministic fixtures\.
### 6\.2Hosted execution environment
Hosted generation experiments ran on AutoDL cloud GPU instances through a vLLM HTTP serving endpoint\. The hosted 7B full\-trace, update\-stress, filing\-QA, and concurrency runs used a single NVIDIA GeForce RTX 5090 GPU with 32,607 MiB of reported memory \(32 GB class\) serving Qwen2\.5\-7B\-Instruct\. The focused 14B model\-size probes used the same RTX 5090\-class hosted environment\. The hosted 32B operator\-suite runs used a single NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with 97,887 MiB of reported memory \(96 GB class\) and NVIDIA driver 595\.58\.03, serving Qwen2\.5\-32B\-Instruct\. The deterministic capacity replay, near\-collision probes, interleaved metadata stress, SQLite/WAL backend validation, SLO replay, energy replay, and normalizer audit were computed from archived traces or metadata fixtures and did not invoke a GPU\. GPU\-second and Wh results use archived per\-request hosted timing metrics and explicit board\-power assumptions; they are resource\-sensitivity estimates rather than direct whole\-system energy\-meter readings\.
### 6\.3Baselines
The evaluation compares FinCacheServe with no\-cache serving, retrieval\-cache serving, TTL answer caching, semantic answer caching without freshness checks, exact answer reuse with document invalidation, versioned semantic caching, entity\-period semantic caching, grounded\-style reuse, financial\-signature reuse without freshness invalidation, and ablated FinCacheServe variants\. The strongest safe semantic baseline combines semantic similarity with document\-version checks\. The grounded\-style baseline checks evidence overlap and source support before reuse\. The exact\-invalidation baseline measures repeated\-text reuse under document invalidation\.
### 6\.4Metrics
The main efficiency metrics are LLM invocations, skipped LLM calls, skip rate, provider\-call closure, prompt\-token savings, completion\-token savings, TTFT, end\-to\-end latency, throughput, SLO goodput, GPU seconds, and power\-sensitivity estimates where available\. SLO goodput counts requests that are error\-free, dependency\-fresh, and complete within a latency budget\. Energy sensitivity reports Wh per 1,000 dependency\-fresh SLO successes under explicit board\-power assumptions\. The cache\-management replay additionally reports saved relative generation cost, oracle skip gap, eviction count, invalidation count, and lookup/invalidation p95 latency\. The metadata\-backend validation reports lookup p95, invalidation p95, storage bytes per entry, invalidated entries per update, and stale serves\. The main consistency metrics are answer\-cache stale serves, retrieval\-evidence stale serves, tool\-output stale serves, version mismatches, evidence\-hash mismatches, and tool\-hash mismatches\. Wilson intervals are reported for selected proportion estimates where intervals are shown\. Paired request\-level comparisons report skip\-rate differences when paired route\-level outcomes are available\.
## 7Results
### 7\.1Full hosted 7B trace
Table[3](https://arxiv.org/html/2607.26076#S7.T3)reports the main hosted 7B trace\. FinCacheServe skips 1,188 of 2,230 LLM calls \(53\.27%\) with zero observed dependency\-stale outputs\. The no\-cache and retrieval\-cache baselines invoke the model for every measured request\. TTL and semantic no\-freshness variants reach similar reuse levels but expose stale outputs after dependency updates\.
Table 3:Full hosted Qwen2\.5\-7B trace\. Dependency\-stale counts aggregate answer\-cache stale, retrieval\-evidence stale, and tool\-output stale categories\.Figure[7\.1](https://arxiv.org/html/2607.26076#S7.SS1)summarizes cost reduction and dependency\-freshness behavior\. The main trace establishes that answer\-level reuse can remove more than half of hosted LLM calls under full provider\-call accounting\. The single stale output in retrieval\-cache serving also motivates stale\-origin reporting: stale dependencies can enter through cached retrieval evidence as well as answer\-cache hits\.
Figure 2:LLM\-call reduction and dependency\-stale behavior across primary hosted conditions\.
### 7\.2Paraphrase and update stress
The 208\-request hosted stress workload exercises repeated financial intent, paraphrases, and filing\-update events\. FinCacheServe skips 52\.88% of LLM calls with zero dependency\-stale outputs\. Unsafe answer\-cache controls expose the freshness cost of reuse without dependency invalidation: TTL, semantic no\-freshness, signature no\-freshness, and FinCacheServe without freshness invalidation each serve 46 dependency\-stale outputs\. Grounded\-style safe reuse avoids stale outputs but skips 11\.06% of calls\. The result separates two axes: aggressive answer reuse improves GPU cost, and dependency checks govern freshness under updates\.
Table 4:Hosted paraphrase/update stress with Qwen2\.5\-7B\.
### 7\.3Strong 32B baselines across three hosted seeds
Table[5](https://arxiv.org/html/2607.26076#S7.T5)reports three hosted Qwen2\.5\-32B operator\-suite seeds\. FinCacheServe skips 290 of 544 requests \(53\.31%, 95% CI \[49\.11, 57\.46\]\) with zero observed dependency\-stale outputs\. The strongest safe semantic baseline skips 212 requests \(38\.97%\), grounded\-style reuse skips 122 \(22\.43%\), entity\-period semantic caching skips 57 \(10\.48%\), and exact answer reuse with document invalidation skips 14 \(2\.57%\)\. Unsafe semantic or signature caches skip many calls but serve 126–134 dependency\-stale outputs\.
Table 5:Hosted 32B operator\-suite comparison across three seeds\. PairedΔ\\Deltaskip is FinCacheServe minus baseline in percentage points\.The 32B result identifies the main performance mechanism\. Removing the financial\-signature layer reduces safe reuse to 10\.48%, matching the entity\-period semantic baseline\. Removing freshness invalidation preserves skip rate but exposes 126 dependency\-stale outputs\. The safe reuse gain therefore comes from the combination of financial\-intent indexing and dependency invalidation\.
### 7\.4Capacity\-limited admission and eviction
Hosted answer caches operate under bounded memory\. The 32B operator\-suite traces therefore support a deterministic metadata\-plane replay that preloads the same 120 warmup requests used by the hosted runs, replays 895 measured requests and 110 filing\-update events across five archived traces, and varies answer\-cache capacity\. This replay evaluates cache\-management decisions while keeping the dependency\-consistency gate fixed\.
Figure[3](https://arxiv.org/html/2607.26076#S7.F3)shows that eviction policy dominates performance when the answer cache is smaller than the warmup working set\. At 64 entries, LRU skips 1\.01% of measured calls and LFU skips 1\.45%; both policies evict entries that become useful during the measured phase\. Utility\-aware admission and eviction skips 36\.66% of calls with zero dependency\-stale serves, within 0\.56 percentage points of the Belady\-style offline oracle\. Online CacheOpt uses sliding\-window request and update history\. With the conservative 128\-event window used in the main replay, it skips 19\.44% at the same capacity, exceeding LRU, LFU, and TinyLFU while retaining a 17\.77 percentage\-point oracle gap\. At 128 entries, LRU, LFU, CacheOpt, Online CacheOpt, and the offline oracle all reach 68\.06% skipped calls; TinyLFU remains lower in this replay because its frequency sketch retains fewer measured\-phase reusable entries\.
Figure 3:Capacity\-limited metadata\-plane replay\. Utility\-aware admission and eviction preserves reusable answer entries under bounded capacity and approaches the offline oracle at 32 and 64 entries\. Error bars show standard deviation across five archived traces\.Table 6:Capacity\-limited answer\-cache replay over five archived 32B traces\. Oracle gap is the skip\-rate difference from the offline Belady\-style oracle at the same capacity\.Figure[7\.4](https://arxiv.org/html/2607.26076#S7.SS4)reports the sliding\-window online policy over replay windows\. The first measured window contains many warmed entries and therefore has high reuse opportunity\. Later windows contain fewer repeated signatures after invalidation, reducing the online hit rate\. The result gives a deployable policy curve alongside the offline policy ceiling and separates deployable history\-based adaptation from trace\-wide utility estimation\.
Figure 4:Online CacheOpt behavior at 64 answer entries\. The policy estimates reuse demand and update hazard from recent serving history\. Error bars show standard deviation across five archived traces\.Figure[7\.4](https://arxiv.org/html/2607.26076#S7.SS4)varies the policy\-history window at 64 entries\. A 32\-event window skips 36\.10% of calls with zero dependency\-stale serves and a 1\.12 percentage\-point oracle gap\. Windows of 128–512 events skip 19\.44%, reflecting older update history that suppresses admission for entries with near\-term reuse\. The scan preserves the distinction between the trace\-wide offline policy, the deployable online policy, and the Belady\-style oracle\.
Figure 5:Online CacheOpt policy\-history sensitivity at 64 answer entries\. The online policy remains dependency\-fresh across all tested windows; the shortest tested window best matches the locality in the archived traces\.Figure[7\.4](https://arxiv.org/html/2607.26076#S7.SS4)varies the filing\-update rate at 64 entries\. Utility\-aware cache management retains a stable advantage over LRU when update pressure is low or moderate, while dependency invalidation drives all safe policies toward lower reuse under high update pressure\. The unsafe control skips 37\.87% of calls across update rates while serving stale dependencies in 22\.78–32\.17% of measured requests, showing that invalidation is required for deployable call saving\.
Figure 6:Update\-rate sensitivity at 64 answer entries\. Higher filing\-update pressure reduces safe answer reuse through dependency invalidation; the unsafe control preserves apparent reuse by serving stale dependencies\.
### 7\.5Model\-size sensitivity
The matched model\-size probes use Qwen2\.5\-7B, 14B, and 32B under the same route\-level workloads\. Workload structure and reuse policy determine skip behavior, while the monetary and latency value of each skipped call grows with model size\. The 14B and 32B self\-hosted vLLM runs also expose memory pressure and lower single\-sequence serving throughput, strengthening the performance motivation for answer\-level skipping on larger models\.
Table 7:Matched hosted model\-size probes\. Values summarize measured route\-level requests\.
### 7\.6Hosted concurrency and throughput
Figure[7\.6](https://arxiv.org/html/2607.26076#S7.SS6)reports the hosted concurrency sweep over a bounded 53\-request slice\. FinCacheServe improves throughput over retrieval\-cache serving by 1\.58×\\timeson average across the tested concurrency settings while preserving zero observed dependency\-stale outputs\. The result indicates that skipped model calls translate into serving\-path throughput under controlled concurrent query phases\.
Figure 7:Hosted throughput under bounded concurrent query phases\. Throughput is normalized against retrieval\-cache serving for the same phase\.
### 7\.7SLO\-constrained goodput and GPU\-second accounting
The hosted 32B metrics also support a service\-level replay over latency SLOs\. Goodput counts requests that finish within the SLO, return without error, and serve dependency\-fresh output\. Figure[7\.7](https://arxiv.org/html/2607.26076#S7.SS7)reports goodput at 0\.25s–5s budgets\. FinCacheServe reaches 53\.31% dependency\-fresh goodput at every SLO up to 2s because answer\-cache hits finish within the tight budgets while generation\-path requests require roughly 3s\. At 3s and 5s, generation\-path requests increasingly satisfy the budget, raising FinCacheServe goodput to 77\.02% and 100%\.
Figure 8:SLO\-constrained serving goodput from archived hosted 32B per\-request metrics\. Goodput counts error\-free, dependency\-fresh requests completed within the latency budget\.Table 8:SLO replay at a 2s latency budget over hosted 32B metrics\. GPU\-sec/1k good is GPU seconds per 1,000 dependency\-fresh SLO successes\.Table[7\.7](https://arxiv.org/html/2607.26076#S7.SS7)shows the difference between apparent speed and deployable goodput\. Signature no\-freshness has the highest skip rate and lowest GPU seconds per SLO success, but 23\.16% of requests are dependency\-stale\. FinCacheServe matches its 2s goodput while eliminating observed dependency\-stale output in the same hosted metrics\.
Figure[7\.7](https://arxiv.org/html/2607.26076#S7.SS7)converts the same GPU\-second estimates into Wh per 1,000 dependency\-fresh 2s\-SLO successes under explicit board\-power assumptions\. At 450 W, FinCacheServe uses 2\.80 Wh per 1,000 fresh SLO successes, compared with 5\.03 Wh for versioned semantic caching and 11\.15 Wh for grounded\-style reuse\. This is a 44\.30% reduction relative to the strongest safe semantic baseline\. The signature no\-freshness control reaches 1\.41 Wh but serves dependency\-stale outputs in 23\.16% of requests, so its resource curve represents an unsafe negative control\.
Figure 9:Energy sensitivity from archived hosted 32B metrics at a 2s latency SLO\. Wh values use measured GPU\-second estimates and explicit board\-power assumptions; unsafe baselines are shown as negative controls\.
### 7\.8Field\-level gate attribution
Table[9](https://arxiv.org/html/2607.26076#S7.T9)reports controlled field\-level gate ablations\. The full gate admits the benign paraphrase and rejects every unsafe boundary case\. Removing query family, period, tool hash, model identity, generation parameters, or evidence fingerprint exposes at least one unsafe reuse case\. The result supports the inclusion of these fields in the reuse contract\.
Table 9:Field\-level gate ablation over controlled boundary probes\.
### 7\.9Controlled financial near\-collision probes
The field\-level ablation isolates individual gates\. A larger controlled probe suite evaluates full cache contracts under financial near\-collisions\. The suite contains 2,376 probes across issuers, periods, filing scopes, query families, tool\-dependent families, evidence drifts, tool\-output drifts, model changes, generation\-parameter changes, and document\-version updates\. Each probe is semantically similar to a cached request; reuse is safe for benign paraphrases and unsafe for dependency\-changing probes\.
Figure 10:Unsafe accept rate on controlled financial near\-collision probes\. The suite isolates cache\-contract boundaries and measures reuse\-gate behavior under dependency\-changing financial requests\.Table 10:Controlled near\-collision cache\-safety suite\. Safe accept is the benign paraphrase accept rate; unsafe accept is the accept rate over dependency\-changing probes\.Table[10](https://arxiv.org/html/2607.26076#S7.T10)explains why single\-field cache contracts are insufficient for mutable financial RAG\. Versioned semantic caching blocks version\-update probes but accepts cross\-company, period, scope, evidence, tool, model, and generation\-parameter collisions\. Grounded\-style reuse blocks evidence and version collisions, yet it accepts model/configuration and several financial\-identity drifts\. FinCacheServe combines the financial signature, evidence/tool fingerprints, model/configuration identity, and version invalidation, blocking every unsafe probe while preserving benign paraphrase reuse in this controlled suite\.
### 7\.10Interleaved query/update consistency
Table[7\.10](https://arxiv.org/html/2607.26076#S7.SS10)reports the metadata\-plane stress\. Each setting runs five repeats with 4,096 queries and 512 filing updates per repeat\. FinCacheServe records zero dependency\-stale serves across 90 raw runs\. The no\-invalidation control records hundreds to more than one thousand stale serves per setting\. Lookup and invalidation p95 latencies remain below 0\.04 ms in the in\-memory implementation\.
Table 11:Interleaved query/update consistency stress\. Unsafe is the mean dependency\-stale serves in the no\-invalidation control\.
### 7\.11Transactional metadata\-backend validation
The interleaved stress above uses the in\-memory metadata path from the hosted implementation\. To check backend portability, a separate CPU validation maps the same answer\-entry table, document\-to\-entry reverse index, and version store to SQLite/WAL\. The workload covers 10k and 100k answer entries, fan\-out 1, 16, and 128, three repeats per setting, 4,096 lookup operations, and up to 512 document\-update transactions per run\.
Figure[11](https://arxiv.org/html/2607.26076#S7.F11)reports p95 lookup and invalidation latency for the in\-memory and SQLite/WAL backends\. At 100k entries and fan\-out 16, in\-memory lookup p95 is 2\.13μ\\mus and invalidation p95 is 27\.00μ\\mus\. SQLite/WAL lookup p95 is 84\.42μ\\mus and transactional invalidation p95 is 63\.44 ms while invalidating 16 entries per update on average\. Across 36 raw backend runs, both backends record zero dependency\-stale serves\. The result confirms that the dependency contract can be implemented over a persistent transactional backend, while high fan\-out updates introduce a measurable metadata\-write cost\.
Figure 11:Metadata backend validation over in\-memory and SQLite/WAL implementations\.
### 7\.12Normalizer and signature audit
Table[7\.12](https://arxiv.org/html/2607.26076#S7.SS12)reports the human audit of request metadata used by the reuse signature\. The audit covers 500 annotated rows and excludes 13 rows without auditable query/request metadata, leaving 487 analyzable rows\. Final signature\-safe labels accept 384 rows and conservatively route 103 rows to generation\. The audit records no hard contradiction in which a row is marked signature\-safe while a component field is unsafe\.
Table 12Human audit of metadata fields used for answer\-reuse signatures\.
The final signature\-safe rate is 78\.9% with Wilson 95% CI \[75\.0%, 82\.2%\]\. Among analyzable rows, deterministic query\-family consistency with the trace family is 87\.7% with Wilson 95% CI \[84\.5%, 90\.3%\]\. Rows marked unsafe by the audit become conservative generation routes, so metadata ambiguity primarily reduces reuse opportunity and preserves dependency freshness\.
## 8Discussion
FinCacheServe improves GPU\-serving efficiency by moving reuse from retriever outputs or model\-state reuse to dependency\-checked answers\. The 7B full trace establishes provider\-call savings at full\-run scale\. The stress and operator\-suite workloads show that the same mechanism keeps answer reuse fresh under filing updates\. The 32B results strengthen the performance argument because the skipped call has higher serving value as model size increases\. The capacity\-limited replay adds a resource\-management layer: a freshness\-consistent cache also needs admission and eviction policies that preserve entries with future serving value when warmup traffic exceeds memory capacity\. The SLO and energy replays translate skipped calls into dependency\-fresh goodput and resource sensitivity, showing that low\-latency success depends on both fast answers and dependency freshness\. The SQLite/WAL validation adds a backend\-realism check by mapping the metadata contract to persistent transactional storage\.
The results also clarify the role of each mechanism\. Financial\-signature indexing recovers paraphrased analytical intent that exact matching misses\. Versioned invalidation prevents stale reuse after filing updates\. Evidence and tool fingerprints protect boundary cases where document versions alone leave ambiguity\. Model identity and decoding configuration keep answer reuse aligned with the serving policy used at admission\. The near\-collision suite shows that these fields act as a joint contract: semantic similarity, version checks, evidence overlap, or financial signatures alone leave unsafe acceptance paths\. The complete contract creates a compact reuse object that can be audited, logged, and invalidated\.
The evaluation separates conservative reuse, unsafe reuse, and dependency\-consistent answer reuse\. Exact reuse and grounded\-style safe reuse preserve dependency freshness but leave many repeated financial requests on the GPU path\. Semantic and signature no\-freshness variants save calls aggressively and expose stale outputs\. FinCacheServe preserves most of the safe answer\-level savings while converting update\-sensitive cases into recomputation\. The CacheOpt replay further shows that simple recency policies can discard the reusable portion of a warmed answer cache under tight capacity\. Trace\-wide utility estimates approach the offline oracle, and sliding\-window Online CacheOpt provides a deployable policy curve with measurable oracle gap\. The policy\-history scan shows that online utility estimation is sensitive to workload locality; short recent\-history windows recover near\-offline reuse on the archived traces, while longer windows remain safe and more conservative\.
## 9Validity scope
The evidence covers public SEC\-derived workloads, hosted vLLM execution, Qwen2\.5 models, an in\-memory metadata plane with service\-level linearization, and a SQLite/WAL backend validation for persistent metadata operations\. The consistency claim is dependency freshness with respect to recorded source versions, evidence fingerprints, and tool fingerprints\. Factual financial correctness remains a separate application\-quality property and requires source review, answer audit, and domain governance in deployment\.
The hosted traces include repeated analytical requests and generated paraphrase/update stress over real filing\-derived dependencies\. The near\-collision probes are controlled mechanism tests that isolate dependency\-changing boundary conditions\. Production traffic can differ in query diversity, update cadence, access skew, tool composition, SLO distribution, and cache capacity\. The metadata replay evaluates capacity, update\-rate, and policy\-history sensitivity over archived traces\. The metadata stress validates the reverse\-index and invalidation path up to 100k entries and 64 query workers in the evaluated implementation; the SQLite/WAL validation shows transactional backend portability and high fan\-out invalidation cost\. Energy estimates use GPU\-second traces and explicit board\-power assumptions\. Distributed cache deployment, multi\-replica invalidation, and cross\-region consistency require additional engineering validation\.
## 10Conclusion
FinCacheServe demonstrates dependency\-consistent answer reuse for GPU\-efficient RAG serving over mutable financial documents\. The system treats generated answers as cacheable serving objects with explicit dependencies on financial identity, evidence support, tool outputs, model state, decoding configuration, and source\-document versions\. On a 2,230\-request hosted 7B trace, FinCacheServe skips 53\.27% of LLM calls with zero observed dependency\-stale outputs\. Across three hosted 32B operator\-suite seeds, it skips 53\.31% of requests, exceeding versioned semantic caching by 14\.34 percentage points and grounded\-style reuse by 30\.88 percentage points under the same dependency\-stale criterion\. Under a 64\-entry answer\-cache budget, utility\-aware admission skips 36\.66% of measured calls and stays within 0\.56 percentage points of an offline oracle; a 32\-event Online CacheOpt window skips 36\.10% with a 1\.12 percentage\-point oracle gap\. A 2,376\-probe near\-collision suite blocks every unsafe candidate under the complete reuse contract, and a hosted\-metrics replay reaches 53\.31% dependency\-fresh goodput at a 2s SLO\. At 450 W, the same hosted metrics imply 2\.80 Wh per 1,000 dependency\-fresh 2s\-SLO successes, 44\.30% below versioned semantic caching\. The metadata\-plane experiments reach 100k\-entry indexes, 64 query workers in interleaved stress, and SQLite/WAL transactional invalidation with zero observed dependency\-stale serves\. The results support answer\-level caching as a practical performance mechanism for RAG services when reuse is coupled to dependency tracking, capacity\-aware admission, and update\-driven invalidation\.
## Supplementary information
No supplementary artifact is submitted with this arXiv version\.
## Acknowledgements
The authors thank collaborators who provided early feedback on the problem setting and figure design\.
## Declarations
### Funding
No external funding was used for this study\.
### Conflict of interest
The authors declare no competing interests\.
### Ethics approval and consent to participate
Not applicable\.
### Data availability
No data artifact is submitted with this arXiv version\. The source filings are public SEC EDGAR documents\. A curated reproducibility package with accession identifiers, derived traces, summaries, and checksums may be prepared for a future venue or archival release\.
### Code availability
No code artifact is submitted with this arXiv version\. A curated reproducibility package with workload reconstruction, cache\-policy replay, trace validation, and table/figure generation code may be prepared for a future venue or archival release\.
### Author contribution
Lingteng Zeng: conceptualization, methodology, software, validation, formal analysis, investigation, data curation, visualization, writing, and artifact preparation\. Yifan Jin: data curation, validation, and investigation\.
## References
- \\bibcommenthead
- Kwon et al\. \[2023\]Kwon, W\., Li, Z\., Zhuang, S\., Sheng, Y\., Zheng, L\., Yu, C\.H\., Gonzalez, J\.E\., Zhang, H\., Stoica, I\.: Efficient memory management for large language model serving with PagedAttention\. In: Proceedings of the 29th ACM Symposium on Operating Systems Principles, pp\. 611–626 \(2023\)\.[https://doi\.org/10\.1145/3600006\.3613165](https://doi.org/10.1145/3600006.3613165)
- Zheng et al\. \[2023\]Zheng, L\., Yin, L\., Xie, Z\., Sun, C\., Huang, J\., Yu, C\.H\., Cao, S\., Kozyrakis, C\., Stoica, I\., Gonzalez, J\.E\., Barrett, C\., Sheng, Y\.: SGLang: Efficient Execution of Structured Language Model Programs \(2023\)\.[https://doi\.org/10\.48550/arXiv\.2312\.07104](https://doi.org/10.48550/arXiv.2312.07104)
- Jin et al\. \[2024\]Jin, C\., Zhang, Z\., Jiang, X\., Liu, F\., Liu, X\., Liu, X\., Jin, X\.: RAGCache: Efficient Knowledge Caching for Retrieval\-Augmented Generation \(2024\)\.[https://doi\.org/10\.48550/arXiv\.2404\.12457](https://doi.org/10.48550/arXiv.2404.12457)
- Yao et al\. \[2024\]Yao, J\., Li, H\., Liu, Y\., Ray, S\., Cheng, Y\., Zhang, Q\., Du, K\., Lu, S\., Jiang, J\.: CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion \(2024\)\.[https://doi\.org/10\.48550/arXiv\.2405\.16444](https://doi.org/10.48550/arXiv.2405.16444)
- Jiang et al\. \[2025\]Jiang, Y\., Huang, Y\., Cheng, L\., Deng, C\., Sun, X\., Mai, L\.: ContextPilot: Fast Long\-Context Inference via Context Reuse \(2025\)\.[https://doi\.org/10\.48550/arXiv\.2511\.03475](https://doi.org/10.48550/arXiv.2511.03475)
- Feng et al\. \[2025\]Feng, S\., Li, H\., Du, K\., Gu, Z\., Liu, Y\., Yao, J\., Ray, S\., Shen, S\., Cheng, Y\., Ananthanarayanan, G\., Jiang, J\.: AdaptCache: KV Cache Native Storage Hierarchy for Low\-Delay and High\-Quality Language Model Serving\. Accepted at SOSP 2025 BigMem Workshop \(2025\)\.[https://doi\.org/10\.48550/arXiv\.2509\.00105](https://doi.org/10.48550/arXiv.2509.00105)
- Johnson et al\. \[2021\]Johnson, J\., Douze, M\., Jégou, H\.: Billion\-scale similarity search with GPUs\. IEEE Transactions on Big Data7\(3\), 535–547 \(2021\)[https://doi\.org/10\.1109/TBDATA\.2019\.2921572](https://doi.org/10.1109/TBDATA.2019.2921572)
- Yang et al\. \[2024\]Yang, A\., Yang, B\., Zhang, B\., Hui, B\., Zheng, B\., Yu, B\., Li, C\., Liu, D\., Huang, F\., Wei, H\., Lin, H\., et al\.: Qwen2\.5 Technical Report \(2024\)\.[https://doi\.org/10\.48550/arXiv\.2412\.15115](https://doi.org/10.48550/arXiv.2412.15115)
- Chen et al\. \[2021\]Chen, Z\., Chen, W\., Smiley, C\., Shah, S\., Borova, I\., Langdon, D\., Moussa, R\., Beane, M\., Huang, T\.\-H\., Routledge, B\., Wang, W\.Y\.: FinQA: A dataset of numerical reasoning over financial data\. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp\. 3697–3711 \(2021\)\.[https://doi\.org/10\.18653/v1/2021\.emnlp\-main\.300](https://doi.org/10.18653/v1/2021.emnlp-main.300)
- Islam et al\. \[2023\]Islam, P\., Kannappan, A\., Kiela, D\., Qian, R\., Scherrer, N\., Vidgen, B\.: FinanceBench: A New Benchmark for Financial Question Answering \(2023\)\.[https://doi\.org/10\.48550/arXiv\.2311\.11944](https://doi.org/10.48550/arXiv.2311.11944)
- U\.S\. Securities and Exchange Commission \[2025\]U\.S\. Securities and Exchange Commission: EDGAR Application Programming Interfaces \(APIs\)\.[https://www\.sec\.gov/search\-filings/edgar\-application\-programming\-interfaces](https://www.sec.gov/search-filings/edgar-application-programming-interfaces)\. Last reviewed or updated April 8, 2025; accessed June 29, 2026 \(2025\)
- Liu et al\. \[2025\]Liu, X\., Atalar, B\., Dai, X\., Zuo, J\., Wang, S\., Lui, J\.C\.S\., Chen, W\., Joe\-Wong, C\.: Semantic Caching for Low\-Cost LLM Serving: From Offline Learning to Online Adaptation\. Accepted to IEEE INFOCOM 2026 \(2025\)\.[https://doi\.org/10\.48550/arXiv\.2508\.07675](https://doi.org/10.48550/arXiv.2508.07675)
- Singh et al\. \[2026\]Singh, A\.K\., Wang, H\., Attaluri, L\.N\.S\., Chiam, T\., Zhu, W\.: Asynchronous Verified Semantic Caching for Tiered LLM Architectures \(2026\)\.[https://doi\.org/10\.48550/arXiv\.2602\.13165](https://doi.org/10.48550/arXiv.2602.13165)
- Shah \[2026\]Shah, S\.H\.: Grounded Cache Routing for Retrieval\-Augmented Generation: When Is It Safe to Reuse an Answer? \(2026\)\.[https://doi\.org/10\.48550/arXiv\.2605\.27494](https://doi.org/10.48550/arXiv.2605.27494)
- Haqiq et al\. \[2025\]Haqiq, K\., Jahan, M\.V\., Farimani, S\.A\., Masoom, S\.M\.F\.: MinCache: A hybrid cache system for efficient chatbots with hierarchical embedding matching and LLM\. Future Generation Computer Systems170, 107822 \(2025\)[https://doi\.org/10\.1016/j\.future\.2025\.107822](https://doi.org/10.1016/j.future.2025.107822)
- Nottingham et al\. \[2022\]Nottingham, M\., Fielding, R\.T\., Reschke, J\.: HTTP Caching\. RFC 9111 \(2022\)\.[https://doi\.org/10\.17487/RFC9111](https://doi.org/10.17487/RFC9111)\.[https://www\.rfc\-editor\.org/rfc/rfc9111](https://www.rfc-editor.org/rfc/rfc9111)
- Gupta and Mumick \[1995\]Gupta, A\., Mumick, I\.S\.: Maintenance of materialized views: Problems, techniques, and applications\. IEEE Data Engineering Bulletin18\(2\), 3–18 \(1995\)
- Mistry et al\. \[2001\]Mistry, H\., Roy, P\., Sudarshan, S\., Ramamritham, K\.: Materialized view selection and maintenance using multi\-query optimization\. In: Proceedings of the 2001 ACM SIGMOD International Conference on Management of Data, pp\. 307–318 \(2001\)\.[https://doi\.org/10\.1145/375663\.375706](https://doi.org/10.1145/375663.375706)
- Buneman et al\. \[2001\]Buneman, P\., Khanna, S\., Tan, W\.\-C\.: Why and where: A characterization of data provenance\. In: Database Theory – ICDT 2001, pp\. 316–330 \(2001\)\.[https://doi\.org/10\.1007/3\-540\-44503\-X\_20](https://doi.org/10.1007/3-540-44503-X_20)
- DeCandia et al\. \[2007\]DeCandia, G\., Hastorun, D\., Jampani, M\., Kakulapati, G\., Lakshman, A\., Pilchin, A\., Sivasubramanian, S\., Vosshall, P\., Vogels, W\.: Dynamo: Amazon’s highly available key\-value store\. In: Proceedings of Twenty\-First ACM SIGOPS Symposium on Operating Systems Principles, pp\. 205–220 \(2007\)\.[https://doi\.org/10\.1145/1294261\.1294281](https://doi.org/10.1145/1294261.1294281)
- Cooper et al\. \[2008\]Cooper, B\.F\., Ramakrishnan, R\., Srivastava, U\., Silberstein, A\., Bohannon, P\., Jacobsen, H\.\-A\., Puz, N\., Weaver, D\., Yerneni, R\.: PNUTS: Yahoo\!’s hosted data serving platform\. In: Proceedings of the VLDB Endowment, vol\. 1, pp\. 1277–1288 \(2008\)\.[https://doi\.org/10\.14778/1454159\.1454167](https://doi.org/10.14778/1454159.1454167)相似文章
语义缓存蒸馏:通过重用与选择性修补实现高效状态转移
本文提出语义缓存蒸馏(SCD),一种带损失约束的框架,用紧凑的语义码替换原始KV缓存传输,在保持生成质量在oracle的5% F1内的同时,实现高达2.65倍的TTFT加速。
GRACE-RAG:规范证据合成的受控检索架构,支持在封闭领域机构环境中轻量化部署
本文介绍了 GRACE-RAG,这是一种检索受控、图增强的 RAG 架构,它将结构推理从生成过程外化到结构化的检索层,从而能够在封闭领域的机构环境中实现轻量化部署。实验表明,在中规模模型上质量提升高达 20%,同时减少了计算和延迟开销。
当更多文档损害RAG:通过领域限定、模型无关的检索缓解向量搜索稀释
本文识别了RAG系统在扩展到大规模异构文档集合时出现的“向量搜索稀释”现象,并提出MASDR-RAG,一种利用组织元数据进行领域限定的检索方法,显著提升了检索准确率。
面向金融文档问答的代理式检索增强生成
本文介绍了 FinAgent-RAG,这是一个用于金融文档问答的代理式框架,它结合了迭代检索、程序化思维推理和自适应资源分配,以提高准确性并降低成本。
预填充即服务:下一代模型的 KVCache 可跨数据中心
研究者提出“预填充即服务”(PrfaaS),将长上下文预填充卸载到远程计算密集型集群,并通过普通以太网流式传输 KVCache,实现独立扩缩容,使 1T 参数混合模型吞吐提升 32–54%。