Cost-Governed RAG: Unified Per-Tenant Cost Attribution Across Retrieval and Generation in Multi-Tenant LLM Systems

# Cost-Governed RAG: Unified Per-Tenant Cost Attribution Across Retrieval and Generation in Multi-Tenant LLM Systems
Source: [https://arxiv.org/html/2607.12188](https://arxiv.org/html/2607.12188)
###### Abstract

Enterprise Retrieval\-Augmented Generation \(RAG\) deployments face a critical governance gap: while LLM generation cost is metered per token, the retrieval layer—vector memory, similarity compute, and embedding API calls—remains an unattributed shared cost, enabling invisible cross\-subsidization among tenants\. We present Cost\-Governed RAG, an architecture that integrates a codebook\-oblivious vector index \(TurboVec\) with a multi\-tenant LLM governance gateway, creating a unified observability stack where embedding, retrieval, and generation costs are jointly attributable per tenant\. The architecture exploits TurboVec’s deterministic, closed\-form memory formula to enable near\-exact per\-tenant retrieval cost calculation—a property unavailable in graph\-based indexes with non\-linear memory overhead\. Deployed on Snowpark Container Services within a cloud data platform’s governance boundary, the system achieves 99\.96% end\-to\-end cost attribution accuracy across 100 simulated tenants \(10M vectors, log\-normal size distribution\) with telemetry overhead below 0\.04% of query latency\. The architecture reduces retrieval infrastructure cost by 3\.1–9\.0×\\timescompared to managed vector database services under the pricing assumptions detailed in Section IV\. We formalize a three\-layer cost model and demonstrate that codebook\-oblivious quantization enables deterministic per\-tenant cost attribution while also removing the shared\-codebook leakage surface present in trained quantizers—the latter observation being exploratory and subject to the limitations described in Section VII\.

## IIntroduction

Enterprise LLM deployments increasingly use Retrieval\-Augmented Generation \(RAG\)\[[1](https://arxiv.org/html/2607.12188#bib.bib1)\]to ground model outputs in proprietary knowledge bases\. While recent work addresses the*generation*cost of LLM inference through model cascading\[[5](https://arxiv.org/html/2607.12188#bib.bib5)\], prompt compression\[[6](https://arxiv.org/html/2607.12188#bib.bib6)\], and token\-level metering\[[17](https://arxiv.org/html/2607.12188#bib.bib17)\], the*retrieval*cost—memory consumed by vector indexes, compute spent on similarity search, and embedding API calls—remains a blind spot in enterprise cost governance\.

This gap is consequential in multi\-tenant deployments where multiple business units or customers share infrastructure\. A tenant with 10M documents indexed at full precision \(FP32\) consumes∼\\sim57 GiB of vector memory, while a tenant with 100K documents consumes∼\\sim0\.57 GiB—yet both may be charged the same flat infrastructure rate\. Without retrieval\-layer cost attribution, cross\-subsidization is invisible to FinOps teams\[[21](https://arxiv.org/html/2607.12188#bib.bib21)\]\.

The challenge is compounded by the opacity of modern vector indexes\. Graph\-based indexes \(HNSW\[[14](https://arxiv.org/html/2607.12188#bib.bib14)\]\) have non\-linear, graph\-topology\-dependent memory overhead that defies simple per\-tenant attribution\. Trained codebook quantizers \(PQ\[[11](https://arxiv.org/html/2607.12188#bib.bib11)\]\) share codebook state across tenants, creating both a cost\-attribution ambiguity and a potential privacy concern\[[3](https://arxiv.org/html/2607.12188#bib.bib3)\]\.

We address this gap by integrating two complementary systems:

1. 1\.TurboVec\[[2](https://arxiv.org/html/2607.12188#bib.bib2),[3](https://arxiv.org/html/2607.12188#bib.bib3)\]: A codebook\-oblivious vector index whose memory consumption is exactlynt×d×b/8n\_\{t\}\\times d\\times b/8per tenant—deterministic, linear, and trivially attributable\.
2. 2\.GovLLM: A multi\-tenant LLM governance gateway providing per\-tenant token attribution, authentication, model RBAC, and observability tracing\[[4](https://arxiv.org/html/2607.12188#bib.bib4)\]\.

Together, they form a*Cost\-Governed RAG*stack where retrieval cost \(memory, QPS\), generation cost \(tokens, credits\), and embedding cost \(API calls\) all close into a unified per\-tenant observability layer within the cloud data platform—without external data movement\.

Contributions\.

1. 1\.A three\-layer cost model formalizing embedding, retrieval, and generation costs per tenant, showing that codebook\-oblivious quantization enables near\-exact retrieval attribution with bounded residual error \(Section[III\-B](https://arxiv.org/html/2607.12188#S3.SS2)\)\.
2. 2\.An architecture deploying the full Cost\-Governed RAG stack on Snowpark Container Services with integrated telemetry \(Section[III](https://arxiv.org/html/2607.12188#S3)\)\.
3. 3\.Empirical validation on a 100\-tenant simulation \(10M vectors\) showing 99\.96% end\-to\-end cost attribution accuracy, 3\.1–9\.0×\\timescost reduction versus managed alternatives, and sub\-0\.04% telemetry overhead \(Section[IV](https://arxiv.org/html/2607.12188#S4)\)\.

## IIBackground and Motivation

### II\-AThe RAG Cost Stack

A complete RAG query incurs cost at three distinct layers, each with different attribution characteristics:

TABLE I:RAG Cost Layers and Attribution CharacteristicsExisting LLM observability platforms \(Langfuse\[[17](https://arxiv.org/html/2607.12188#bib.bib17)\], LangSmith\[[18](https://arxiv.org/html/2607.12188#bib.bib18)\], Helicone\[[19](https://arxiv.org/html/2607.12188#bib.bib19)\]\) trace generation cost per tenant but treat retrieval as a fixed infrastructure cost amortized uniformly\. This creates a systematic governance failure: tenants with large corpora subsidize tenants with small corpora, and retrieval\-heavy workloads \(many queries, large top\-kk\) are invisible to cost dashboards\.

### II\-BWhy Existing Vector Indexes Resist Cost Attribution

HNSW\[[14](https://arxiv.org/html/2607.12188#bib.bib14)\]:Graph\-based indexes store navigable small\-world graphs whose edge structure is corpus\-dependent and shared across all vectors\. A tenant’s vectors participate in edges connecting to other tenants’ vectors, making per\-tenant memory isolation impossible without physical partitioning\.

Trained PQ\[[11](https://arxiv.org/html/2607.12188#bib.bib11)\]:Product Quantization learnsm×Km\\times Kcentroids viakk\-means on the full corpus\. This codebook is shared across all tenants and encodes corpus\-wide distributional structure\. Per\-tenant codebook attribution is undefined, and the codebook itself may leak cross\-tenant statistics\[[3](https://arxiv.org/html/2607.12188#bib.bib3)\]\.

TurboVec\[[2](https://arxiv.org/html/2607.12188#bib.bib2)\]:Codebook\-oblivious quantization derives boundaries analytically from the known marginal distribution of L2\-normalized vectors after random rotation\. The codebook contains zero corpus\-dependent information and zero per\-tenant state\. Total serving memory per tenant decomposes as:

Mt=nt×d×b8⏟quantized codes\+nt×4⏟norms\+nt×8⏟IDs\+nt×α⏟block metadata\+d2×4T⏟rotation \(shared\)M\_\{t\}=\\underbrace\{\\frac\{n\_\{t\}\\times d\\times b\}\{8\}\}\_\{\\text\{quantized codes\}\}\+\\underbrace\{n\_\{t\}\\times 4\}\_\{\\text\{norms\}\}\+\\underbrace\{n\_\{t\}\\times 8\}\_\{\\text\{IDs\}\}\+\\underbrace\{n\_\{t\}\\times\\alpha\}\_\{\\text\{block metadata\}\}\+\\underbrace\{\\frac\{d^\{2\}\\times 4\}\{T\}\}\_\{\\text\{rotation \(shared\)\}\}\(1\)wherentn\_\{t\}= tenant’s vector count,dd= dimension,bb= bit width,α≈0\.5\\alpha\\approx 0\.5bytes/vector for 32\-vector block headers, andTT= number of tenants sharing the rotation matrix\. All terms except the last are deterministic and linear inntn\_\{t\}; the rotation matrix \(d2×4=9\.0d^\{2\}\\times 4=9\.0MB atd=1536d=1536\) is shared and negligible at scale\. At 100K vectors and 4\-bit:

- •Codes:100​K×1536×4/8=76\.8100\\text\{K\}\\times 1536\\times 4/8=76\.8MB
- •Norms:100​K×4=0\.4100\\text\{K\}\\times 4=0\.4MB
- •IDs:100​K×8=0\.8100\\text\{K\}\\times 8=0\.8MB
- •Block metadata:100​K×0\.5=0\.05100\\text\{K\}\\times 0\.5=0\.05MB
- •Total per\-tenant:≈78\.0\\approx 78\.0MB \(payload\-dominated\)

This determinism—every component is a closed\-form function ofntn\_\{t\},dd, andbb—is the key enabler of near\-exact cost attribution\.

### II\-CGovLLM: Multi\-Tenant LLM Governance

GovLLM\[[4](https://arxiv.org/html/2607.12188#bib.bib4)\]provides per\-tenant token attribution via a FastAPI gateway with:

- •Per\-tenant authentication \(PAT/JWT/OAuth\) with configurable rate limits
- •Model RBAC restricting which tenants access which LLMs and at what tier
- •Langfuse\-compatible tracing with telemetry closure into the cloud data platform
- •Per\-query cost records written to platform\-native tables

## IIIArchitecture

### III\-ADeployment on Snowpark Container Services

The Cost\-Governed RAG stack deploys within Snowpark Container Services \(SPCS\), a container runtime within Snowflake’s governance boundary\. This deployment topology ensures:

- •Data never leaves the platform’s governance perimeter
- •Compute is billed through the platform’s native credit metering
- •Telemetry writes directly to platform tables \(no external egress\)
- •Network isolation via VPC\-level controls

Governance BoundaryTenant RequestGovLLM GatewayEmbedding ServiceTurboVec \(SPCS\)LLM \(Cortex API\)Response \+ Cost Recordauth \+ rate limitCembedC\_\{\\text\{embed\}\}loggedCretrieveC\_\{\\text\{retrieve\}\}loggedCgenerateC\_\{\\text\{generate\}\}loggedFigure 1:Cost\-Governed RAG architecture\. Each layer emits per\-tenant cost telemetry into a unified observability table within the platform governance boundary\.
### III\-BPer\-Tenant Cost Attribution Model

For tenantttmaking queryqq, the total attributed cost is:

C​\(t,q\)=Cembed​\(q\)\+Cretrieve​\(t,q\)\+Cgenerate​\(q\)C\(t,q\)=C\_\{\\text\{embed\}\}\(q\)\+C\_\{\\text\{retrieve\}\}\(t,q\)\+C\_\{\\text\{generate\}\}\(q\)\(2\)
Where each component is defined as:

Cembed​\(q\)\\displaystyle C\_\{\\text\{embed\}\}\(q\)=tokens​\(q\)×rembed\\displaystyle=\\text\{tokens\}\(q\)\\times r\_\{\\text\{embed\}\}\(3\)Cretrieve​\(t,q\)\\displaystyle C\_\{\\text\{retrieve\}\}\(t,q\)=MtMtotal×rmem⏟memory share\+blocks\_scanned​\(t,q\)×rcpu⏟compute\\displaystyle=\\underbrace\{\\frac\{M\_\{t\}\}\{M\_\{\\text\{total\}\}\}\\times r\_\{\\text\{mem\}\}\}\_\{\\text\{memory share\}\}\+\\underbrace\{\\text\{blocks\\\_scanned\}\(t,q\)\\times r\_\{\\text\{cpu\}\}\}\_\{\\text\{compute\}\}\(4\)Cgenerate​\(q\)\\displaystyle C\_\{\\text\{generate\}\}\(q\)=\(Tin\+Tout\)×rmodel\\displaystyle=\(T\_\{\\text\{in\}\}\+T\_\{\\text\{out\}\}\)\\times r\_\{\\text\{model\}\}\(5\)
The retrieval cost decomposes into two attributable components:

- •Memory share:Tenanttt’s fraction of total index memory, computed from \([1](https://arxiv.org/html/2607.12188#S2.E1)\)\. Because TurboVec’s memory is deterministic and linear inntn\_\{t\}, this fraction has bounded error arising only from the amortized shared rotation matrix \(<<0\.1% at 10M vectors\)\.
- •Compute share:The number of 32\-vector SIMD blocks actually scanned for tenanttt’s query\. With kernel\-level allowlist filtering, only the tenant’s own blocks are scored, making compute attribution precise\.

Key insight:TurboVec’s codebook\-oblivious design makes retrieval cost attribution*near\-exact*\(99\.88%\+ accuracy\) because: \(1\) memory is deterministic per tenant with only a small shared rotation matrix to amortize, and \(2\) kernel\-level filtering ensures compute is physically isolated per query\. This combination is, to our knowledge, unavailable in HNSW or PQ\-based indexes where shared graph edges or trained codebook state resist clean per\-tenant decomposition\.

### III\-CTelemetry Pipeline

Each TurboVec search emits a structured telemetry record:

```
{tenant_id, query_id, timestamp,
 vectors_scanned, blocks_skipped,
 latency_us, k_returned,
 index_memory_bytes_tenant,
 bit_width, compression_ratio}
```

This record is joined with the GovLLM generation trace \(tokens consumed, model used, cost\) in the platform’s unified telemetry table, producing a complete per\-query cost record across all three layers\. Standard SQL queries against this table power cost dashboards, anomaly detection, and chargeback reports\.

### III\-DCost Dashboard Integration

The unified telemetry table enables SQL\-native cost analytics:

```
SELECT tenant_id,
  SUM(embed_cost + retrieve_cost
      + generate_cost) as total_cost,
  SUM(retrieve_cost)/SUM(total_cost)
      as retrieval_pct
FROM cost_telemetry
WHERE ts >= DATEADD(’day’, -30, CURRENT_DATE)
GROUP BY tenant_id ORDER BY total_cost DESC;
```

This query is not straightforward with external vector databases \(Pinecone, Qdrant\) because they do not, to our knowledge, expose per\-tenant memory utilization APIs that feed directly into the governance layer’s tables\.

## IVEvaluation

### IV\-AExperimental Setup

We simulate a 100\-tenant deployment with the following configuration:

- •Total corpus:10M vectors,d=1536d=1536\(OpenAI text\-embedding\-3\-large withdimensions=1536\)
- •Tenant distribution:Log\-normal \(μ=11\.5\\mu=11\.5,σ=1\.0\\sigma=1\.0\), ranging from∼\\sim10K to∼\\sim500K vectors per tenant
- •Query load:1,000 QPS aggregate, distributed proportionally to tenant size
- •Quantization:TurboVec 4\-bit \(8×\\timescompression\)
- •Compute:SPCSCPU\_X64\_Sinstance family \(1 node\) for benchmarks at 100K scale

### IV\-BCost Attribution Accuracy

We compare attributed cost \(from telemetry records\) against actual resource consumption measured at the infrastructure level\. Attribution accuracy for each tenantiiis defined asAi=1−\|C^i−Ci\|/CiA\_\{i\}=1\-\|\\hat\{C\}\_\{i\}\-C\_\{i\}\|/C\_\{i\}, whereC^i\\hat\{C\}\_\{i\}is the telemetry\-derived attributed cost andCiC\_\{i\}is the ground\-truth cost computed from direct memory measurement and CPU cycle counters\. The reported end\-to\-end figure is computed as follows: for each of the 10,000 simulated queries, we compute per\-query attribution error; we then average across queries within each tenant to obtainAiA\_\{i\}; finally, we report the mean ofAiA\_\{i\}across all 100 tenants\. Because the simulation uses a fixed random seed \(42\) and all index operations are deterministic \(flat\-scan with no stochastic components\), repeated runs produce identical results—variance arises only from the tenant size distribution, not from algorithmic randomness\.

TABLE II:Cost Attribution Accuracy Across Layers \(100 Tenants, 10M Vectors, Log\-Normal Distribution\)\. Accuracy = mean of1−\|C^i−Ci\|/Ci1\-\|\\hat\{C\}\_\{i\}\-C\_\{i\}\|/C\_\{i\}across tenants; max error = worst\-case single\-tenant deviation\.The 0\.12% retrieval attribution error arises solely from the shared rotation matrix term in \([1](https://arxiv.org/html/2607.12188#S2.E1)\): atd=1536d=1536, the matrix occupiesd2×4=9\.0d^\{2\}\\times 4=9\.0MB, which is amortized uniformly across tenants rather than attributed proportionally to corpus size\. All other terms \(codes, norms, IDs, block metadata\) are exactly linear inntn\_\{t\}and thus perfectly attributable\. At 10M total vectors, the shared 9\.0 MB represents<<0\.12% of the total 7\.3 GiB index, producing the observed attribution gap\.

### IV\-CMemory Cost Reduction

TABLE III:Monthly Retrieval Infrastructure Cost Comparison \(10M Vectors,d=1536d=1536, 1000 QPS\)\. Memory derived from Eq\.[1](https://arxiv.org/html/2607.12188#S2.E1)\.The “Attribution” column highlights a key differentiator: managed vector databases do not, to our knowledge, provide APIs for per\-tenant memory consumption reporting, making cost governance difficult at the retrieval layer\. Self\-hosted FP32 indexes can theoretically be attributed \(memory is linear in vector count\) but lack the compression benefits\.

Pricing assumptions\.TurboVec SPCS cost assumes a singleCPU\_X64\_Snode at Snowflake’s published SPCS credit rate \(us\-west\-2, June 2026\)\. Pinecone estimate assumes the Serverless plan at 10M stored vectors with 1,000 reads/sec \(no writes\); Qdrant estimate assumes a dedicated cluster sized for 10M vectors atd=1536d=1536in AWS us\-east\-1\. All estimates exclude replication, backups, and availability SLAs\. These figures are illustrative scenarios, not rigorous iso\-quality benchmarks\.

### IV\-DSPCS Deployment Performance

TABLE IV:TurboVec SPCS Service Performance \(100K Vectors,d=1536d=1536, 4\-bit,CPU\_X64\_SInstance Family\)Filtered search is*faster*than unfiltered because the kernel short\-circuits SIMD blocks with no allowed tenant vectors\. Telemetry record creation adds 0\.005 ms per query—negligible relative to search latency—enabling per\-query cost attribution without measurable performance degradation\.

### IV\-EComparison Against Alternative Index Architectures

To contextualize TurboVec’s governance advantage, we benchmark alternative FAISS index types at 999K vectors \(d=1536d=1536\) from the same DBpedia dataset:

TABLE V:Index Architecture Comparison at 999K Vectors: Recall, Latency, Memory, and Cost Attribution PropertiesMethodR@5ms/qMemTrainAttrib\.TurboQuant 4\-bit0\.96817\.1767 MBNoneExactTurboQuant 2\-bit0\.9019\.1384 MBNoneExactFAISS PQ 4\-bit0\.883154384 MB61sAmbiguousFAISS PQ 8\-bit0\.961222767 MB132sAmbiguousFAISS IVF\-PQ0\.8405\.7390 MB35sAmbiguousHNSW\-Flat∗0\.9840\.53200 MB758sImpossible∗HNSW measured at 500K \(infeasible at 999K on 16 GB test machine\)\.HNSW achieves the highest recall \(0\.984\) but its graph\-based memory is non\-decomposable per tenant—edges cross tenant boundaries, making exact cost attribution structurally impossible without physical partitioning\. We note that the HNSW comparison is at 500K rather than 999K due to hardware memory constraints \(\>\>6\.4 GiB required for HNSW\-Flat at 999K on our 16 GiB test machine\); at 999K, HNSW memory would be approximately 6\.4 GiB, further widening the memory gap versus TurboVec’s 767 MB\. IVF\-PQ and flat PQ share trained codebook state across tenants, creating attribution ambiguity for the codebook memory component\. Only TurboVec’s codebook\-oblivious, flat\-scan architecture supports exact per\-tenant memory decomposition via \([1](https://arxiv.org/html/2607.12188#S2.E1)\)\.

### IV\-FTenant Scaling Analysis

To understand cost attribution behavior as tenant count varies, we measure attribution accuracy acrossT∈\{10,50,100,500,1000\}T\\in\\\{10,50,100,500,1000\\\}tenants \(10M vectors, fixed total\):

TABLE VI:Cost Attribution Accuracy vs\. Tenant Count \(10M Vectors Total, TurboVec 4\-bit\)Attribution accuracy degrades gracefully as tenant count increases because the fixed overhead \(OfixedO\_\{\\text\{fixed\}\}\) is amortized over more tenants with smaller per\-tenant allocations\. Even at 1000 tenants, attribution remains above 99\.5%—well within acceptable FinOps tolerance\[[21](https://arxiv.org/html/2607.12188#bib.bib21)\]\.

### IV\-GCost Breakdown Analysis

We analyze the relative contribution of each cost layer across tenant sizes:

TABLE VII:Cost Layer Breakdown by Tenant Size \(1000 QPS Total, Claude 3\.5 Sonnet Generation\)For large tenants, retrieval constitutes up to 28% of total RAG cost—far from negligible\. Without retrieval\-layer attribution, these tenants would be cross\-subsidized by smaller tenants under flat\-rate pricing\.

## VDiscussion

Why cloud\-native deployment matters for governance\.External vector databases \(Pinecone, Qdrant, Weaviate\) cannot export per\-tenant memory utilization back to the enterprise governance layer\. By deploying TurboVec within SPCS, retrieval telemetry writes to the same Snowflake tables as generation telemetry—enabling unified SQL\-based cost dashboards, anomaly alerts, and automated chargeback without cross\-platform data movement or API polling\.

Codebook\-oblivious quantization as a governance enabler\.The connection between codebook design and cost governance is under\-explored\. TurboVec’s codebook contains no corpus\-dependent state, which simultaneously: \(a\) removes the shared\-codebook leakage surface present in trained PQ codebooks\[[3](https://arxiv.org/html/2607.12188#bib.bib3)\], and \(b\) makes per\-tenant memory deterministically attributable \(no shared learned state to amortize\)\. This dual benefit suggests that leakage\-resistant system design can be synergistic with—rather than in tension with—governance requirements\. We note that this observation is limited to the codebook surface; it does not constitute end\-to\-end privacy guarantees, which would require a formal threat model and empirical evaluation beyond the scope of this paper\.

Sustainability implications\.At enterprise scale \(100M\+ vectors\), the 8×\\timesmemory reduction from 4\-bit quantization translates directly to fewer compute nodes\. Concretely: 100M vectors at FP32 require∼\\sim572 GiB of index memory, necessitating∼\\sim5 r6i\.4xlarge instances \(128 GiB RAM each, at 90% utilization\)\. At 4\-bit, the same corpus fits in∼\\sim112 GiB, requiring∼\\sim1 instance—a reduction of 4 nodes\. At∼\\sim0\.25 kW per instance and US average grid carbon intensity of 0\.39 kg CO2/kWh\[[23](https://arxiv.org/html/2607.12188#bib.bib23)\], the annual savings are approximately4×0\.25×8760×0\.39≈3\.44\\times 0\.25\\times 8760\\times 0\.39\\approx 3\.4tonnes CO2per 100M\-vector deployment—aligning enterprise RAG infrastructure with sustainability goals\[[22](https://arxiv.org/html/2607.12188#bib.bib22)\]\.

Integration with FinOps frameworks\.The three\-layer cost model maps directly to the FinOps Foundation’s FOCUS specification\[[21](https://arxiv.org/html/2607.12188#bib.bib21)\]: each telemetry record contains a resource identifier \(tenant\_id\), usage quantity \(vectors\_scanned, tokens\), and unit rate—enabling standard FinOps tooling \(CloudHealth, Apptio\) to process RAG costs alongside traditional cloud spend\.

## VIRelated Work

LLM cost optimization\.UCCI\[[5](https://arxiv.org/html/2607.12188#bib.bib5)\]routes queries to cheaper models via cascade policies\. ProCut\[[6](https://arxiv.org/html/2607.12188#bib.bib6)\]compresses prompts to reduce input token costs\. FrugalGPT\[[7](https://arxiv.org/html/2607.12188#bib.bib7)\]combines model selection, caching, and query adaptation\. These operate exclusively at the generation layer; to our knowledge, none address retrieval cost attribution\.

RAG system optimization\.RAG\-Stack\[[8](https://arxiv.org/html/2607.12188#bib.bib8)\]co\-optimizes retrieval quality and throughput but does not formalize cost per tenant\. RAGO\[[9](https://arxiv.org/html/2607.12188#bib.bib9)\]provides systematic RAG serving performance optimization\. HyperRAG\[[10](https://arxiv.org/html/2607.12188#bib.bib10)\]addresses distributed retrieval scaling\. To our knowledge, none provide per\-tenant cost observability\.

Vector quantization for ANN\.Product Quantization\[[11](https://arxiv.org/html/2607.12188#bib.bib11)\]and OPQ\[[12](https://arxiv.org/html/2607.12188#bib.bib12)\]achieve high compression via corpus\-dependent training\. TurboQuant\[[2](https://arxiv.org/html/2607.12188#bib.bib2)\]derives boundaries analytically without training\. RaBitQ\[[13](https://arxiv.org/html/2607.12188#bib.bib13)\]provides theoretical error bounds\. Our contribution is connecting quantization design choices to cost governance properties\.

Multi\-tenant AI systems\.OptiLeak\[[15](https://arxiv.org/html/2607.12188#bib.bib15)\]demonstrates prompt reconstruction risks in shared LLM KV\-caches\. Multi\-tenant database isolation is well\-studied\[[16](https://arxiv.org/html/2607.12188#bib.bib16)\], but multi\-tenant vector index isolation for both leakage reduction and cost attribution is, to our knowledge, novel\.

MLOps and AI observability\.Langfuse\[[17](https://arxiv.org/html/2607.12188#bib.bib17)\]and LangSmith\[[18](https://arxiv.org/html/2607.12188#bib.bib18)\]provide LLM tracing\. MLflow\[[20](https://arxiv.org/html/2607.12188#bib.bib20)\]tracks ML experiments\. Helicone\[[19](https://arxiv.org/html/2607.12188#bib.bib19)\]provides LLM cost analytics\. To our knowledge, none integrate retrieval\-layer cost attribution into the observability pipeline\.

FinOps for AI\.The FinOps Foundation’s FOCUS specification\[[21](https://arxiv.org/html/2607.12188#bib.bib21)\]standardizes cloud cost data but has no provisions for AI\-specific cost components \(embeddings, retrieval, generation\)\. Our three\-layer model extends FinOps principles to the RAG stack\.

## VIILimitations and Future Work

1. 1\.Simulated workload only\.The 100\-tenant evaluation uses synthetic log\-normal tenant distributions and proportional query loads\. Real enterprise workloads exhibit bursty, time\-varying patterns that may stress the attribution model differently\. Validation against anonymized production traces from a multi\-tenant RAG deployment is planned as immediate future work\.
2. 2\.CPU flat\-scan only\.TurboVec currently uses flat\-scan search\. Graph\-based indexes \(HNSW\) have non\-linear memory profiles that would require a different attribution formula\. Extending the cost model to graph\-augmented TurboVec is future work\.
3. 3\.Single pricing snapshot\.Cost comparisons use 2026 public pricing that may change\. The architecture is pricing\-model\-agnostic; only the rate constants \(rmemr\_\{\\text\{mem\}\},rcpur\_\{\\text\{cpu\}\},rmodelr\_\{\\text\{model\}\}\) require updating\.
4. 4\.No dynamic rebalancing\.The current model assumes static tenant\-to\-vector assignments\. Tenant growth, churn, and data migration would require periodic reconciliation of attribution baselines\.
5. 5\.Single embedding model\.All cost projections assumed=1536d=1536\(OpenAI text\-embedding\-3\-large with thedimensions=1536parameter; the model’s native output isd=3072d=3072\)\. Lower\-dimensional embeddings would reduce per\-tenant memory proportionally, potentially shifting the cost balance toward generation\-dominated workloads\.

Future Work\.Natural extensions include: \(a\) integration with graph\-based search layers and corresponding non\-linear attribution models; \(b\) real production deployment with actual chargeback validation; \(c\) cost\-aware query routing that dynamically selects bit\-width based on tenant SLA tier; and \(d\) extending the model to multimodal RAG \(image \+ text embeddings with heterogeneous dimensionality\)\.

## VIIIConclusion

Cost\-Governed RAG demonstrates that codebook\-oblivious vector quantization simultaneously removes the shared\-codebook leakage surface and enables deterministic per\-tenant cost attribution—bridging two communities \(retrieval systems and AI governance\) that have operated largely independently despite serving the same enterprise deployments\. Deployed on a cloud data platform’s container service, the architecture achieves 99\.96% end\-to\-end cost attribution accuracy while reducing retrieval infrastructure cost by 3\.1–9\.0×\\timescompared to managed alternatives under the pricing assumptions described\. For large tenants, retrieval constitutes up to 28% of total RAG cost—a governance blind spot that our unified observability stack makes visible\. As enterprise AI moves toward production\-grade multi\-tenant deployment, cost governance must evolve from token\-level metering to full\-stack attribution spanning every layer of the RAG pipeline\.

## Acknowledgment

Navnit Shukla conceived the Cost\-Governed RAG architecture, designed the three\-layer cost model, conducted all deployment experiments and cost analysis, and authored the complete manuscript\.

## References

- \[1\]P\. Lewis et al\., “Retrieval\-Augmented Generation for Knowledge\-Intensive NLP Tasks,” inProc\. NeurIPS, 2020\.
- \[2\]J\. Zhang et al\., “TurboQuant: Online Vector Quantization with Near\-optimal Distortion Rate,” inProc\. ICLR, 2026\.
- \[3\]N\. Shukla, “TurboVec: A Case Study in Cost\-Efficient Private Retrieval for Enterprise RAG via Codebook\-Oblivious Quantization,” inProc\. FLLM, 2026\.
- \[4\]N\. Shukla, “Building Multi\-Tenant AI Chat Applications with Cortex REST API,” Snowflake Developer Guide, 2026\.
- \[5\]V\. Kotte, “UCCI: Calibrated Uncertainty for Cost\-Optimal LLM Cascade Routing,” arXiv:2605\.18796, 2026\.
- \[6\]Y\. Xu et al\., “ProCut: LLM Prompt Compression via Attribution Estimation,” arXiv:2508\.02053, 2025\.
- \[7\]L\. Chen et al\., “FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance,” arXiv:2305\.05176, 2023\.
- \[8\]Y\. Chen et al\., “RAG\-Stack: Co\-Optimizing RAG Quality and Performance From the Vector Database Perspective,” arXiv:2510\.20296, 2025\.
- \[9\]Z\. Li et al\., “RAGO: Systematic Performance Optimization for Retrieval\-Augmented Generation Serving,” inProc\. ISCA, 2025\.
- \[10\]X\. Liu et al\., “Towards Hyper\-Efficient RAG Systems in VecDBs: Distributed Parallel Multi\-Resolution Vector Search,” arXiv:2511\.16681, 2025\.
- \[11\]H\. Jégou, M\. Douze, and C\. Schmid, “Product Quantization for Nearest Neighbor Search,”IEEE Trans\. PAMI, vol\. 33, no\. 1, pp\. 117–128, 2011\.
- \[12\]T\. Ge, K\. He, Q\. Ke, and J\. Sun, “Optimized Product Quantization for Approximate Nearest Neighbor Search,” inProc\. IEEE CVPR, 2013, pp\. 2946–2953\.
- \[13\]J\. Gao et al\., “RaBitQ: Quantizing High\-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search,” inProc\. SIGMOD, 2024\.
- \[14\]Y\. Malkov and D\. Yashunin, “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs,”IEEE Trans\. PAMI, vol\. 42, no\. 4, pp\. 824–836, 2020\.
- \[15\]Z\. Wang et al\., “OptiLeak: Efficient Prompt Reconstruction via Reinforcement Learning in Multi\-tenant LLM Services,” arXiv:2602\.20595, 2026\.
- \[16\]S\. Aulbach et al\., “Multi\-Tenant Databases for Software as a Service: Schema\-Mapping Techniques,” inProc\. ACM SIGMOD, 2008, pp\. 1195–1206\.
- \[17\]M\. Kilian et al\., “Langfuse: Open\-Source LLM Engineering Platform,” 2024\. \[Online\]\. Available:[https://langfuse\.com](https://langfuse.com/)
- \[18\]LangChain, “LangSmith: LLM Application Observability,” 2024\. \[Online\]\. Available:[https://smith\.langchain\.com](https://smith.langchain.com/)
- \[19\]Helicone, “LLM Observability for Developers,” 2025\. \[Online\]\. Available:[https://helicone\.ai](https://helicone.ai/)
- \[20\]M\. Zaharia et al\., “Accelerating the Machine Learning Lifecycle with MLflow,”IEEE Data Eng\. Bull\., vol\. 41, no\. 4, 2018\.
- \[21\]FinOps Foundation, “FOCUS: FinOps Open Cost and Usage Specification,” 2024\. \[Online\]\. Available:[https://focus\.finops\.org](https://focus.finops.org/)
- \[22\]R\. Schwartz et al\., “Green AI,”Communications of the ACM, vol\. 63, no\. 12, pp\. 54–63, 2020\.
- \[23\]US EPA, “eGRID Summary Tables 2022,” 2024\. \[Online\]\. Available:[https://www\.epa\.gov/egrid](https://www.epa.gov/egrid)