RAG-Stack: Co-Optimizing RAG Serving Performance and Quality

# RAG-Stack: Co-Optimizing RAG Serving Performance and Quality
Source: [https://arxiv.org/html/2608.03487](https://arxiv.org/html/2608.03487)
###### Abstract\.

Retrieval\-augmented generation \(RAG\), which augments LLM generation with information retrieved from databases, has become a widely used approach for knowledge\-intensive applications\. Modern RAG systems, however, expose many configuration choices, such as retrieval indexes, model selections, and how models invoke retrievals\. Each configuration yields a different trade\-off between answer quality and serving performance, making it challenging to choose the optimal setting for a specific application deployment\. In this paper, we presentRAG\-Stack, a framework for efficiently discovering quality–performance Pareto frontiers across diverse RAG applications and serving systems\. Specifically, it consists ofRAG\-PE, an iterative design\-space exploration algorithm that selects the next RAG configuration to evaluate;RAG\-IR, a workload abstraction for diverse RAG algorithms; andRAG\-CM, a performance model that predicts the optimal deployment and serving performance on the given hardware\. Together, these components allowRAG\-Stackto search the joint algorithm–system configuration space without deploying every candidate and to transfer an existing Pareto frontier to a new serving system\. Given the same number of optimization iterations across diverse datasets, the Pareto frontiers found byRAG\-Stackcover 52\.5–153\.2% more of the normalized quality–performance space than those found by state\-of\-the\-art configuration\-search methods evaluated over the same RAG design space\.

††copyright:none## 1\.Introduction

Retrieval\-augmented generation \(RAG\) augments large language models \(LLMs\) with information retrieved from external data sources\(Vuet al\.,[2024](https://arxiv.org/html/2608.03487#bib.bib103); Lewiset al\.,[2020](https://arxiv.org/html/2608.03487#bib.bib60); Gaoet al\.,[2024](https://arxiv.org/html/2608.03487#bib.bib35)\)\. By combining models and databases, RAG addresses several limitations of model\-only systems\. First, RAG can incorporate up\-to\-date information that may not be encoded in a model’s parameters\(Vuet al\.,[2024](https://arxiv.org/html/2608.03487#bib.bib103)\)\. Second, by grounding generation in retrieved evidence, RAG can improve factual accuracy and reduce hallucinations\(Shusteret al\.,[2021](https://arxiv.org/html/2608.03487#bib.bib91)\)\. Third, RAG enables models to use private or domain\-specific corpora that were unavailable during pretraining\(Zhanget al\.,[2024](https://arxiv.org/html/2608.03487#bib.bib117); Siriwardhanaet al\.,[2023](https://arxiv.org/html/2608.03487#bib.bib95)\)\. Together, these capabilities have made RAG a widely adopted approach for knowledge\-intensive applications\(Zhanget al\.,[2024](https://arxiv.org/html/2608.03487#bib.bib117); Zhuet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib120); Chenet al\.,[2024a](https://arxiv.org/html/2608.03487#bib.bib22)\)\.

![Refer to caption](https://arxiv.org/html/2608.03487v1/x1.png)Figure 1\.Modern RAG system comprises diverse model components and execution workflows\.Although the core idea of RAG is simple, modern RAG systems expose a large design space spanning both \(1\) pipeline components and \(2\) execution workflows \(Figure[1](https://arxiv.org/html/2608.03487#S1.F1)\), inducing a broad spectrum of trade\-offs between answer quality and serving performance\. At the component level, a RAG pipeline contains a retrieval system and a generative LLM and may additionally include stages such as query rewriting and result reranking\. Each RAG component can introduce a unique choice of model, prompt, and stage\-specific parameters\. At the workflow level, these components can be orchestrated either as a*\(a\) sequential pipeline*, in which every query follows a fixed execution order, or as an*\(b\) agentic pipeline*, in which an LLM\-based controller dynamically decides when to retrieve and whether additional retrieval rounds are needed\(Liet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib61)\)\. Naively enabling every pipeline component, selecting the most capable models, or maximizing the number of retrieval rounds may improve answer quality, but doing so also increases serving cost and reduces system performance \(worse latency and throughput\)\. In this paper, we ask:How can we efficiently explore this quality–performance trade\-off space and identify the Pareto frontier across diverse RAG applications and serving systems?

However, identifying the Pareto frontier for a given RAG application and serving system requires solving three key problems\.\(P1\) Existing RAG configuration exploration algorithms either neglect cross\-stage interactions or overlook per\-stage signals\.One approach to explore RAG configurations is to proceed stage by stage: it identifies the best configuration for one stage, holds that configuration fixed, and then optimizes the next stage\([51](https://arxiv.org/html/2608.03487#bib.bib73)\)\. However, the best end\-to\-end configuration depends on interactions among stages and therefore cannot generally be obtained by optimizing each stage independently\. For example, increasing the retrieval top\-kkcan improve quality when a strong reranker filters irrelevant passages or a long\-context LLM effectively uses the additional evidence, but can degrade quality when noisy passages reach a weaker generator without reranking\. By contrast, global RAG optimization captures these interactions by jointly tuning parameters across all stages using end\-to\-end accuracy as the optimization signal\(Barkeret al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib15)\)\. However, this end\-to\-end signal does not reveal which stage is responsible for a gain or loss, potentially leading to poorly directed exploration and wasted evaluations\.\(P2\) Existing multi\-objective RAG optimizers overlook the system design space\.Existing quality–performance optimizers search over algorithmic configurations while holding the serving deployment fixed\(Barkeret al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib15); Rayet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib85)\), even though different algorithms may require different batching policies, parallelization strategies, and stage placements to achieve their best serving performance\(Jianget al\.,[2025a](https://arxiv.org/html/2608.03487#bib.bib47)\)\.\(P3\) Deployment\-based serving\-performance measurements are costly and do not transfer across systems\.Measuring serving performance requires deploying and benchmarking each candidate configuration, and the resulting measurements are specific to the target system\(Liet al\.,[2026](https://arxiv.org/html/2608.03487#bib.bib68)\)\. When the application is moved to a system with different hardware, configurations on the original Pareto frontier may no longer be Pareto\-optimal, requiring the search to be repeated to reconstruct the frontier\(Luet al\.,[2021](https://arxiv.org/html/2608.03487#bib.bib71); Jamshidiet al\.,[2018](https://arxiv.org/html/2608.03487#bib.bib43)\)\.

![Refer to caption](https://arxiv.org/html/2608.03487v1/x2.png)Figure 2\.Overview ofRAG\-Stackand its two operating modes\.To address these problems, we presentRAG\-Stack, an efficient framework for discovering the quality–performance Pareto frontier across arbitrary RAG applications and serving systems \(Figure[2](https://arxiv.org/html/2608.03487#S1.F2)\)\.RAG\-Stacktakes four inputs from the user: \(1\) an algorithm design space, \(2\) a new RAG application \(i\.e\., an evaluation dataset\), \(3\) an optimization target comprising quality and performance requirements, and \(4\) the available system resources\.RAG\-Stackcan either discover a Pareto frontier from scratch or transfer an existing frontier to a new system by reusing archived quality measurements\.

For a new RAG application,RAG\-Stackdiscovers the Pareto frontier with an iterative optimization loop\. First,RAG\-PEproposes an algorithm configuration for quality evaluation on the application dataset\. Next,RAG\-IRabstracts the executed workflow, andRAG\-CMpredicts its best achievable serving performance under the available resources without requiring deployment on the target system\. Finally,RAG\-PEuses the measured quality and predicted performance to select the next configuration\. This loop continues until a user\-specified stopping criterion or search budget is reached\. We now introduce the main components inRAG\-Stack\.

First,RAG\-PE\(PlanExploration\) is the core multi\-objective Bayesian optimizer inRAG\-Stack\. It takes as input the algorithm design space and feedback from previous trials, including quality measurements on the dataset and performance estimates produced byRAG\-CM\. Its output is the next algorithm configuration to evaluate, which is sent through the RAG execution path and subsequently represented byRAG\-IR\.RAG\-PEaddressesP1by jointly optimizing end\-to\-end quality and serving performance while using intermediate stage\-level quality signals to guide exploration\.

Second,RAG\-IR\(IntermediateRepresentation\) bridgesRAG\-PEandRAG\-CMby translating each RAG algorithm configuration into a workload representation comprising a workflow schema and an execution trace of stage invocations and input/output sizes\. The output ofRAG\-IRis a*system\-agnostic*workload representation that decouples each stage’s logical work from its physical deployment, enablingRAG\-CMto explore deployment choices and re\-estimate the performance of the same workload on new hardware\. Together withRAG\-CM, this representation addressesP2andP3\.

Third,RAG\-CM\(CostModel\) is an ML–analytical fusion performance model\. It takes as input the workload representation produced byRAG\-IR, the system design space, and the user’s available hardware resources\. It then searches the system design space internally and returns the best predicted deployment and serving performance for the current RAG configuration to RAG\-PE, thus effectively addressingP2\. Moreover, RAG\-CM can re\-evaluate the archived algorithm configurations under a new hardware configuration without repeating their quality evaluations, enabling efficient frontier transfer and further addressingP3\.

We evaluateRAG\-Stackon RAGEval\(Zhuet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib120)\)and MS MARCO\(Bajajet al\.,[2018](https://arxiv.org/html/2608.03487#bib.bib12)\)using two system configurations: one equipped with four NVIDIA H100 GPUs and the other with eight NVIDIA A100 GPUs\. With the same number of exploration iterations, the Pareto frontiers found byRAG\-Stack, averaged across seeds, cover 52\.5% and 153\.2% more of the normalized quality–performance space on RAGEval and MS MARCO, respectively, than those found by state\-of\-the\-art configuration\-search methods evaluated over the same RAG design space\. When transferring a Pareto frontier to a new serving system,RAG\-Stackreuses quality measurements from previous evaluations and performs only a small number of additional optimization iterations to discover the new frontier\. The resulting adapted frontier covers 182\.2% more of the normalized quality–performance space than the frontier obtained by re\-optimizing from scratch on the new system\.

In summary, the paper makes the followingcontributions:

- •We presentRAG\-Stack, an end\-to\-end framework that efficiently discovers the Pareto frontier between RAG answer quality and serving performance for arbitrary applications and systems\.
- •We designRAG\-PE, a multi\-objective, sub\-metric\-aware Bayesian optimizer that uses both end\-to\-end and intermediate signals to efficiently navigate the RAG configuration space\.
- •We introduceRAG\-CM, a hybrid ML\-analytical performance model that predicts serving performance and searches optimal deployment configurations on the given hardware\.

## 2\.Background and Motivation

### 2\.1\.Vector Search for Retrieval

RAG retrievers use*vector search*to match queries with passages based on semantic similarity rather than exact lexical overlap\. Before serving, corpus passages are embedded and indexed\. At query time, the retriever embeds the query and returns the nearest indexed passages\. Exact search scans the entire corpus and guarantees the true nearest neighbors under the chosen metric, but scales poorly\. Production systems therefore rely on*approximate nearest\-neighbor*\(ANN\) indexes, which prune most candidates to reduce latency and increase throughput, but may lower recall and consequently degrade answer quality when relevant evidence is missed\.

Two index families\.ANN indexes are either clustering\-based or graph\-based\. The*IVF \(inverted\-file\) family*is clustering\-based: it groups the vectors into many lists, and for each query scans only the few lists closest to it\. Within this family, IVF\-Flat keeps the full vectors and favors recall; IVF\-PQ compresses each vector into a short code to save memory and bandwidth; and IVF\-PQ FastScan speeds up the distance computation with SIMD\-friendly kernels\(Jégouet al\.,[2011](https://arxiv.org/html/2608.03487#bib.bib44); Douzeet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib29)\)\.*HNSW*is graph\-based: it links the vectors into a navigable graph and answers a query by walking the graph greedily toward the nearest ones\. HNSW often reaches high recall at low latency, but it needs extra memory to store the graph\(Malkov and Yashunin,[2018](https://arxiv.org/html/2608.03487#bib.bib72)\)\.

Knobs trade recall for speed\.Largernprobein IVF orefSearchin HNSW examines more candidates, improving recall at the cost of latency and throughput; IVF\-PQ reduces memory use but may lower recall\. Because the best choice depends on the workload and deployment, FAISS is well suited to studying these trade\-offs: it directly exposes both index families and their CPU/GPU knobs\(Douzeet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib29)\), unlike vector databases such as Milvus, whose serving layer hides many low\-level choices\(Wanget al\.,[2021](https://arxiv.org/html/2608.03487#bib.bib105)\)\.

### 2\.2\.RAG Pipelines and Serving

A RAG pipeline is built from a few stages\. The two core stages are retrieval \(§[2\.1](https://arxiv.org/html/2608.03487#S2.SS1)\) and generation: the retriever runs a vector search over the corpus, and the retrieved passages—optionally after query rewriting or reranking—are placed in a prompt for an LLM that prefills and decodes\. The same stages can be wired into different*workflows*\. The simplest pipeline retrieves once and generates once\. Iterative and active\-retrieval workflows interleave several rounds of retrieval and generation to improve the answer\(Asaiet al\.,[2023b](https://arxiv.org/html/2608.03487#bib.bib6)\)\. An agentic pipeline goes further: an LLM controller decides at run time which stages to run, when to retrieve, and whether to iterate\(Singhet al\.,[2026](https://arxiv.org/html/2608.03487#bib.bib94); Liet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib61); Bestaet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib17)\)\. Choosing the components and the workflow is already a quality\-tuning problem, and frameworks such as FlashRAG let users assemble, swap, and evaluate these pipelines to raise answer quality\(Jinet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib49)\)\.

The choices that raise quality also raise the work per request\. A larger top\-kk, an added reranker, iterative retrieval, or a bigger generator each helps the answer, but each also adds retrieval work, model calls, prompt tokens, or decoding time\. Serving then adds a second layer of choices that leave the answer unchanged but set how fast it is produced: how the stages are placed on the hardware, how much parallelism they use, how requests are batched, and how the index and runtime knobs are configured\.

A large body of systems work optimizes this serving layer for a fixed RAG algorithm—tensor\(Shoeybiet al\.,[2020](https://arxiv.org/html/2608.03487#bib.bib90)\), pipeline\(Huanget al\.,[2019](https://arxiv.org/html/2608.03487#bib.bib41); Harlapet al\.,[2018](https://arxiv.org/html/2608.03487#bib.bib40)\), and data\(Liet al\.,[2020](https://arxiv.org/html/2608.03487#bib.bib67)\)parallelism, continuous batching\(Yuet al\.,[2022](https://arxiv.org/html/2608.03487#bib.bib112)\), and paged KV\-cache management\(Kwonet al\.,[2023](https://arxiv.org/html/2608.03487#bib.bib57)\)—together with RAG\-specific techniques such as adaptive pipeline parallelism\(Jianget al\.,[2025c](https://arxiv.org/html/2608.03487#bib.bib46)\)and disaggregated accelerators\(Jianget al\.,[2025b](https://arxiv.org/html/2608.03487#bib.bib45)\)\.

### 2\.3\.Motivation: Performance and Quality Co\-optimization for RAG

![Refer to caption](https://arxiv.org/html/2608.03487v1/x3.png)Figure 3\.Median quality–throughput tension across algorithm choices\. \(a\) By chunk size\. \(b\) Single\-choice contrasts \(quality: first−\-second; QPS: first//second\); error bars: interquartile range\.The discussion above of RAG pipelines and serving shows that co\-optimizing answer quality and serving performance better reflects the practical needs of RAG serving\. This view also accounts for RAG serving\-system design, because placement, batching, and parallelism determine how efficiently each pipeline runs\. Manual co\-optimization is difficult because measurements reveal trade\-offs that intuition misses \(Figure[3](https://arxiv.org/html/2608.03487#S2.F3)\): agentic ReAct\(Yaoet al\.,[2023](https://arxiv.org/html/2608.03487#bib.bib111)\)can be faster than a sequential pipeline at a modest quality cost, query expansion sacrifices throughput for a quality gain that appears only in some configurations, and even a larger generator does not always improve quality\(Zhanget al\.,[2026](https://arxiv.org/html/2608.03487#bib.bib118)\)\. A configuration that avoids such unrewarded cost can deliver the same quality at higher throughput and thus*dominate*alternatives; only non\-dominated configurations are worth serving\.

Finding the frontier is a search problem\.Let𝒳\\mathcal\{X\}denote a candidate space and𝐲​\(𝐱\)∈ℝm\\mathbf\{y\}\(\\mathbf\{x\}\)\\in\\mathbb\{R\}^\{m\}the objective vector of𝐱∈𝒳\\mathbf\{x\}\\in\\mathcal\{X\}, with all objectives oriented so that larger values are better\. An objective point𝐲\\mathbf\{y\}*dominates*𝐲′\\mathbf\{y\}^\{\\prime\}if𝐲≥𝐲′\\mathbf\{y\}\\geq\\mathbf\{y\}^\{\\prime\}componentwise with at least one strict inequality\(Debet al\.,[2002](https://arxiv.org/html/2608.03487#bib.bib28)\)\. A candidate is*non\-dominated*if no other candidate’s objective point dominates its own; the objective points of all non\-dominated candidates form the*Pareto frontier*ℱ\\mathcal\{F\}\. The*hypervolume*HV​\(ℱ;𝐫\)\\mathrm\{HV\}\(\\mathcal\{F\};\\mathbf\{r\}\)is the volume of the objective\-space region that is dominated byℱ\\mathcal\{F\}and dominates a fixed*reference point*𝐫∈ℝm\\mathbf\{r\}\\in\\mathbb\{R\}^\{m\}that lower\-boundsℱ\\mathcal\{F\}; a larger hypervolume indicates broader coverage of desirable trade\-offs\(Daultonet al\.,[2021](https://arxiv.org/html/2608.03487#bib.bib27)\)\. Throughout, objectives are min–max normalized per experiment and dataset, with𝐫=𝟎\\mathbf\{r\}=\\mathbf\{0\}\. The frontier thus provides a set of operating points from which users can choose\. A general configuration\-search method finds it iteratively: it proposes a candidate, evaluates its objective vector, and uses the observations collected so far to select the next candidate\. Because this search requires only𝒳\\mathcal\{X\}and the evaluated objective values, it can in principle use any multi\-objective optimization method\. The next section shows why such methods, applied directly to RAG, fall short\.

### 2\.4\.Limitations of Existing Approaches

Existing RAG\-specific optimizers cover only parts of the quality–performance problem\. A RAG configuration spans two design spaces: the*algorithm design space*, whose choices affect answer quality—chunking, top\-kk, reranking, and the generator—and the*system design space*, whose choices affect how fast that answer is served—placement, batching, parallelism, and hardware\. Quality\-side tools search the algorithm design space against answer quality alone\([51](https://arxiv.org/html/2608.03487#bib.bib73);[J\. Jin, Y\. Zhu, G\. Dong, Y\. Zhang, X\. Yang, C\. Zhang, T\. Zhao, Z\. Yang, Z\. Dou, and J\. Wen \(2025\)](https://arxiv.org/html/2608.03487#bib.bib49);[J\. Fu, X\. Qin, F\. Yang, L\. Wang, J\. Zhang, Q\. Lin, Y\. Chen, D\. Zhang, S\. Rajmohan, and Q\. Zhang \(2024\)](https://arxiv.org/html/2608.03487#bib.bib33)\)\. System\-side tools such as RAGO search the system design space for serving performance, but only for a fixed RAG algorithm\(Jianget al\.,[2025a](https://arxiv.org/html/2608.03487#bib.bib47)\)\. Online methods such as METIS adapt a small set of serving choices per query, but they also operate within a fixed deployment\(Rayet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib85)\)\.

A natural alternative is to treat RAG tuning as a generic multi\-objective black\-box optimization problem\. Recent work takes this route by applying standard multi\-objective Bayesian optimization, including LogNEHVI, to search RAG hyperparameters for quality–cost trade\-offs\(Barkeret al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib15)\)\. More broadly, possible black\-box optimizers include plain sampling, Bayesian optimization\(Lindaueret al\.,[2022](https://arxiv.org/html/2608.03487#bib.bib66); Watanabe,[2026](https://arxiv.org/html/2608.03487#bib.bib106); Olsonet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib76); Balandatet al\.,[2020](https://arxiv.org/html/2608.03487#bib.bib13); Lévesqueet al\.,[2017](https://arxiv.org/html/2608.03487#bib.bib59)\), evolutionary search\(Hansen,[2023](https://arxiv.org/html/2608.03487#bib.bib39); Beyer and Schwefel,[2002](https://arxiv.org/html/2608.03487#bib.bib18)\), and multi\-fidelity optimization with cheaper proxy runs\(Kandasamyet al\.,[2017](https://arxiv.org/html/2608.03487#bib.bib51)\)\. These optimizers can also be steered by LLMs\([J\. Lao, Y\. Wang, Y\. Li, J\. Wang, Y\. Zhang, Z\. Cheng, W\. Chen, M\. Tang, and J\. Wang \(2025\)](https://arxiv.org/html/2608.03487#bib.bib58);[A\. Novikov, N\. Vũ, M\. Eisenberger, E\. Dupont, P\. Huang, A\. Z\. Wagner, S\. Shirobokov, B\. Kozlovskii, F\. J\. R\. Ruiz, A\. Mehrabian, M\. P\. Kumar, A\. See, S\. Chaudhuri, G\. Holland, A\. Davies, S\. Nowozin, P\. Kohli, and M\. Balog \(2025\)](https://arxiv.org/html/2608.03487#bib.bib75);[1](https://arxiv.org/html/2608.03487#bib.bib2)\)or extended to the multi\-objective setting through scalarization\(Knowles,[2006](https://arxiv.org/html/2608.03487#bib.bib54); Pariaet al\.,[2019](https://arxiv.org/html/2608.03487#bib.bib80)\), dominance\(Debet al\.,[2002](https://arxiv.org/html/2608.03487#bib.bib28); Zhang and Li,[2007](https://arxiv.org/html/2608.03487#bib.bib116)\), or hypervolume criteria\(Daultonet al\.,[2021](https://arxiv.org/html/2608.03487#bib.bib27); Amentet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib4)\)\. These general tools are our starting point\. Applied directly to RAG, however, they and the RAG\-specific optimizers above run into three problems\.

![Refer to caption](https://arxiv.org/html/2608.03487v1/x4.png)Figure 4\.Cross\-stage interactions in the algorithm design space\. Each arrow traces one configuration as retrieval top\-kkrises; annotated percentages are the resulting quality change\.\(P1\)Existing RAG optimizers either break cross\-stage interactions or lose stage\-level signals\.Figure[4](https://arxiv.org/html/2608.03487#S2.F4)shows that increasing retrieval top\-kkalways reduces throughput, but its quality effect flips with chunk size and compression, demonstrating that stage choices cannot be optimized independently\. Stage\-wise optimizers such as AutoRAG retain stage\-level feedback but miss configurations that work only through cross\-stage combinations\([51](https://arxiv.org/html/2608.03487#bib.bib73)\); global optimization accounts for these interactions by tuning complete pipelines, but end\-to\-end feedback alone cannot reveal which stage drives a gain or loss, leading to poorly directed exploration and wasted evaluations\.

\(P2\)Existing multi\-objective RAG optimizers overlook the system design space\.Existing quality–performance RAG optimizers search algorithm choices but measure performance on one fixed deployment\(Barkeret al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib15)\), while online methods adapt only a few serving parameters within that deployment\(Rayet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib85)\); neither finds an algorithm configuration’s best achievable serving performance\. Simply folding the full system design space into the multi\-objective search is wasteful because system parameters do not affect answer quality, yet every variant would still incur a full quality evaluation\.

\(P3\)Deployment\-based serving\-performance measurements are costly and non\-transferable across systems\.RAG serving performance is typically obtained by deploying and timing each configuration end to end, making dense deployment search prohibitively expensive\(Liet al\.,[2026](https://arxiv.org/html/2608.03487#bib.bib68); Lianget al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib63)\)\. Such measurements are system\-specific: a configuration on the original Pareto frontier may become dominated on a new system, so migration requires remeasurement and frontier reconstruction; candidate hardware also cannot be evaluated before acquisition\. Although prediction could avoid these costs, existing RAG performance models cover only narrow portions of the stack\(Jianget al\.,[2025a](https://arxiv.org/html/2608.03487#bib.bib47); Kim and Mahajan,[2026](https://arxiv.org/html/2608.03487#bib.bib53)\), leaving no portable full\-stack estimator\.

## 3\.RAG\-Stack: System Overview

We presentRAG\-Stack\(Fig\.[2](https://arxiv.org/html/2608.03487#S1.F2)\), an efficient framework that finds a RAG system’s quality \(QQ\)–performance \(PP\) Pareto frontier across the full algorithm and system design space, without deploying each candidate to measure its performance\. This addresses the limitations in §[2\.4](https://arxiv.org/html/2608.03487#S2.SS4), which leave RAG without a practical way to co\-optimize answer quality and serving performance across its full design space\.

Inputs\.In from\-scratch mode, a user drivesRAG\-Stackwith a single declarative specification of four things:\(i\)the*algorithm design space*to search, a hierarchical, conditional space of components and parameters detailed in §[4\.1](https://arxiv.org/html/2608.03487#S4.SS1)\(Fig\.[5](https://arxiv.org/html/2608.03487#S4.F5)\);\(ii\)an evaluation*dataset*of queries with reference answers;\(iii\)an*optimization target*—the quality and performance metrics to trade off, plus serving SLOs; and\(iv\)the available*system resources*—GPUs, CPU, and interconnect—bounding the system design spaceRAG\-CMsearches \(Table[1](https://arxiv.org/html/2608.03487#S4.T1)\)\. In system\-transfer mode, the input is instead the previous Pareto result and a new system\-resource specification, as described underTwo operating modesbelow\.

Output\.RAG\-Stackreturns the Pareto\-optimal configurations it found—each a complete algorithm\-and\-system deployment—so the user picks the operating point that fits their workload, hardware, and SLO\.

Overview\.RAG\-Stack\(Fig\.[2](https://arxiv.org/html/2608.03487#S1.F2)\) rests on one split: a RAG configuration’s parameters divide into an*algorithm design space*, whose choices change the produced answer and so move both quality and performance, and a*system design space*, whose choices change only how that fixed computation is served\.RAG\-Stacksearches the two in an iterative loop over three components\. Each round,RAG\-PE\(PlanExploration, §[4](https://arxiv.org/html/2608.03487#S4)\) proposes one algorithm configuration𝐱\\mathbf\{x\}; the quality evaluator runs that pipeline on the dataset and returns its qualityQ​\(𝐱\)Q\(\\mathbf\{x\}\);RAG\-IR\(IntermediateRepresentation, §[5](https://arxiv.org/html/2608.03487#S5)\) abstracts the executed run into a workload representation;RAG\-CM\(CostModel, §[6](https://arxiv.org/html/2608.03487#S6)\) searches the system design space for that representation and predicts𝐱\\mathbf\{x\}’s best serving performanceP​\(𝐱\)P\(\\mathbf\{x\}\); andRAG\-PEuses the measuredQ​\(𝐱\)Q\(\\mathbf\{x\}\)and predictedP​\(𝐱\)P\(\\mathbf\{x\}\)to choose the next configuration\. The loop stops when the optimization target is met or the budget is spent\. Quality comes from this one evaluation run, but performance is always predicted byRAG\-CM—no candidate is ever deployed to measure it\.

Two operating modes\.The loop above optimizes a RAG system*from scratch*, discovering its frontier on the current hardware\.RAG\-Stackalso supports a*system\-transfer*mode: to retarget an already\-optimized system to new hardware, the user provides only the previous Pareto result and a new system\-resource specification\.RAG\-Stackthen reuses the quality results from the first run, first invokingRAG\-CMto re\-score the previous deployments under the new resources and subsequently running a small number ofRAG\-PEpolishing iterations to refine the transferred frontier\.

Benefits\.RAG\-Stack’s design yields three benefits, each removing one limitation of prior optimizers\.B1 \(solving[P1](https://arxiv.org/html/2608.03487#S2.F4)\): stage\-aware multi\-objective optimization without breaking cross\-stage interactions\.RAG\-PEstill optimizes the end\-to\-end\(Q,P\)\(Q,P\)Pareto frontier, so each trial is a complete pipeline configuration and cross\-stage interactions are preserved\. At the same time, it reads stage\-level sub\-metrics to guide exploration: for example, low context recall with high faithfulness points to retrieval as the bottleneck rather than generation\. These sub\-metrics guide whereRAG\-PEsearches next without becoming separate objectives, giving one optimizer both global frontier optimization and stage\-level diagnosis \(§[4\.2](https://arxiv.org/html/2608.03487#S4.SS2)\)\.B2 \(solving[P2](https://arxiv.org/html/2608.03487#S2.SS4)\): near wallclock\-free exhaustive system search\.Given the algorithm/system split above,RAG\-PEspends expensive quality evaluations only on algorithm configurations, whileRAG\-CMexhaustively searches the system design space for each one inside the cost model\. Because this search uses predicted performance rather than real deployments, it is nearly wall\-clock\-free, lettingRAG\-Stackreport the best deployment\-side operating point instead of inheriting the performance of one fixed deployment \(§[4\.1](https://arxiv.org/html/2608.03487#S4.SS1), §[6](https://arxiv.org/html/2608.03487#S6)\)\.B3 \(solving[P3](https://arxiv.org/html/2608.03487#S2.SS4)\): cheap system transfer and planning\.BecauseRAG\-CMpredicts performance from a hardware description instead of a real run, an optimized system is retargeted to new hardware by re\-scoring the quality archive, and candidate machines are compared before purchase—neither step builds a real deployment\.

We now detail the three components in turn:RAG\-PE\(§[4](https://arxiv.org/html/2608.03487#S4)\),RAG\-IR\(§[5](https://arxiv.org/html/2608.03487#S5)\), andRAG\-CM\(§[6](https://arxiv.org/html/2608.03487#S6)\)\.

## 4\.RAG\-PE: Plan Exploration

RAG\-PEpartitions the design space \(§[4\.1](https://arxiv.org/html/2608.03487#S4.SS1)\), runs the RAG\-specific optimizer \(§[4\.2](https://arxiv.org/html/2608.03487#S4.SS2)\), and coordinates quality evaluation with performance modeling \(§[5](https://arxiv.org/html/2608.03487#S5), §[6](https://arxiv.org/html/2608.03487#S6)\)\.

### 4\.1\.Search Space

The space of RAG configurations is large and hierarchical\.RAG\-PEsplits it into an algorithm part and a system part, searches the algorithm part directly, and delegates the system part toRAG\-CM\. We define this split and then organize the partRAG\-PEsearches as a hierarchy\.

#### 4\.1\.1\.Two Design Spaces: Algorithm and System Design Spaces\.

Let𝒳full=∏j=1nΘj\\mathcal\{X\}\_\{\\mathrm\{full\}\}=\\prod\_\{j=1\}^\{n\}\\Theta\_\{j\}be the full configuration space of the RAG stack, with eachΘj\\Theta\_\{j\}denoting the domain of one parameter \(e\.g\.,nprobe,thread\_count,top\_k\), and letQ​\(𝐱\)Q\(\\mathbf\{x\}\)andP​\(𝐱\)P\(\\mathbf\{x\}\)be the answer\-quality and serving\-performance objectives under the user’s chosen metrics \(e\.g\., latency, throughput, or SLO satisfaction\)\. We partition the parameters by what each one changes: the*logical computation*that produces the answer, or only the*physical execution*of that fixed computation\. An*algorithm parameter*changes the logical computation \(Fig\.[5](https://arxiv.org/html/2608.03487#S4.F5)\); conversely,*a parameter belongs to the system design space whenever it leaves the logical computation unchanged*\. System parameters change only how that fixed computation is executed and served \(Table[1](https://arxiv.org/html/2608.03487#S4.T1)\)—placement, batching, parallelism, thread counts, and hardware\. Batching, for example, merely groups identical per\-request computations and so leaves the answer unchanged; varying parallelism or thread count may perturb the output slightly through floating\-point reduction order, but the logical computation remains unchanged\.

Formally, letIYI\_\{Y\}denote the parameters relevant to objectiveY∈\{P,Q\}Y\\in\\\{P,Q\\\}\. Any parameter relevant to answer qualityQQchanges the logical computation and must also be considered for serving performancePP; therefore,*IQ⊆IPI\_\{Q\}\\subseteq I\_\{P\}*\. This yields𝒳full=𝒳algo×𝒳cm\\mathcal\{X\}\_\{\\mathrm\{full\}\}=\\mathcal\{X\}\_\{\\mathrm\{algo\}\}\\times\\mathcal\{X\}\_\{\\mathrm\{cm\}\}, where𝒳algo:=∏j∈IQΘj\\mathcal\{X\}\_\{\\mathrm\{algo\}\}:=\\prod\_\{j\\in I\_\{Q\}\}\\Theta\_\{j\}contains parameters that affect bothQQandPP, while𝒳cm:=∏j∈IP∖IQΘj\\mathcal\{X\}\_\{\\mathrm\{cm\}\}:=\\prod\_\{j\\in I\_\{P\}\\setminus I\_\{Q\}\}\\Theta\_\{j\}contains parameters that affect onlyPP\.RAG\-Stackoptimizes𝒳algo\\mathcal\{X\}\_\{\\mathrm\{algo\}\}against\(Q,P\)\(Q,P\)and delegates𝒳cm\\mathcal\{X\}\_\{\\mathrm\{cm\}\}toRAG\-CMforPPalone\.

![Refer to caption](https://arxiv.org/html/2608.03487v1/x5.png)Figure 5\.Algorithm design space\(Panet al\.,[2024](https://arxiv.org/html/2608.03487#bib.bib79); Gaoet al\.,[2022](https://arxiv.org/html/2608.03487#bib.bib34); Malkov and Yashunin,[2018](https://arxiv.org/html/2608.03487#bib.bib72); Jégouet al\.,[2011](https://arxiv.org/html/2608.03487#bib.bib44); Khattab and Zaharia,[2020](https://arxiv.org/html/2608.03487#bib.bib52)\)searched byRAG\-PE\. The highlighted*selected*nodes are one example, showing how a single evaluation’s algorithm configuration is assembled: each trial fixes one value at every active branch of the hierarchy \(retrieval\-side in purple, model\-side in yellow\), and the starred callouts mark heuristic\-aware parameters coupled across branches\.Table 1\.System design space searched byRAG\-CM\.A one\-column table listing example system design choices\.CategoryDesign choiceExample rangeVector DBSearch threads\{16,32,64\}\\\{16,32,64\\\}Parallel mode\{intra\-query, inter\-query\}LLM servingParallelism strategy\{TP, PP, DP, hybrid\}GPUs per stage\{1,2,3,4\}\\\{1,2,3,4\\\}Prefill–decode deployment\{collocated, disaggregated\}PlacementStage\-to\-device mappingPrefill→\\toGPUs \{0–7\}BatchingRequest batch size\{1,2,…,256\}\\\{1,2,\\ldots,256\\\}Decode batch size\{16,32,64,128,256\}\\\{16,32,64,128,256\\\}Dynamic batching wait time\{0,1,2,5,10\}​ms\\\{0,1,2,5,10\\\}\\,\\mathrm\{ms\}Hardware\(Optional\)CPU model\{AMD EPYC, Intel Xeon\}GPU inventory\{4×\\timesH100, 8×\\timesA100\}
#### 4\.1\.2\.Search\-space organization\.

RAG\-PEparses the user’s declarative specification into the two design spaces above\. System\-only choices in𝒳cm\\mathcal\{X\}\_\{\\mathrm\{cm\}\}are sent toRAG\-CM\(§[6](https://arxiv.org/html/2608.03487#S6)\), which exhaustively searches them inside the cost model to return the best predicted performanceP~​\(𝐱\)\\tilde\{P\}\(\\mathbf\{x\}\)for a given algorithm configuration\.RAG\-PEtherefore runs the expensive ground\-truth optimization only over𝒳algo\\mathcal\{X\}\_\{\\mathrm\{algo\}\}, whileRAG\-CMhandles the internal system search\. It organizes𝒳algo\\mathcal\{X\}\_\{\\mathrm\{algo\}\}as a hierarchical design space \(Fig\.[5](https://arxiv.org/html/2608.03487#S4.F5)\) governed by two rules:*conditional activation*and*heuristic constraints*that remove invalid or low\-value trials\. Under conditional activation, a child parameter is active only when its parent choice is active; for example, query\-rewriter parameters are searched only if query rewriting is enabled\. The heuristic constraints \(starred nodes in Fig\.[5](https://arxiv.org/html/2608.03487#S4.F5)\) use smooth proxies for derived knobs: IVF setsnlist=factor⋅N\\texttt\{nlist\}=\\texttt\{factor\}\\cdot\\sqrt\{N\}for corpus sizeNN, while PQ selects a valid divisorMMof the embedding dimensionDD\.

### 4\.2\.The Optimizer

As discussed in §[2\.4](https://arxiv.org/html/2608.03487#S2.SS4), RAG’s algorithm design space is entangled by cross\-stage interactions: the best setting for one stage depends on the choices made in the others \(Fig\.[4](https://arxiv.org/html/2608.03487#S2.F4)\)\. Optimizing stages independently can therefore miss configurations that work well only as a complete pipeline\.RAG\-PEconsequently uses*multi\-objective Bayesian optimization*\(MOBO\) as its global foundation, evaluating and ranking complete configurations against the end\-to\-end\(Q,P\)\(Q,P\)Pareto frontier\. MOBO also suits the small evaluation budget: its probabilistic surrogate and acquisition function use prior observations and uncertainty to select the configuration expected to improve the frontier most\.

A global optimizer alone, however, sees only the end\-to\-end objectives and lacks directional information about which stage should change\.RAG\-PEtherefore augments the global MOBO with stage\-level feedback: stage diagnostics guide exploration toward promising changes, while the global acquisition function still arbitrates complete configurations according to how much they are expected to improve the\(Q,P\)\(Q,P\)frontier\. Stage information thus directs the search without becoming a separate objective or breaking cross\-stage interactions\. We first introduce the MOBO foundation and then present our*stage–global co\-aware*extensions\.

#### 4\.2\.1\.Preliminary: multi\-objective Bayesian optimization\.

*Multi\-objective Bayesian optimization*\(MOBO\) follows the general Pareto\-search formulation in §[2\.3](https://arxiv.org/html/2608.03487#S2.SS3)\. In our setting,𝒳=𝒳algo\\mathcal\{X\}=\\mathcal\{X\}\_\{\\mathrm\{algo\}\}and𝐲​\(𝐱\)=\(P​\(𝐱\),Q​\(𝐱\)\)\\mathbf\{y\}\(\\mathbf\{x\}\)=\\bigl\(P\(\\mathbf\{x\}\),Q\(\\mathbf\{x\}\)\\bigr\), using the serving\-performance and answer\-quality objectives defined above\. Because evaluating quality is expensive, MOBO seeks to identify the frontier using as few evaluations as possible\.

MOBO consists of two components\. The first is a cheap probabilistic*surrogate*for each objective—a Gaussian process \(GP\) fitted to all configurations evaluated so far\. Because𝒳algo\\mathcal\{X\}\_\{\\mathrm\{algo\}\}contains many categorical variables and has a hierarchical structure, each GP uses a mixed kernel: a Matérn kernel for the numeric knobs and a Hamming kernel for the categorical ones\. Together, they provide an appropriate notion of similarity for this design space\(Lévesqueet al\.,[2017](https://arxiv.org/html/2608.03487#bib.bib59)\)\.

The second ingredient is an*acquisition function*α\\alphathat scores any unevaluated configuration by its expected gain in this hypervolume\. Let𝒟t\\mathcal\{D\}\_\{t\}denote the data afterttrounds\. The*hypervolume improvement*\(HVI\) of a candidate outcome𝐲\\mathbf\{y\}overℱ\\mathcal\{F\}isHVI​\(𝐲∣ℱ,𝐫\)=HV​\(ℱ∪\{𝐲\};𝐫\)−HV​\(ℱ;𝐫\)\\mathrm\{HVI\}\(\\mathbf\{y\}\\mid\\mathcal\{F\},\\mathbf\{r\}\)=\\mathrm\{HV\}\(\\mathcal\{F\}\\cup\\\{\\mathbf\{y\}\\\};\\mathbf\{r\}\)\-\\mathrm\{HV\}\(\\mathcal\{F\};\\mathbf\{r\}\)\. HVI alone cannot score a candidate, because the outcome𝐲\\mathbf\{y\}is unknown before the evaluation\.*Expected hypervolume improvement*\(EHVI\) resolves this by averaging HVI over the GP posterior prediction at𝐱\\mathbf\{x\}, measured against the frontier of the outcomes observed so far\(Daultonet al\.,[2021](https://arxiv.org/html/2608.03487#bib.bib27)\)\. EHVI, however, trusts those observations to be exact, while our quality scores are noisy: the observed outcomes need not form the true frontier\.*Noisy*EHVI \(NEHVI\) also treats the frontier as uncertain, averaging HVI over posterior draws of the evaluated configurations and the candidate:αNEHVI​\(𝐱\)=𝔼𝐟∼p​\(𝐟∣𝒟t\)​\[HVI​\(𝐟​\(𝐱\)∣ℱ𝐟,𝐫\)\]\\alpha\_\{\\mathrm\{NEHVI\}\}\(\\mathbf\{x\}\)=\\mathbb\{E\}\_\{\\mathbf\{f\}\\sim p\(\\mathbf\{f\}\\mid\\mathcal\{D\}\_\{t\}\)\}\[\\mathrm\{HVI\}\(\\mathbf\{f\}\(\\mathbf\{x\}\)\\mid\\mathcal\{F\}\_\{\\mathbf\{f\}\},\\mathbf\{r\}\)\]\. We estimate this expectation usingNNSobol draws from the joint GP posterior; each draw𝐟j\\mathbf\{f\}\_\{j\}induces a frontierℱj\\mathcal\{F\}\_\{j\}over the evaluated configurations\(Daultonet al\.,[2021](https://arxiv.org/html/2608.03487#bib.bib27)\)\.

In practice, we maximize*LogNEHVI*, a numerically stabilized variant of NEHVI, using the BoTorch implementation\(Amentet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib4); Balandatet al\.,[2020](https://arxiv.org/html/2608.03487#bib.bib13)\)\. Each round,RAG\-PEfits the GPs, picks the next configuration by maximizing the acquisition over the whole space,𝐱t\+1=arg​max𝐱∈𝒳algo⁡αLogNEHVI​\(𝐱\)\\mathbf\{x\}\_\{t\+1\}=\\operatorname\*\{arg\\,max\}\_\{\\mathbf\{x\}\\in\\mathcal\{X\}\_\{\\mathrm\{algo\}\}\}\\alpha\_\{\\mathrm\{LogNEHVI\}\}\(\\mathbf\{x\}\), evaluates it, and appends the result\.

These considerations lead us to choose GP\+LogNEHVI over SMAC’s combination of a random\-forest \(RF\) surrogate and ParEGO\(Lindaueret al\.,[2022](https://arxiv.org/html/2608.03487#bib.bib66); Knowles,[2006](https://arxiv.org/html/2608.03487#bib.bib54)\)\. LogNEHVI directly rewards expected improvement in the noisy Pareto hypervolume, whereas ParEGO targets the frontier indirectly through scalarized objectives\(Daultonet al\.,[2021](https://arxiv.org/html/2608.03487#bib.bib27)\)\. The RF instead learns similarity through tree partitions; its piecewise\-constant predictions can create plateaus in the acquisition landscape, leaving little signal for local refinement\.

Despite this principled acquisition rule, the standard loop is poorly directed when applied to RAG’s large hierarchical design space\. The surrogate is fitted only on the two end\-to\-end objectives, so its posterior uncertainty indicates where the search has not looked, but not*why*a configuration succeeds or which pipeline stage is worth changing\. Under a tight quality\-evaluation budget, it can therefore spend trials on configurations that are novel yet unlikely to be useful\.

#### 4\.2\.2\.Our optimizer: stage–global co\-aware MOBO

RAG\-PEextends conventional MOBO for RAG with stage\-aware diagnostics, heterogeneous candidate generation, and global arbitration, making exploration more directed while preserving end\-to\-end Pareto optimization\. This design is necessary because stage effects are non\-separable—the benefit of changing one stage depends on the choices made in the others \(Fig\.[4](https://arxiv.org/html/2608.03487#S2.F4)\)\. Greedy stage\-wise optimization can therefore miss configurations that work well only as a complete pipeline\([51](https://arxiv.org/html/2608.03487#bib.bib73)\)\. Composite\-function BO also exploits intermediate outputs, but assumes that the final objective is a known function of those outputs\(Astudillo and Frazier,[2019](https://arxiv.org/html/2608.03487#bib.bib8); Kudvaet al\.,[2026](https://arxiv.org/html/2608.03487#bib.bib56)\)\. RAG provides useful per\-stage diagnostics, but no known function maps them to end\-to\-end quality and performance\.RAG\-PEtherefore uses these diagnostics to direct candidate generation while leaving complete\-configuration comparison to the global MOBO objective\.RAG\-PEcombines three mechanisms, shown in Fig\.[6](https://arxiv.org/html/2608.03487#S4.F6)and detailed below\.

![Refer to caption](https://arxiv.org/html/2608.03487v1/x6.png)Figure 6\.The optimizer ofRAG\-PE\.##### \(1\) Sub\-metric Awareness\.

RAG\-PEoptimizes only end\-to\-end qualityQQand performancePP, while its*agentic analyzer*\(![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x7.png)\) uses stage\-level sub\-metrics to diagnose where a configuration falls short\. The quality diagnostics are context recall and precision for retrieval and answer faithfulness for the Main LLM; the performance diagnostics areRAG\-CM’s per\-resource capacitiesκr\\kappa\_\{r\}\(§[6\.5](https://arxiv.org/html/2608.03487#S6.SS5)\), which expose the bottleneck stage and remaining headroom\. We record them as𝐬i\\mathbf\{s\}\_\{i\}alongside each observation, yielding𝒟t=\{\(𝐱i,Pi,Qi\)\}i=1t\\mathcal\{D\}\_\{t\}=\\\{\(\\mathbf\{x\}\_\{i\},P\_\{i\},Q\_\{i\}\)\\\}\_\{i=1\}^\{t\}for the GPs andℋt=\{\(𝐱i,Pi,Qi,𝐬i\)\}i=1t\\mathcal\{H\}\_\{t\}=\\\{\(\\mathbf\{x\}\_\{i\},P\_\{i\},Q\_\{i\},\\mathbf\{s\}\_\{i\}\)\\\}\_\{i=1\}^\{t\}for the analyzer\. Readingℋt\\mathcal\{H\}\_\{t\}, the analyzer formsMtM\_\{t\}stage\-guided candidate configurationsℛtLLM=\{𝐜1,…,𝐜Mt\}\\mathcal\{R\}^\{\\mathrm\{LLM\}\}\_\{t\}=\\\{\\mathbf\{c\}\_\{1\},\\ldots,\\mathbf\{c\}\_\{M\_\{t\}\}\\\}\. Each𝐜m=𝐛m⊕𝜹m\\mathbf\{c\}\_\{m\}=\\mathbf\{b\}\_\{m\}\\oplus\\bm\{\\delta\}\_\{m\},m=1,…,Mtm=1,\\ldots,M\_\{t\}, is constructed by applying a targeted edit𝜹m\\bm\{\\delta\}\_\{m\}to a previously evaluated base𝐛m\\mathbf\{b\}\_\{m\}and snapping the result to the valid grid;⊕\\oplusoverwrites only the edited parameters\(Liuet al\.,[2024](https://arxiv.org/html/2608.03487#bib.bib69); Laoet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib58)\)\. Thus, sub\-metrics guide candidate generation without becoming surrogate inputs or optimization objectives\.

##### \(2\) Heterogeneous Candidate Pool\.

Standard acquisition optimization uses multiple random restarts, but all candidates are refined against the same surrogate and therefore inherit the same model bias\.RAG\-PEinstead constructs𝒞t\\mathcal\{C\}\_\{t\}from four complementary channels: Sobol coverage, stage\-guided candidates, DLS\-based acquisition\-guided local refinement, and Pareto\-tension crossover \(Fig\.[6](https://arxiv.org/html/2608.03487#S4.F6)\)\.

Two channels determine where to search:*Sobol breadth*draws a hierarchy\-aware, space\-filling candidate setℬt\\mathcal\{B\}\_\{t\}, while the*agentic analyzer*supplies stage\-guided candidatesℛtLLM\\mathcal\{R\}^\{\\mathrm\{LLM\}\}\_\{t\}\(![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x7.png)\)\.

The stage\-guided candidates and quasi\-random restartsℛtrand\\mathcal\{R\}^\{\\mathrm\{rand\}\}\_\{t\}seed*discrete local search*\(DLS,![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x8.png)\)\. Let𝐳\(k\)\\mathbf\{z\}^\{\(k\)\}denote the discrete configuration afterkkDLS steps, and let𝒩​\(𝐳\)\\mathcal\{N\}\(\\mathbf\{z\}\)denote its hierarchy\-valid one\-parameter neighbors\. Each𝐜m\\mathbf\{c\}\_\{m\}initializes a DLS trajectory at𝐳\(0\)=𝐜m\\mathbf\{z\}^\{\(0\)\}=\\mathbf\{c\}\_\{m\}\. Writing𝒩\+​\(𝐳\)=𝒩​\(𝐳\)∪\{𝐳\}\\mathcal\{N\}^\{\+\}\(\\mathbf\{z\}\)=\\mathcal\{N\}\(\\mathbf\{z\}\)\\cup\\\{\\mathbf\{z\}\\\}, DLS greedily applies𝐳\(k\+1\)=arg​max𝐳∈𝒩\+​\(𝐳\(k\)\)⁡αLogNEHVI​\(𝐳\)\\mathbf\{z\}^\{\(k\+1\)\}=\\operatorname\*\{arg\\,max\}\_\{\\mathbf\{z\}\\in\\mathcal\{N\}^\{\+\}\(\\mathbf\{z\}^\{\(k\)\}\)\}\\alpha\_\{\\mathrm\{LogNEHVI\}\}\(\\mathbf\{z\}\)until convergence\. Thus, the agent chooses a promising region and DLS refines the configuration within it\.

*Pareto\-tension crossover*\(![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x9.png)\) starts from the best observed configuration for each objective and applies the single\-knob crossover that most strongly moves it toward the other objective\. Foro∈\{P,Q\}o\\in\\\{P,Q\\\}, denote that anchor by𝐱o⋆=arg​max𝐱∈𝒟t⁡o​\(𝐱\)\\mathbf\{x\}\_\{o\}^\{\\star\}=\\operatorname\*\{arg\\,max\}\_\{\\mathbf\{x\}\\in\\mathcal\{D\}\_\{t\}\}o\(\\mathbf\{x\}\)and the other objective byo¯\\bar\{o\}\. Let𝐱o,j=𝐱o⋆⊕jxo¯,j⋆\\mathbf\{x\}\_\{o,j\}=\\mathbf\{x\}\_\{o\}^\{\\star\}\\oplus\_\{j\}x\_\{\\bar\{o\},j\}^\{\\star\}denote the hierarchy\-valid crossover that replaces only knobjjwith its value from𝐱o¯⋆\\mathbf\{x\}\_\{\\bar\{o\}\}^\{\\star\}\. It selectsjo⋆=arg​maxj⁡τo,jj\_\{o\}^\{\\star\}=\\operatorname\*\{arg\\,max\}\_\{j\}\\tau\_\{o,j\}, where hats denote min–max normalization andτo,j=o¯^​\(𝐱o,j\)−o¯^​\(𝐱o⋆\)−\[o^​\(𝐱o⋆\)−o^​\(𝐱o,j\)\]\+\\tau\_\{o,j\}=\\hat\{\\bar\{o\}\}\(\\mathbf\{x\}\_\{o,j\}\)\-\\hat\{\\bar\{o\}\}\(\\mathbf\{x\}\_\{o\}^\{\\star\}\)\-\[\\hat\{o\}\(\\mathbf\{x\}\_\{o\}^\{\\star\}\)\-\\hat\{o\}\(\\mathbf\{x\}\_\{o,j\}\)\]\_\{\+\}, yieldingℛt×\\mathcal\{R\}^\{\\times\}\_\{t\}\. The candidate pool is𝒞t=ℬt∪ℛtLLM∪DLS​\(ℛtLLM∪ℛtrand\)∪ℛt×\\mathcal\{C\}\_\{t\}=\\mathcal\{B\}\_\{t\}\\cup\\mathcal\{R\}^\{\\mathrm\{LLM\}\}\_\{t\}\\cup\\mathrm\{DLS\}\\\!\\left\(\\mathcal\{R\}^\{\\mathrm\{LLM\}\}\_\{t\}\\cup\\mathcal\{R\}^\{\\mathrm\{rand\}\}\_\{t\}\\right\)\\cup\\mathcal\{R\}^\{\\times\}\_\{t\}, with DLS applied independently to each seed and source labels retained for arbitration \(![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x10.png)\)\.

##### \(3\) Arbitration\.

On ordinary rounds, LogNEHVI scores every candidate in𝒞t\\mathcal\{C\}\_\{t\}, andRAG\-PEevaluates the candidate with the largest expected hypervolume improvement, independent of its generator\. When the number of consecutive ordinary evaluations with no realized normalized\-hypervolume gain reaches a stagnation thresholdτs\\tau\_\{s\},RAG\-PEactivates its*forced channels*\. These channels target the largest gap between adjacent points on the normalized realized Pareto frontier and draw candidates from the agent and Pareto\-tension crossover\. The candidates bypass LogNEHVI and are ranked by the product of their range\-normalized posterior standard deviations, with hierarchy\-aware novelty breaking ties\. Finally,*quality reuse*skips generation and judging when the retrieved passages and quality\-relevant downstream configuration match an earlier evaluation:RAG\-PEreuses its quality score, recomputes performance withRAG\-CM, records the point, and repeats arbitration\.

## 5\.RAG\-IR: Intermediate Representation

RAG\-IRbridges quality evaluation andRAG\-CMthrough a common workload representation with two properties\. It is*system\-agnostic*, separating logical work from deployment soRAG\-CMcan search deployment choices and re\-cost the same workload on new hardware without rerunning it\. It is*RAG\-workflow\-agnostic*, encoding sequential and agentic pipelines with a common schema soRAG\-PEcan explore workflows whileRAG\-CMevaluates them without workflow\-specific modeling logic \(Fig\.[2](https://arxiv.org/html/2608.03487#S1.F2)\)\.

RAG\-IRrecords an order\-free workflow schema for throughput and per\-request execution traces for latency \(§[6\.5](https://arxiv.org/html/2608.03487#S6.SS5)\)\.

Workflow schema\.An order\-free summary of each stage’s performance\-relevant attributes and aggregate logical work, used byRAG\-CMto predict throughput\.

Per\-request trace\.For each requestqq,RAG\-IRrecords an execution DAG𝒢i\(q\)\\mathcal\{G\}\_\{i\}^\{\(q\)\}\. Each node is one stage invocation and stores its stage type and input/output token counts; edges record dependencies between calls\. Repeated calls in iterative or agentic pipelines appear as separate nodes, preserving their order and multiplicity\.RAG\-IRforwards the schema and traces, together with𝐱i\\mathbf\{x\}\_\{i\}, toRAG\-CM\(§[6](https://arxiv.org/html/2608.03487#S6)\)\.

## 6\.RAG\-CM: Cost Model

RAG\-CMis the performance\-estimation component ofRAG\-Stack\. For a fixed RAG algorithm configuration𝐱i\\mathbf\{x\}\_\{i\}, it estimates the best attainable serving performance on a target system by searching the system design space𝒳cm\\mathcal\{X\}\_\{\\mathrm\{cm\}\}, whose parameters affect serving performance but not answer quality\. Its inputs are the workload representation produced byRAG\-IR\(a workflow schema and per\-request execution traces𝒢i\\mathcal\{G\}\_\{i\}\), the algorithm configuration𝐱i\\mathbf\{x\}\_\{i\}, and a resource specification of the target system’s devices, communication topology, and serving resources\.

We denote by𝐬∈𝒳cm\\mathbf\{s\}\\in\\mathcal\{X\}\_\{\\mathrm\{cm\}\}one concrete assignment of all system\-design parameters \(Table[1](https://arxiv.org/html/2608.03487#S4.T1)\)\. For each pair\(𝐱i,𝐬\)\(\\mathbf\{x\}\_\{i\},\\mathbf\{s\}\),RAG\-CMpredicts throughput and end\-to\-end latency,𝐩~​\(𝐱i,𝐬\)\\tilde\{\\mathbf\{p\}\}\(\\mathbf\{x\}\_\{i\},\\mathbf\{s\}\)\.

Why a cost model for performance\.Predicting performance instead of measuring it serves four roles inRAG\-Stack\.\(R1\) A smaller optimizer space\.By moving the system design space intoRAG\-CM, the optimizer only searches the algorithm design space𝒳algo\\mathcal\{X\}\_\{\\mathrm\{algo\}\}, so the scarce ground\-truth budget is spent only on parameters that affect quality \(§[4](https://arxiv.org/html/2608.03487#S4)\)\.\(R2\) RAG serving planning and transfer\.RAG\-CMlets users plan RAG serving before acquiring the target system and efficiently retarget it to new hardware by re\-scoring the serving performance of previously identified Pareto configurations on the new system\.\(R3\) Faster optimization iterations\.BecauseRAG\-CMpredicts performance without deploying each candidate or benchmarking its performance, every optimization iteration avoids this overhead, substantially reducing wall\-clock time relative to deployment\-based optimizers\.\(R4\) Exhaustive system search\.RAG\-CMexhaustively searches the system design space for each algorithm configuration, yielding higher performance and more effective optimization\.

The four\-layer structure\.RAG\-CMcomputes𝐩~\\tilde\{\\mathbf\{p\}\}, and henceP~\\tilde\{P\}, in four layers \(Fig\.[7](https://arxiv.org/html/2608.03487#S6.F7)\): an*algorithm*layer that produces one hardware\-agnostic Operator Work Profile per RAG stage \(§[6\.1](https://arxiv.org/html/2608.03487#S6.SS1)\), a*performance*layer that maps each Operator Work Profile to time on the host \(§[6\.3](https://arxiv.org/html/2608.03487#S6.SS3)\), a*communication*layer that prices the data moved between operators \(§[6\.4](https://arxiv.org/html/2608.03487#S6.SS4)\), and an*assembly*layer that sweeps the system design space𝒳cm\\mathcal\{X\}\_\{\\mathrm\{cm\}\}and composes the other three layers into the throughput and latency of each algorithm–system configuration pair \(§[6\.5](https://arxiv.org/html/2608.03487#S6.SS5)\)\.

![Refer to caption](https://arxiv.org/html/2608.03487v1/x11.png)Figure 7\.Overview ofRAG\-CM\. A: analytical; ML: fused machine learning and analytical\.### 6\.1\.Algorithm Modeling

GivenRAG\-IR’s workflow and execution trace, the algorithm layer models every RAG stage and outputs one hardware\-agnostic*Operator Work Profile*per stage—a record of that stage’s per\-phase operation counts, data movement, and execution characteristics \(![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x7.png)in Fig\.[7](https://arxiv.org/html/2608.03487#S6.F7); §[6\.2](https://arxiv.org/html/2608.03487#S6.SS2)\)\.RAG\-CMuses GenZ\(Bambhaniyaet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib14)\)for model\-inference operators; the following subsections present its retrieval models, grouped into Clustering Indices \(§[6\.1\.1](https://arxiv.org/html/2608.03487#S6.SS1.SSS1)\) and Graph Indices \(§[6\.1\.2](https://arxiv.org/html/2608.03487#S6.SS1.SSS2)\)\.

#### 6\.1\.1\.Clustering Indices

All three clustering indices share one search skeleton: a query is run through a*coarse quantizer*that selects thenproben\_\{\\text\{probe\}\}inverted lists nearest to it, the candidate vectors in those lists are*scanned*to compute distances, a*top\-kkheap*keeps the best candidates, and an optional*re\-rank*recomputes exact distances on the survivors\. They are*analytical*: given the configuration, the work in every stage is fixed in closed form\. Table[2](https://arxiv.org/html/2608.03487#S6.T2)lists every stage, its compute kernel, the peak it is bounded by, and which variants use it\. For most stages, the corresponding Operator Work Profile follows directly from the FAISS source: the ops, bytes, and access pattern of each kernel are fixed by the configuration, so a closed\-form formula gives the profile for each such stage listed in Table[2](https://arxiv.org/html/2608.03487#S6.T2), with variant\-specific formulas where the kernels differ; for example, standard PQ builds its distance table per probed list, whereas Fastscan builds it once per query\. Only the scan stage requires special treatment because its work cannot be read directly from the configuration, so we model it below\.

Table 2\.IVF\-family search stages \(Flat: IVF\-Flat; PQ: IVF\-PQ; FS: IVF\-PQ Fastscan; FSR: Fastscan with residual\)\.StageCompute ModePeakUsed by1\. Coarse quantizerBLAS GEMM if per\-thread batch≥20\\geq 20, else hand\-vectorized SIMD distance kernelFP32all2\. LUT buildnon\-residual, built once/query \(FS\); residual\-dependent \(PQ, FSR\)FP32PQ, FS, FSR3a\. PQ scanADC table lookup \(PQ\) /vpshufb\(FS, FSR\)intPQ, FS, FSR3b\. Flat scanfull\-vector SIMD L2intFlat4\. Top\-kkheapbranch\-heavy integerintall5\. Refinementexact SIMD L2 on candidatesFP32optional##### PQ/flat scan: data\-aware imbalance\.

The scan’s cost is driven byNscanN\_\{\\text\{scan\}\}, the number of database vectors a query touches—a vector*count*, not an op or byte total\. Under a fixed PQ configuration, scoring each scanned vector requires reading itsbcodeb\_\{\\text\{code\}\}\-byte PQ code and summing one precomputed distance\-table entry per subquantizer, so the scan’s ops areNscanN\_\{\\text\{scan\}\}times the per\-vector work and its bytes areNscan​bcodeN\_\{\\text\{scan\}\}\\,b\_\{\\text\{code\}\}; this is howNscanN\_\{\\text\{scan\}\}sets the scan operator’s profile\. IVF makesNscanN\_\{\\text\{scan\}\}far smaller than the corpus sizeNN: a query scans only thenproben\_\{\\text\{probe\}\}ofnlistn\_\{\\text\{list\}\}inverted lists nearest to it, a fractionρ=nprobe/nlist\\rho=n\_\{\\text\{probe\}\}/n\_\{\\text\{list\}\}of all vectors\. The naive estimateNscan=ρ​NN\_\{\\text\{scan\}\}=\\rho Nassumes equal\-sized lists, but real embeddings cluster, so a query’s nearest lists are larger than average and it scans more; we correct with a cell\-imbalance factorffmeasured from the built index, so thatNscan=f​\(nprobe\)​ρ​NN\_\{\\text\{scan\}\}=f\(n\_\{\\text\{probe\}\}\)\\,\\rho\\,N, whereffis the ratio of the vectors a real query actually scans—obtained from the index’s per\-list sizes and the nearest\-list assignments of a query sample—to the balancedρ​N\\rho N\. Thisffis the only index\-dependent input the model needs, and it is a pure property of the corpus and embedding—never of the hardware—while the remaining index knobs enter the cost model as analytical parameters\.RAG\-CMtherefore cachesffonce per \(corpus, embedding\)—only a handful of combinations across a whole design space—and from then on predicts the scan count for any configuration without ever building an index again\.

#### 6\.1\.2\.Graph Indices

HNSW is*ML\-analytical*\. A query first descends greedily through theL≈logM⁡NL\\approx\\log\_\{M\}Nupper layers—a few hops each overMM\-neighbor lists—to reach a good entry point, then runs a bounded best\-first \(beam\) search of width𝑒𝑓s\\mathit\{ef\}\_\{s\}at the base layer, which performs almost all of the work\. Where IVF’s scan count follows in closed form from the configuration, the size of the base\-layer neighborhood an HNSW query explores is data\-dependent—it grows with the data’s intrinsic geometry and shrinks on clustered corpora where the beam converges fast—and has no closed form\. The model therefore*predicts*the two quantities that drive the Operator Work Profile: the per\-query distance computationsndisn\_\{\\text\{dis\}\}and graph hopsnhopsn\_\{\\text\{hops\}\}\. These fold into the profile’sWWandVVexactly as IVF’sNscanN\_\{\\text\{scan\}\}does \(§[6\.2](https://arxiv.org/html/2608.03487#S6.SS2)\)—ndisn\_\{\\text\{dis\}\}setting the dominant base\-layer FLOPs and random\-vector reads, andnhopsn\_\{\\text\{hops\}\}the visited\-set and heap overhead\. Concretely, the HNSW operator model estimatesn^dis=cM​g^dis​\(𝐳\)\\hat\{n\}\_\{\\text\{dis\}\}=c\_\{M\}\\,\\hat\{g\}\_\{\\text\{dis\}\}\(\\mathbf\{z\}\)andn^hops=g^hops​\(𝐳\)\\hat\{n\}\_\{\\text\{hops\}\}=\\hat\{g\}\_\{\\text\{hops\}\}\(\\mathbf\{z\}\)with feature vector𝐳=\(d,M,𝑒𝑓s,LID\)\\mathbf\{z\}=\(d,M,\\mathit\{ef\}\_\{s\},\\mathrm\{LID\}\): a learned predictorg^\\hat\{g\}emits the workload counts𝐧^=\(n^dis,n^hops\)\\hat\{\\mathbf\{n\}\}=\(\\hat\{n\}\_\{\\text\{dis\}\},\\hat\{n\}\_\{\\text\{hops\}\}\), and a factorcMc\_\{M\}obtained through one\-time per\-corpus calibration fixes the scale ofn^dis\\hat\{n\}\_\{\\text\{dis\}\}\. These counts determine the HNSW Operator Work Profile; the following paragraph definesg^\\hat\{g\}and its lightweight calibration\.

##### Workload predictor\.

The predictorg^\\hat\{g\}is two independent gradient\-boosted regressors,g^dis\\hat\{g\}\_\{\\text\{dis\}\}andg^hops\\hat\{g\}\_\{\\text\{hops\}\}; both are predicted because their ratio is not constant—it drifts with𝑒𝑓s\\mathit\{ef\}\_\{s\}—and each contributes a different part of the Operator Work Profile\. The lone data\-dependent feature is the corpus local intrinsic dimensionalityLID\\mathrm\{LID\}, estimated by an Amsaleg MLE on a query sample; it carries the geometry that lets a single model generalize across corpora—from high\-LID synthetic training data to the low\-LID, tightly clustered distributions of real image and text embeddings, on which a configuration\-only estimate over\-counts because the beam terminates sooner than the ambientddsuggests\. Crucially, every feature in𝐳\\mathbf\{z\}is either a search\-space knob or a property of the corpus itself: the predictor uses*no*statistic of a constructed graph \(degree, layer counts\), so it scores candidate configurations whose indexes have not been built—exactly the regime the optimizer explores\. The predictor requires only a one\-time calibration for each corpus, which is reused across all unbuilt configurations and therefore adds little overhead\.

### 6\.2\.Operator Work Profile

Each operator emits a hardware\-agnostic*Operator Work Profile*, the interface between the algorithm and performance layers\. Computed once per operator phase, it containswork\(W,V\)\(W,V\)—the operation count and bytes moved after cache reuse—and fourexecution characteristics: \(1\) the*compute mode*\(e\.g\.blas/simd\), which selects compute efficiency; \(2\) the*bytes location*, determined by the working\-set size and unique byte volumeVuniq≤VV\_\{\\text\{uniq\}\}\\\!\\leq\\\!V, which selects the memory level; \(3\) the*access\-block size*, which distinguishes sequential from scattered bandwidth; and \(4\) the*parallel fraction*for Amdahl scaling\. The performance layer maps these fields to effective compute and memory rates\(πeff,βeff\)\(\\pi\_\{\\text\{eff\}\},\\beta\_\{\\text\{eff\}\}\)\. Data\-dependent corrections modifyWWandVVrather than the profile schema, allowing one performance model to serve all operators\.

### 6\.3\.Performance Modeling

The performance layer maps an*Operator Work Profile*to wall\-clock time in two composable steps \(![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x9.png),![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x10.png)in Fig\.[7](https://arxiv.org/html/2608.03487#S6.F7)\): a*roofline*bounds each phase by the slower of its compute and memory time, and*Amdahl’s law*composes the per\-phase times across threads under the chosen parallel mode\. The intermediate points below exist only to feed these two: the hardware model supplies the roofline’s two rates \(πeff\\pi\_\{\\text\{eff\}\},βeff\\beta\_\{\\text\{eff\}\}\), and the parallel model sets how the per\-phase results combine into end\-to\-end latency and throughput\.

##### Roofline kernel\.

For a phase withWWoperations and data volumeVV, the model predictsT=Tovh\+max⁡\(Wπeff,Vβeff\)T=T\_\{\\mathrm\{ovh\}\}\+\\max\\\!\\left\(\\frac\{W\}\{\\pi\_\{\\mathrm\{eff\}\}\},\\frac\{V\}\{\\beta\_\{\\mathrm\{eff\}\}\}\\right\), whereπeff\\pi\_\{\\mathrm\{eff\}\}andβeff\\beta\_\{\\mathrm\{eff\}\}are the effective compute throughput and bandwidth, andTovhT\_\{\\mathrm\{ovh\}\}captures fixed per\-call costs\.

##### Hardware and execution factors\.

To derive these effective rates and compose phase times,RAG\-CMaccounts for \(1\)*CPU topology*, including heterogeneous cores and NUMA effects; \(2\)*effective cache capacity*under sharing among concurrent threads; \(3\)*scattered–sequential memory access*, which is sequential within a block but random across blocks; and \(4\)*parallel execution*, using separate compute and memory thread counts with Amdahl scaling\.

### 6\.4\.Communication Modeling

Given a deployment,RAG\-CM’s communication model estimates the time to move every inter\-stage and intra\-LLM payload across its selected devices \(![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x12.png)in Fig\.[7](https://arxiv.org/html/2608.03487#S6.F7)\)\. It derives these costs from the concrete topology: every GPU pair carries its own bandwidth and startup latency\(βi​j,ℓi​j\)\(\\beta\_\{ij\},\\ell\_\{ij\}\), and GPU–CPU hops use a host\-transfer class—covering tensor\-parallel \(TP\) and pipeline\-parallel \(PP\) collectives, data\-parallel \(DP\) replicas, GPU–GPU stage handoffs such as the prefill→\\todecode KV\-cache transfer under 1P/1D disaggregation, and CPU\-side boundaries\. A cross\-device edgeeesums its fastest endpoint\-disjoint pair bandwidths intoβe\\beta\_\{e\}and takes its slowest startup asℓe\\ell\_\{e\}; LLM\-internal TP/PP traffic folds into the prefill/decode service time rather than adding assembly edges; and each explicit cross\-device edge is priced asℓe\+Me/βe\\ell\_\{e\}\+M\_\{e\}/\\beta\_\{e\}over its payloadMeM\_\{e\}, with text and dataframe boundaries \(e\.g\., retrieval→\\toreranker\) priced as host transfers\.

### 6\.5\.RAG\-CMAssembly

The assembly layer \(![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x13.png)in Fig\.[7](https://arxiv.org/html/2608.03487#S6.F7)\) has two functions: it enumerates and materializes each𝐬∈𝒳cm\\mathbf\{s\}\\in\\mathcal\{X\}\_\{\\mathrm\{cm\}\}as one deployment and execution plan, then composes stage\-level operator and communication predictions into steady\-state, end\-to\-end serving performance\. For each candidate it prices, assembly predicts the saturation throughputT~​\(𝐬\)\\tilde\{T\}\(\\mathbf\{s\}\)and the corresponding mean latencyL~​\(𝐬\)\\tilde\{L\}\(\\mathbf\{s\}\)once serving reaches steady state; startup behavior is outside the model\.RAG\-CMuses the performance objective and SLOs in the user’s optimization target to select among these configurations and exposes the selected performance toRAG\-PEasP~​\(𝐱i\)\\tilde\{P\}\(\\mathbf\{x\}\_\{i\}\)\.

#### 6\.5\.1\.Steady\-state performance under saturated closed\-loop serving

We model a saturated closed\-loop deployment with a fixed number of concurrent clients, each issuing its next query after the previous response\. The resultingT~\\tilde\{T\}is the maximum sustainable throughput, andL~\\tilde\{L\}is the steady\-state response latency at that operating point\. This target characterizes the deployment’s attainable capacity without introducing an external arrival rate\. An open\-system latency instead depends on the offered load and can make the same deployment appear lightly loaded, near saturation, or unstable\.

#### 6\.5\.2\.Trace\-driven assembly

For each algorithm–system pair\(𝐱i,𝐬\)\(\\mathbf\{x\}\_\{i\},\\mathbf\{s\}\),RAG\-CMinstantiates a closed discrete\-event model fromRAG\-IR’s workload representation and the operator and communication models \(§[6\.4](https://arxiv.org/html/2608.03487#S6.SS4)\)\. The simulation captures four mechanisms: \(1\)Continuous and dynamic batching:LLM stages use continuous batching, while other stages dispatch batches by size or timeout; \(2\)Automatic prefix caching:repeated calls prefill only the uncached prompt suffix within KV\-cache capacity; \(3\)Decoupled batch limits:generator decode and other stages use independent batch\-size limits\(Jianget al\.,[2025a](https://arxiv.org/html/2608.03487#bib.bib47)\); and \(4\)Shared\-resource contention:collocated stages time\-share GPUs and CPUs\. At concurrencyNN,T^​\(𝐬,N\)\\widehat\{T\}\(\\mathbf\{s\},N\)andL^​\(𝐬,N\)\\widehat\{L\}\(\\mathbf\{s\},N\)denote steady\-state throughput \(QPS\) and mean end\-to\-end latency, respectively\. To scale to thousands of candidates,RAG\-CMuses a cheap analytical model to shortlist the best batch settings per deployment topology, then simulates only those candidates to throughput saturation and reports only simulation results\.

## 7\.Evaluation

Our evaluation is organized around four questions\.

RQ1 \(End\-to\-end value\)\.Across random seeds and under the same evaluation budget, doesRAG\-Stackconsistently find better quality–performance Pareto frontiers than existing optimization methods?

RQ2 \(Optimizer ablation\)\.With the rest ofRAG\-Stackheld fixed, does the optimizer insideRAG\-PEoutperform existing alternatives?

RQ3 \(Cost\-model accuracy\)\.How accurate isRAG\-CM, in absolute error and, more importantly, in ranking candidate deployments?

RQ4 \(System transfer\)\.After migrating to new hardware, how much doesRAG\-Stackimprove normalized hypervolume over re\-optimizing from scratch under the same evaluation budget?

### 7\.1\.Experimental Setup

Hardware\.We use two NUMA servers as measured deployment targets with different CPU and GPU resources:SysAhas two AMD EPYC 9124 sockets, 32 CPU cores in total \(16 cores per socket, 3\.0 GHz base and up to 3\.7 GHz boost\), two NUMA nodes, and four NVIDIA H100 GPUs;SysBhas two AMD EPYC 7742 sockets, 128 CPU cores in total \(64 cores per socket, 2\.25 GHz base and up to 3\.4 GHz boost\), two NUMA nodes, and eight NVIDIA A100 80GB GPUs\.

Datasets and Quality Metrics\.For the end\-to\-end evaluation, we use 100 queries each from RAGEval\(Zhuet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib120)\)to avoid contamination from LLM training data\(Xuet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib107); Qiet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib84)\), and MS MARCO\(Bajajet al\.,[2018](https://arxiv.org/html/2608.03487#bib.bib12)\)to evaluate on a larger, more diverse dataset\. TheRAG\-CMaccuracy evaluation uses additional datasets, which we describe in §[7\.4](https://arxiv.org/html/2608.03487#S7.SS4)\. The quality objectiveQQis RAGAS answer correctness\(Eset al\.,[2024](https://arxiv.org/html/2608.03487#bib.bib30)\), which scores factual agreement with the gold answer rather than lexical overlap—so valid paraphrases are not penalized—in\[0,1\]\[0,1\], averaged over queries\. As per\-stage diagnostics,RAG\-PEreads three RAGAS sub\-metrics \(§[4\.2](https://arxiv.org/html/2608.03487#S4.SS2)\): context recall, context precision, and faithfulness\. All LLM\-as\-judge scoring uses DeepSeek\-V4\-Flash\.

Algorithm Design Spaces\.Our algorithm design space follows Fig\.[5](https://arxiv.org/html/2608.03487#S4.F5); here, we specify only the choices not detailed in the figure, while all other components and parameter ranges remain as shown\. For RAGEval, the chunk size is chosen from\{128,256,512,1024,2048\}\\\{128,\\allowbreak 256,\\allowbreak 512,\\allowbreak 1024,\\allowbreak 2048\\\}; the embedding model is BGE\-small\([20](https://arxiv.org/html/2608.03487#bib.bib32)\)\(384\-d\), all\-mpnet\-base\-v2\(Songet al\.,[2020](https://arxiv.org/html/2608.03487#bib.bib96)\)\(768\-d\), or BGE\-M3\(Chenet al\.,[2024b](https://arxiv.org/html/2608.03487#bib.bib23)\)\(1024\-d\); the reranker is TART\(Asaiet al\.,[2023a](https://arxiv.org/html/2608.03487#bib.bib7)\), ColBERT\(Khattab and Zaharia,[2020](https://arxiv.org/html/2608.03487#bib.bib52)\), SentenceTransformer\(Reimers and Gurevych,[2019](https://arxiv.org/html/2608.03487#bib.bib86)\), or FlagEmbedding\([20](https://arxiv.org/html/2608.03487#bib.bib32);[J\. Chen, S\. Xiao, P\. Zhang, K\. Luo, D\. Lian, and Z\. Liu \(2024b\)](https://arxiv.org/html/2608.03487#bib.bib23)\); and the query rewriter and main LLM independently use a Qwen2\.5\(Baiet al\.,[2023](https://arxiv.org/html/2608.03487#bib.bib11)\)model from\{1\.5,3,7,14\}\\\{1\.5,\\allowbreak 3,\\allowbreak 7,\\allowbreak 14\\\}B\. For MS MARCO, we fix the chunk size at 2048 and the embedding model to all\-mpnet\-base\-v2 to keep indexing its 8\.84M passages tractable\.

System Design Spaces\.We instantiate the system axes of Table[1](https://arxiv.org/html/2608.03487#S4.T1)\. The end\-to\-end and optimizer\-ablation experiments \(§[7\.2](https://arxiv.org/html/2608.03487#S7.SS2), §[7\.3](https://arxiv.org/html/2608.03487#S7.SS3)\) share the four\-H100 SysA space:RAG\-CMmaximizes throughput over collocated and disaggregated GPU\-stage placements, all feasible stage\-to\-GPU mappings and LLM parallelism plans \(at most four GPUs per stage\), request and decode batch sizes in\{1,2,4,…,256\}\\\{1,2,4,\\ldots,256\\\}withBdecode≥BrequestB\_\{\\mathrm\{decode\}\}\\geq B\_\{\\mathrm\{request\}\}, and dynamic\-batching waits in\{2,10,50\}​ms\\\{2,10,50\\\}\\,\\mathrm\{ms\}; FAISS threads followmin⁡\(retrieval batch,physical cores\)\\min\(\\text\{retrieval batch\},\\text\{physical cores\}\)under inter\-query parallelism\. For transfer to the eight\-A100 SysB \(§7\.5\), only the GPU budget and per\-stage cap grow from four to eight\.

*Baseline system space\.*The baselines in §[7\.2](https://arxiv.org/html/2608.03487#S7.SS2)have no performance model, so a fair comparison lets them search the algorithm design space jointly with the system axes as one entangled space\. But every candidate they propose must be deployed and measured, and the full system design space would spend most of this budget on poor system variants\. We therefore compress all system axes thatRAG\-CMsearches into two categorical choices, expert\-curated to be strong on SysA: eight*deployment presets*\(collocated TP/PP plans and 1P/1D–2P/2D prefill/decode disaggregation, with non\-main\-LLM stages either sharing GPUs with the main LLM or using GPUs left unused by it\) crossed with sixBrequest×BdecodeB\_\{\\mathrm\{request\}\}\{\\times\}B\_\{\\mathrm\{decode\}\}*batch presets*\.

### 7\.2\.End\-to\-End Optimization

Figure[8](https://arxiv.org/html/2608.03487#S7.F8)compares the measured quality–throughput Pareto frontiers under the protocol in §[7\.1](https://arxiv.org/html/2608.03487#S7.SS1)\. Following §[2\.3](https://arxiv.org/html/2608.03487#S2.SS3), we compute hypervolume from linearly normalized quality and raw throughput per dataset; throughput is log\-scaled only in the plots for readability\. We then compute normalized hypervolume independently for each seed and report the three\-seed mean\. Against GP\+LogNEHVI\(Daultonet al\.,[2021](https://arxiv.org/html/2608.03487#bib.bib27)\), the strongest baseline on both datasets,RAG\-Stackaverages 0\.514 versus 0\.364 on RAGEval and 0\.635 versus 0\.259 on MS MARCO; the corresponding per\-seed relative improvements average 52\.5% and 153\.2%, respectively\.RAG\-Stack’s per\-seed hypervolume exceeds the strongest baseline’s in all six seed–dataset pairs\.

![Refer to caption](https://arxiv.org/html/2608.03487v1/x14.png)Figure 8\.End\-to\-end quality–performance Pareto frontiers \(log\-scaled performance\)\. Translucent markers show per\-seed frontiers; each line pools seeds 43–45\. SelectedRAG\-Stackconfigurations are re\-measured on SysA\.As a secondary benefit,RAG\-Stackcompletes the end\-to\-end search in 4\.59 hours on RAGEval and 3\.60 hours on MS MARCO, faster than all baselines except Greedy\-Forward \(Table[3](https://arxiv.org/html/2608.03487#S7.T3)\)\. This efficiency holds even thoughRAG\-CMevaluates 570–5,400 system configurations per candidate, because it prices them with the model rather than deploying each one\.

Table 3\.Mean end\-to\-end cold\-start runtime on SysA\.OptimizerRAGEval \(hours\)MS MARCO \(hours\)Greedy\-Forward\([51](https://arxiv.org/html/2608.03487#bib.bib73)\)2\.25±0\.232\.25\\pm 0\.233\.16±0\.063\.16\\pm 0\.06GP\+LogNEHVI\(Daultonet al\.,[2021](https://arxiv.org/html/2608.03487#bib.bib27)\)5\.60±0\.405\.60\\pm 0\.405\.68±0\.515\.68\\pm 0\.51SMAC\(Lindaueret al\.,[2022](https://arxiv.org/html/2608.03487#bib.bib66)\)8\.03±0\.428\.03\\pm 0\.427\.03±0\.377\.03\\pm 0\.37RAG\-Stack\(ours\)4\.59±0\.514\.59\\pm 0\.513\.60±1\.793\.60\\pm 1\.79To explain these trade\-offs, we inspect representative RAG configurations in Figure[8](https://arxiv.org/html/2608.03487#S7.F8)and attribute their gains to algorithm and system choices\.

On RAGEval \(left\),RAG\-Stackpoint![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x8.png)dominates the baseline\-best point![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x7.png)with higher quality at81×81\\timesthe throughput\. Both use a 14B generator, but![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x7.png)adds 3B multi\-query expansion to HNSW \(𝑒𝑓s=128\\mathit\{ef\}\_\{s\}\{=\}128, top\-k=16k\{=\}16\) without reranking and uses a fixed 1P/2D plan \(one prefill and two decode replicas, batch16×3216\{\\times\}32, with 14B prefill on one H100\)\.![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x8.png)disables expansion and uses 2048\-character chunks with ColBERT after top\-k=8k\{=\}8IVF\-PQ to rank the retrieved documents by relevance, raising answer correctness from 0\.594 to 0\.638, whileRAG\-CMselects a faster serving plan\.

GP\+LogNEHVI is seed\-dependent: only a few of its points rise aboveRAG\-Stack’s frontier, all of them from seed 43 and confined to narrow throughput ranges, the widest at![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x9.png)\(245\.6 vs\. 204\.1 QPS; 0\.335 vs\. 0\.326 quality\)\.RAG\-Stackstill has higher seed\-43 hypervolume, and SMAC and Greedy\-Forward never win on both axes\.![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x9.png)replacesRAG\-Stack’s top\-k=1k\{=\}1plus ColBERT with reranker\-free top\-k=4k\{=\}4retrieval over shorter chunks, then adds 1P/2D disaggregation and a larger batch\. Once seed 43 finds this region, LogNEHVI concentrates later evaluations nearby; the other seeds never enter it—stochastic discovery followed by exploitation, not a stable baseline advantage\.

On MS MARCO \(right\),RAG\-Stackpoint![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x12.png)dominates the quality\-best baseline point![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x10.png)with globally best quality at8\.6×8\.6\\timesthe throughput:![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x10.png)relies on top\-k=64k\{=\}64, whereas![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x12.png)uses top\-k=16k\{=\}16and aRAG\-CM\-selected disaggregated layout\. At the throughput extreme,![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x13.png)dominates the baselines’ fastest point![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x15.png)with1\.65×1\.65\\timesthe throughput and2\.2×2\.2\\timesthe quality\. Both use a 1\.5B generator;![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x13.png)retains accurate, uncompressed HNSW and cuts only to top\-k=1k\{=\}1—sufficient because each fixed 2048\-character passage forms one chunk—then parallelizes generation with TP2×\\timesPP2 and encoding with TP4\.![[Uncaptioned image]](https://arxiv.org/html/2608.03487v1/x15.png)instead combines IVF\-PQ\-FS with ReAct, whose retrieval errors the small generator cannot correct\.

Across the overlapping quality range in Figure[8](https://arxiv.org/html/2608.03487#S7.F8), serving at equal quality is1\.51\.5–7\.5×7\.5\\timescheaper on MS MARCO than on RAGEval, likely due to public\-benchmark leakage into LLM training data\(Xuet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib107); Qiet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib84)\)\. Public benchmarks may therefore understate the serving cost of quality, motivating RAGEval as a fresh\-data control\.

### 7\.3\.Optimizer Ablation

![Refer to caption](https://arxiv.org/html/2608.03487v1/x16.png)Figure 9\.Optimizer ablation: dominated hypervolume versus evaluation budget, holding the rest ofRAG\-Stackfixed and varying only the optimizer insideRAG\-PE\. Lines are means over 3 seeds; bands show seed min/max\.To isolate the contribution ofRAG\-PE’s optimizer, we keep every other part ofRAG\-Stackunchanged and replace only this optimizer with each baseline in turn\. We compareRAG\-PEwith SMAC\(Lindaueret al\.,[2022](https://arxiv.org/html/2608.03487#bib.bib66)\), GP\+LogNEHVI\(Balandatet al\.,[2020](https://arxiv.org/html/2608.03487#bib.bib13)\), Greedy\-Forward, and Greedy\-Lookback\([51](https://arxiv.org/html/2608.03487#bib.bib73)\); the last alternates backward and forward passes after its initial forward pass\. Greedy\-RAG\-PEis a hybrid that switches from Greedy\-Forward toRAG\-PEafter 30 of 50 evaluations\. Each method runs with seeds 43–45, and Figure[9](https://arxiv.org/html/2608.03487#S7.F9)reports mean normalized hypervolume by budget\.

RAG\-PEis the most consistent optimizer across both datasets: against the four standalone baselines, it attains the highest mean hypervolume at 77 of 80 post\-warm\-start budgets\. At budget 50, it reaches 0\.375 versus Greedy\-Forward’s 0\.327 on RAGEval and 0\.681 versus Greedy\-Lookback’s 0\.599 on MS MARCO, relative gains of 14\.8% and 13\.8%\. No single baseline is consistently strongest: the strongest method differs between the two datasets in this ablation, while the end\-to\-end comparison identifies a different strongest baseline \(§[7\.2](https://arxiv.org/html/2608.03487#S7.SS2)\)\. In contrast,RAG\-PEremains strong across settings and contributes independently toRAG\-Stack’s end\-to\-end advantage\.\.

![Refer to caption](https://arxiv.org/html/2608.03487v1/x17.png)Figure 10\.Best attainable quality versus throughput requirement: per optimizer, the highest\-quality configuration meeting the requirement within the budget \(lines: mean over 3 seeds; bands: seed min/max\)\.Because dominated hypervolume also credits configurations in regions no deployment would use \(extremely low quality or throughput\), Figure[10](https://arxiv.org/html/2608.03487#S7.F10)re\-reads the same archives from a deployment perspective: the best quality each optimizer can serve once a minimum throughput is required\.

![Refer to caption](https://arxiv.org/html/2608.03487v1/x18.png)Figure 11\.Per\-channel contributions toRAG\-PE’s results in §[7\.3](https://arxiv.org/html/2608.03487#S7.SS3): shares of all quality evaluations \(left\) and final hypervolume \(right\)\.Figure[11](https://arxiv.org/html/2608.03487#S7.F11)shows that initialization is critical: the Sobol\-plus\-agent warm start accounts for the largest share of the final hypervolume\. The remaining channels also make non\-trivial contributions to the final hypervolume\.

### 7\.4\.RAG\-CMAccuracy

Table 4\.RAG\-CMprediction accuracy\. Error is MAPE;ρs\\rho\_\{s\}andrrdenote Spearman and Pearson correlation;nnis the number of configurations\.SysASysBComponentnnErr\.↓\\downarrowρs↑\\rho\_\{s\}\\uparrowr↑r\\uparrowErr\.↓\\downarrowρs↑\\rho\_\{s\}\\uparrowr↑r\\uparrowIVF\-Flat \(Latency\)1376\.7%0\.9981\.00011\.4%0\.9930\.997IVF\-PQ \(Latency\)13711\.3%0\.9960\.99713\.2%0\.9960\.998IVF\-PQ\-FS \(Latency\)13717\.0%0\.9890\.99712\.8%0\.9940\.998HNSW \(Latency\)108017\.0%0\.9860\.97120\.1%0\.9770\.957Sequential RAG \(Latency\)20111\.9%0\.9730\.95513\.4%0\.9650\.950Sequential RAG \(QPS\)20111\.0%0\.9750\.96914\.6%0\.9650\.950Agentic RAG \(Latency\)308\.1%0\.9660\.98713\.8%0\.9840\.979Agentic RAG \(QPS\)3012\.2%0\.9690\.97420\.3%0\.9670\.980RAG Overall \(Latency\)23111\.4%0\.9780\.95813\.5%0\.9700\.952RAG Overall \(QPS\)23111\.2%0\.9780\.95215\.3%0\.9680\.952Table[4](https://arxiv.org/html/2608.03487#S7.T4)evaluates calibratedRAG\-CMpredictions on SysA and SysB at the operator and RAG pipeline levels\. The operator study spans four vector\-search corpora \(SIFT1M\(Jégouet al\.,[2011](https://arxiv.org/html/2608.03487#bib.bib44)\), GloVe, GIST1M\(Jégouet al\.,[2011](https://arxiv.org/html/2608.03487#bib.bib44)\), and Deep1M\(Yandex and Lempitsky,[2016](https://arxiv.org/html/2608.03487#bib.bib108)\)\) and two embedded RAG corpora \(ELI5\(Fanet al\.,[2019](https://arxiv.org/html/2608.03487#bib.bib31)\)and TriviaQA\(Joshiet al\.,[2017](https://arxiv.org/html/2608.03487#bib.bib50)\)\), varying batch size and CPU parallelism across HNSW and IVF variants \(full grids in the artifact\)\. The RAG pipeline study crosses 77 serving configurations with RAGEval\(Zhuet al\.,[2025](https://arxiv.org/html/2608.03487#bib.bib120)\), ELI5, and TriviaQA, covering sequential and ReAct\-style agentic workflows and varying models, indexes, batching, parallelism, colocation, disaggregation, and dynamic batching; each deployment is compared against measured saturated closed\-loop QPS and mean end\-to\-end latency\.

For guiding system space search, preserving the relative order of configurations matters more than minimizing absolute prediction error\. Across both systems, calibratedRAG\-CMattains Spearman correlations of 0\.968–0\.978 for latency and QPS, while keeping overall MAPE at 11\.2–15\.3%\. Even for random\-access\-heavy IVF\-PQ\-FS and HNSW, where MAPE rises to 20\.1%, Spearman correlation remains at least 0\.977\. Thus, the residual prediction error largely preserves configuration ordering, providing the ranking signal thatRAG\-PEneeds\.

### 7\.5\.System Transfer

![Refer to caption](https://arxiv.org/html/2608.03487v1/x19.png)Figure 12\.System transfer from SysA to SysB\. The transferred Pareto configurations are re\-measured end\-to\-end on SysB; all points shown are measured\.We evaluate system transfer by migrating optimized RAG configurations from SysA to SysB\.RAG\-Stackre\-scores the existing quality archive withRAG\-CMinstantiated for SysB, spends 20 additional evaluations refining the frontier, and re\-measures the selected configurations on SysB\. The*uncalibrated*variant uses only SysB’s hardware description, whereas the*calibrated*variant additionally calibratesRAG\-CMon the target machine\. Figure[12](https://arxiv.org/html/2608.03487#S7.F12)compares both variants with from\-scratch optimization on SysB under 20\- and 50\-evaluation budgets\.

With the same 20\-evaluation budget, uncalibrated and calibrated transfer improve normalized HV over from\-scratch optimization by 108\.4% \(0\.331 vs\. 0\.159\) and 182\.2% \(0\.448 vs\. 0\.159\), respectively\.

## 8\.Limitations and Discussion

Our current support for agentic RAG is trace\-driven:RAG\-IRmust execute the pipeline to observe its runtime control flow beforeRAG\-CMcan predict its performance\. Consequently,RAG\-PEcannot densely sample the performance objective before quality evaluation—which would let a multi\-task Gaussian process \(MTGP\) exploit abundant performance\-only observations to improve sample efficiency—or filter SLO\-violating configurations before they enter the candidate pool\. We have implemented trace\-free staticRAG\-IRandRAG\-CMpaths for sequential RAG, whose control flow is known from the configuration alone; extending such pre\-execution performance prediction to agentic RAG remains future work\.

## 9\.Conclusion

We presentRAG\-Stack, a system for efficiently discovering quality–performance Pareto frontiers across RAG algorithms and serving systems\.RAG\-StackcombinesRAG\-PEfor sub\-metric\-aware multi\-objective exploration,RAG\-IRfor representing executed sequential and agentic workflows, andRAG\-CMfor predicting serving performance and searching system configurations without deploying every candidate\. Experiments show thatRAG\-Stackimproves normalized hypervolume over the strongest end\-to\-end baseline, with three\-seed mean gains of 52\.5% on RAGEval and 153\.2% on MS MARCO\.

###### Acknowledgements\.

We sincerely thank Yongjun He and Gustavo Alonso for their insightful discussions and invaluable suggestions throughout the development of this work\.

## References

- \[1\]\(2026\-06\)Algorithmicsuperintelligence/openevolve\.Note:Algorithmic SuperIntelligence LabsCited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1)\.
- S\. Ament, S\. Daulton, D\. Eriksson, M\. Balandat, and E\. Bakshy \(2025\)Unexpected Improvements to Expected Improvement for Bayesian Optimization\.arXiv\.External Links:2310\.20708,[Document](https://dx.doi.org/10.48550/arXiv.2310.20708)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1),[§4\.2\.1](https://arxiv.org/html/2608.03487#S4.SS2.SSS1.p4.1)\.
- A\. Asai, T\. Schick, P\. Lewis, X\. Chen, G\. Izacard, S\. Riedel, H\. Hajishirzi, and W\. Yih \(2023a\)Task\-aware Retrieval with Instructions\.InFindings of the Association for Computational Linguistics: ACL 2023,A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 3650–3675\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.225)Cited by:[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p3.2)\.
- A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. Hajishirzi \(2023b\)Self\-RAG: Learning to Retrieve, Generate, and Critique through Self\-Reflection\.arXiv\.External Links:2310\.11511,[Document](https://dx.doi.org/10.48550/arXiv.2310.11511)Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p1.1)\.
- R\. Astudillo and P\. Frazier \(2019\)Bayesian Optimization of Composite Functions\.InProceedings of the 36th International Conference on Machine Learning,pp\. 354–363\.External Links:ISSN 2640\-3498Cited by:[§4\.2\.2](https://arxiv.org/html/2608.03487#S4.SS2.SSS2.p1.1)\.
- J\. Bai, S\. Bai, Y\. Chu, Z\. Cui, K\. Dang, X\. Deng, Y\. Fan, W\. Ge, Y\. Han, F\. Huang, B\. Hui, L\. Ji, M\. Li, J\. Lin, R\. Lin, D\. Liu, G\. Liu, C\. Lu, K\. Lu, J\. Ma, R\. Men, X\. Ren, X\. Ren, C\. Tan, S\. Tan, J\. Tu, P\. Wang, S\. Wang, W\. Wang, S\. Wu, B\. Xu, J\. Xu, A\. Yang, H\. Yang, J\. Yang, S\. Yang, Y\. Yao, B\. Yu, H\. Yuan, Z\. Yuan, J\. Zhang, X\. Zhang, Y\. Zhang, Z\. Zhang, C\. Zhou, J\. Zhou, X\. Zhou, and T\. Zhu \(2023\)Qwen Technical Report\.arXiv\.External Links:2309\.16609,[Document](https://dx.doi.org/10.48550/arXiv.2309.16609)Cited by:[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p3.2)\.
- P\. Bajaj, D\. Campos, N\. Craswell, L\. Deng, J\. Gao, X\. Liu, R\. Majumder, A\. McNamara, B\. Mitra, T\. Nguyen, M\. Rosenberg, X\. Song, A\. Stoica, S\. Tiwary, and T\. Wang \(2018\)MS MARCO: A Human Generated MAchine Reading COmprehension Dataset\.arXiv\.External Links:1611\.09268,[Document](https://dx.doi.org/10.48550/arXiv.1611.09268)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p9.1),[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p2.2)\.
- M\. Balandat, B\. Karrer, D\. Jiang, S\. Daulton, B\. Letham, A\. G\. Wilson, and E\. Bakshy \(2020\)BoTorch: A Framework for Efficient Monte\-Carlo Bayesian Optimization\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 21524–21538\.Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1),[§4\.2\.1](https://arxiv.org/html/2608.03487#S4.SS2.SSS1.p4.1),[§7\.3](https://arxiv.org/html/2608.03487#S7.SS3.p1.1)\.
- A\. Bambhaniya, R\. Raj, G\. Jeong, S\. Kundu, S\. Srinivasan, S\. Subramanian, M\. Elavazhagan, M\. Kumar, and T\. Krishna \(2025\)Demystifying AI Platform Design for Distributed Inference of Next\-Generation LLM models\.arXiv\.External Links:2406\.01698,[Document](https://dx.doi.org/10.48550/arXiv.2406.01698)Cited by:[§6\.1](https://arxiv.org/html/2608.03487#S6.SS1.p1.1)\.
- M\. Barker, A\. Bell, E\. Thomas, J\. Carr, T\. Andrews, and U\. Bhatt \(2025\)Faster, Cheaper, Better: Multi\-Objective Hyperparameter Optimization for LLM and RAG Systems\.arXiv\.External Links:2502\.18635,[Document](https://dx.doi.org/10.48550/arXiv.2502.18635)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p3.1),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p4.1)\.
- M\. Besta, L\. Paleari, J\. H\. A\. Jiang, R\. Gerstenberger, Y\. Wu, J\. G\. Hannesson, P\. Iff, A\. Kubicek, P\. Nyczyk, D\. Khimey, N\. Blach, H\. Zhang, T\. Zhang, P\. Ma, G\. Kwaśniewski, M\. Copik, H\. Niewiadomski, and T\. Hoefler \(2025\)Affordable AI Assistants with Knowledge Graph of Thoughts\.arXiv\.External Links:2504\.02670,[Document](https://dx.doi.org/10.48550/arXiv.2504.02670)Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p1.1)\.
- H\. Beyer and H\. Schwefel \(2002\)Evolution strategies – A comprehensive introduction\.Natural Computing1\(1\),pp\. 3–52\.External Links:ISSN 1572\-9796,[Document](https://dx.doi.org/10.1023/A%3A1015059928466)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1)\.
- J\. Chen, P\. Zhou, Y\. Hua, L\. Xin, K\. Chen, Z\. Li, B\. Zhu, and J\. Liang \(2024a\)FinTextQA: A Dataset for Long\-form Financial Question Answering\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 6025–6047\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.328)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p1.1)\.
- J\. Chen, S\. Xiao, P\. Zhang, K\. Luo, D\. Lian, and Z\. Liu \(2024b\)M3\-Embedding: Multi\-Linguality, Multi\-Functionality, Multi\-Granularity Text Embeddings Through Self\-Knowledge Distillation\.InFindings of the Association for Computational Linguistics: ACL 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 2318–2335\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.137)Cited by:[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p3.2)\.
- S\. Daulton, M\. Balandat, and E\. Bakshy \(2021\)Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement\.arXiv\.External Links:2105\.08195,[Document](https://dx.doi.org/10.48550/arXiv.2105.08195)Cited by:[§2\.3](https://arxiv.org/html/2608.03487#S2.SS3.p2.13),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1),[§4\.2\.1](https://arxiv.org/html/2608.03487#S4.SS2.SSS1.p3.12),[§4\.2\.1](https://arxiv.org/html/2608.03487#S4.SS2.SSS1.p5.1),[§7\.2](https://arxiv.org/html/2608.03487#S7.SS2.p1.1),[Table 3](https://arxiv.org/html/2608.03487#S7.T3.4.4.3)\.
- K\. Deb, A\. Pratap, S\. Agarwal, and T\. Meyarivan \(2002\)A fast and elitist multiobjective genetic algorithm: NSGA\-II\.IEEE Transactions on Evolutionary Computation6\(2\),pp\. 182–197\.External Links:ISSN 1941\-0026,[Document](https://dx.doi.org/10.1109/4235.996017)Cited by:[§2\.3](https://arxiv.org/html/2608.03487#S2.SS3.p2.13),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1)\.
- M\. Douze, A\. Guzhva, C\. Deng, J\. Johnson, G\. Szilvasy, P\. Mazaré, M\. Lomeli, L\. Hosseini, and H\. Jégou \(2025\)The Faiss library\.arXiv\.External Links:2401\.08281,[Document](https://dx.doi.org/10.48550/arXiv.2401.08281)Cited by:[§2\.1](https://arxiv.org/html/2608.03487#S2.SS1.p2.1),[§2\.1](https://arxiv.org/html/2608.03487#S2.SS1.p3.1)\.
- S\. Es, J\. James, L\. Espinosa Anke, and S\. Schockaert \(2024\)RAGAs: Automated Evaluation of Retrieval Augmented Generation\.InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations,N\. Aletras and O\. De Clercq \(Eds\.\),St\. Julians, Malta,pp\. 150–158\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.eacl-demo.16)Cited by:[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p2.2)\.
- A\. Fan, Y\. Jernite, E\. Perez, D\. Grangier, J\. Weston, and M\. Auli \(2019\)ELI5: Long Form Question Answering\.arXiv\.External Links:1907\.09190,[Document](https://dx.doi.org/10.48550/arXiv.1907.09190)Cited by:[§7\.4](https://arxiv.org/html/2608.03487#S7.SS4.p1.1)\.
- \[20\]\(2026\-07\)FlagOpen/FlagEmbedding\.Note:FlagOpenCited by:[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p3.2)\.
- J\. Fu, X\. Qin, F\. Yang, L\. Wang, J\. Zhang, Q\. Lin, Y\. Chen, D\. Zhang, S\. Rajmohan, and Q\. Zhang \(2024\)AutoRAG\-HP: Automatic Online Hyper\-Parameter Tuning for Retrieval\-Augmented Generation\.arXiv\.External Links:2406\.19251,[Document](https://dx.doi.org/10.48550/arXiv.2406.19251)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p1.1)\.
- L\. Gao, X\. Ma, J\. Lin, and J\. Callan \(2022\)Precise Zero\-Shot Dense Retrieval without Relevance Labels\.arXiv\.External Links:2212\.10496,[Document](https://dx.doi.org/10.48550/arXiv.2212.10496)Cited by:[Figure 5](https://arxiv.org/html/2608.03487#S4.F5),[Figure 5](https://arxiv.org/html/2608.03487#S4.F5.5.2)\.
- Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, M\. Wang, and H\. Wang \(2024\)Retrieval\-Augmented Generation for Large Language Models: A Survey\.arXiv\.External Links:2312\.10997,[Document](https://dx.doi.org/10.48550/arXiv.2312.10997)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p1.1)\.
- N\. Hansen \(2023\)The CMA Evolution Strategy: A Tutorial\.arXiv\.External Links:1604\.00772,[Document](https://dx.doi.org/10.48550/arXiv.1604.00772)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1)\.
- A\. Harlap, D\. Narayanan, A\. Phanishayee, V\. Seshadri, N\. Devanur, G\. Ganger, and P\. Gibbons \(2018\)PipeDream: Fast and Efficient Pipeline Parallel DNN Training\.arXiv\.External Links:1806\.03377,[Document](https://dx.doi.org/10.48550/arXiv.1806.03377)Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p3.1)\.
- Y\. Huang, Y\. Cheng, A\. Bapna, O\. Firat, M\. X\. Chen, D\. Chen, H\. Lee, J\. Ngiam, Q\. V\. Le, Y\. Wu, and Z\. Chen \(2019\)GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism\.arXiv\.External Links:1811\.06965,[Document](https://dx.doi.org/10.48550/arXiv.1811.06965)Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p3.1)\.
- P\. Jamshidi, M\. Velez, C\. Kästner, and N\. Siegmund \(2018\)Learning to sample: exploiting similarities across environments to learn performance models for configurable systems\.InProceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering,ESEC/FSE 2018,New York, NY, USA,pp\. 71–82\.External Links:[Document](https://dx.doi.org/10.1145/3236024.3236074),ISBN 978\-1\-4503\-5573\-5Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p3.1)\.
- H\. Jégou, M\. Douze, and C\. Schmid \(2011\)Product Quantization for Nearest Neighbor Search\.IEEE Transactions on Pattern Analysis and Machine Intelligence33\(1\),pp\. 117–128\.External Links:ISSN 1939\-3539,[Document](https://dx.doi.org/10.1109/TPAMI.2010.57)Cited by:[§2\.1](https://arxiv.org/html/2608.03487#S2.SS1.p2.1),[Figure 5](https://arxiv.org/html/2608.03487#S4.F5),[Figure 5](https://arxiv.org/html/2608.03487#S4.F5.5.2),[§7\.4](https://arxiv.org/html/2608.03487#S7.SS4.p1.1)\.
- W\. Jiang, S\. Subramanian, C\. Graves, G\. Alonso, A\. Yazdanbakhsh, and V\. Dadu \(2025a\)RAGO: Systematic Performance Optimization for Retrieval\-Augmented Generation Serving\.arXiv\.External Links:2503\.14649,[Document](https://dx.doi.org/10.48550/arXiv.2503.14649)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p3.1),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p1.1),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p5.1),[§6\.5\.2](https://arxiv.org/html/2608.03487#S6.SS5.SSS2.p1.4)\.
- W\. Jiang, M\. Zeller, R\. Waleffe, T\. Hoefler, and G\. Alonso \(2025b\)Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval\-Augmented Language Models\.arXiv\.External Links:2310\.09949,[Document](https://dx.doi.org/10.48550/arXiv.2310.09949)Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p3.1)\.
- W\. Jiang, S\. Zhang, B\. Han, J\. Wang, B\. Wang, and T\. Kraska \(2025c\)PipeRAG: Fast Retrieval\-Augmented Generation via Adaptive Pipeline Parallelism\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\.1,KDD ’25,New York, NY, USA,pp\. 589–600\.External Links:[Document](https://dx.doi.org/10.1145/3690624.3709194),ISBN 979\-8\-4007\-1245\-6Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p3.1)\.
- J\. Jin, Y\. Zhu, G\. Dong, Y\. Zhang, X\. Yang, C\. Zhang, T\. Zhao, Z\. Yang, Z\. Dou, and J\. Wen \(2025\)FlashRAG: A Modular Toolkit for Efficient Retrieval\-Augmented Generation Research\.InCompanion Proceedings of the ACM on Web Conference 2025,pp\. 737–740\.External Links:2405\.13576,[Document](https://dx.doi.org/10.1145/3701716.3715313.)Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p1.1),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p1.1)\.
- M\. Joshi, E\. Choi, D\. S\. Weld, and L\. Zettlemoyer \(2017\)TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension\.arXiv\.External Links:1705\.03551,[Document](https://dx.doi.org/10.48550/arXiv.1705.03551)Cited by:[§7\.4](https://arxiv.org/html/2608.03487#S7.SS4.p1.1)\.
- K\. Kandasamy, G\. Dasarathy, J\. Schneider, and B\. Poczos \(2017\)Multi\-fidelity Bayesian Optimisation with Continuous Approximations\.arXiv\.External Links:1703\.06240,[Document](https://dx.doi.org/10.48550/arXiv.1703.06240)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1)\.
- O\. Khattab and M\. Zaharia \(2020\)ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT\.arXiv\.External Links:2004\.12832,[Document](https://dx.doi.org/10.48550/arXiv.2004.12832)Cited by:[Figure 5](https://arxiv.org/html/2608.03487#S4.F5),[Figure 5](https://arxiv.org/html/2608.03487#S4.F5.5.2),[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p3.2)\.
- J\. Kim and D\. Mahajan \(2026\)VectorLiteRAG: Latency\-Aware and Fine\-Grained Resource Partitioning for Efficient RAG\.arXiv\.External Links:2504\.08930,[Document](https://dx.doi.org/10.48550/arXiv.2504.08930)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p5.1)\.
- J\. Knowles \(2006\)ParEGO: a hybrid algorithm with on\-line landscape approximation for expensive multiobjective optimization problems\.IEEE Transactions on Evolutionary Computation10\(1\),pp\. 50–66\.External Links:ISSN 1941\-0026,[Document](https://dx.doi.org/10.1109/TEVC.2005.851274)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1),[§4\.2\.1](https://arxiv.org/html/2608.03487#S4.SS2.SSS1.p5.1)\.
- A\. Kudva, W\. Tang, and J\. A\. Paulson \(2026\)Multi\-Objective Bayesian Optimization for Networked Black\-Box Systems: A Path to Greener Profits and Smarter Designs\.arXiv\.External Links:2502\.14121,[Document](https://dx.doi.org/10.48550/arXiv.2502.14121)Cited by:[§4\.2\.2](https://arxiv.org/html/2608.03487#S4.SS2.SSS2.p1.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient Memory Management for Large Language Model Serving with PagedAttention\.arXiv\.External Links:2309\.06180,[Document](https://dx.doi.org/10.48550/arXiv.2309.06180)Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p3.1)\.
- J\. Lao, Y\. Wang, Y\. Li, J\. Wang, Y\. Zhang, Z\. Cheng, W\. Chen, M\. Tang, and J\. Wang \(2025\)GPTuner: An LLM\-Based Database Tuning System\.SIGMOD Rec\.54\(1\),pp\. 101–110\.External Links:ISSN 0163\-5808,[Document](https://dx.doi.org/10.1145/3733620.3733641)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1),[§4\.2\.2](https://arxiv.org/html/2608.03487#S4.SS2.SSS2.Px1.p1.15)\.
- J\. Lévesque, A\. Durand, C\. Gagné, and R\. Sabourin \(2017\)Bayesian optimization for conditional hyperparameter spaces\.In2017 International Joint Conference on Neural Networks \(IJCNN\),pp\. 286–293\.External Links:ISSN 2161\-4407,[Document](https://dx.doi.org/10.1109/IJCNN.2017.7965867)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1),[§4\.2\.1](https://arxiv.org/html/2608.03487#S4.SS2.SSS1.p2.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InProceedings of the 34th International Conference on Neural Information Processing Systems,NIPS ’20,Red Hook, NY, USA,pp\. 9459–9474\.External Links:ISBN 978\-1\-7138\-2954\-6Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p1.1)\.
- S\. Li, Y\. Zhou, Y\. Xu, K\. Chen, D\. Waddington, S\. Sundararaman, H\. Franke, and J\. Huang \(2026\)RAGPerf: An End\-to\-End Benchmarking Framework for Retrieval\-Augmented Generation Systems\.arXiv\.External Links:2603\.10765,[Document](https://dx.doi.org/10.48550/arXiv.2603.10765)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p3.1),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p5.1)\.
- S\. Li, Y\. Zhao, R\. Varma, O\. Salpekar, P\. Noordhuis, T\. Li, A\. Paszke, J\. Smith, B\. Vaughan, P\. Damania, and S\. Chintala \(2020\)PyTorch Distributed: Experiences on Accelerating Data Parallel Training\.arXiv\.External Links:2006\.15704,[Document](https://dx.doi.org/10.48550/arXiv.2006.15704)Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p3.1)\.
- Y\. Li, W\. Zhang, Y\. Yang, W\. Huang, Y\. Wu, J\. Luo, Y\. Bei, H\. P\. Zou, X\. Luo, Y\. Zhao, C\. Chan, Y\. Chen, Z\. Deng, Y\. Li, H\. Zheng, D\. Li, R\. Jiang, M\. Zhang, Y\. Song, and P\. S\. Yu \(2025\)Towards Agentic RAG with Deep Reasoning: A Survey of RAG\-Reasoning Systems in LLMs\.arXiv\.External Links:2507\.09477,[Document](https://dx.doi.org/10.48550/arXiv.2507.09477)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p1.1)\.
- N\. Liang, F\. Wenz, J\. Giceva, and L\. W\. Wills \(2025\)Athena: A Plug\-and\-Play Advisor for Retrieval\-Augmented Generation using VectorDB\.In2025 IEEE International Symposium on Workload Characterization \(IISWC\),pp\. 28–41\.External Links:ISSN 2835\-2238,[Document](https://dx.doi.org/10.1109/IISWC66894.2025.00013)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p5.1)\.
- M\. Lindauer, K\. Eggensperger, M\. Feurer, A\. Biedenkapp, D\. Deng, C\. Benjamins, T\. Ruhopf, R\. Sass, and F\. Hutter \(2022\)SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization\.arXiv\.External Links:2109\.09831,[Document](https://dx.doi.org/10.48550/arXiv.2109.09831)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1),[§4\.2\.1](https://arxiv.org/html/2608.03487#S4.SS2.SSS1.p5.1),[§7\.3](https://arxiv.org/html/2608.03487#S7.SS3.p1.1),[Table 3](https://arxiv.org/html/2608.03487#S7.T3.6.6.3)\.
- T\. Liu, N\. Astorga, N\. Seedat, and M\. van der Schaar \(2024\)Large Language Models to Enhance Bayesian Optimization\.arXiv\.External Links:2402\.03921,[Document](https://dx.doi.org/10.48550/arXiv.2402.03921)Cited by:[§4\.2\.2](https://arxiv.org/html/2608.03487#S4.SS2.SSS2.Px1.p1.15)\.
- Z\. Lu, G\. Sreekumar, E\. Goodman, W\. Banzhaf, K\. Deb, and V\. N\. Boddeti \(2021\)Neural Architecture Transfer\.IEEE Transactions on Pattern Analysis and Machine Intelligence43\(9\),pp\. 2971–2989\.External Links:ISSN 1939\-3539,[Document](https://dx.doi.org/10.1109/TPAMI.2021.3052758)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p3.1)\.
- Y\. A\. Malkov and D\. A\. Yashunin \(2018\)Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs\.arXiv\.External Links:1603\.09320,[Document](https://dx.doi.org/10.48550/arXiv.1603.09320)Cited by:[§2\.1](https://arxiv.org/html/2608.03487#S2.SS1.p2.1),[Figure 5](https://arxiv.org/html/2608.03487#S4.F5),[Figure 5](https://arxiv.org/html/2608.03487#S4.F5.5.2)\.
- \[51\]\(2026\-01\)Marker\-Inc\-Korea/AutoRAG\.Note:Markr\.AICited by:[§1](https://arxiv.org/html/2608.03487#S1.p3.1),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p1.1),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p3.1),[§4\.2\.2](https://arxiv.org/html/2608.03487#S4.SS2.SSS2.p1.1),[§7\.3](https://arxiv.org/html/2608.03487#S7.SS3.p1.1),[Table 3](https://arxiv.org/html/2608.03487#S7.T3.2.2.3)\.
- A\. Novikov, N\. Vũ, M\. Eisenberger, E\. Dupont, P\. Huang, A\. Z\. Wagner, S\. Shirobokov, B\. Kozlovskii, F\. J\. R\. Ruiz, A\. Mehrabian, M\. P\. Kumar, A\. See, S\. Chaudhuri, G\. Holland, A\. Davies, S\. Nowozin, P\. Kohli, and M\. Balog \(2025\)AlphaEvolve: A coding agent for scientific and algorithmic discovery\.arXiv\.External Links:2506\.13131,[Document](https://dx.doi.org/10.48550/arXiv.2506.13131)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1)\.
- M\. Olson, E\. Santorella, L\. C\. Tiao, S\. Cakmak, M\. Garrard, S\. Daulton, Z\. J\. Lin, S\. Ament, B\. Beckerman, E\. Onofrey, P\. Igusti, C\. Lara, B\. Letham, C\. Cardoso, S\. S\. Shen, A\. C\. Lin, M\. Grange, E\. Kashtelyan, D\. Eriksson, M\. Balandat, and E\. Bakshy \(2025\)Ax: A Platform for Adaptive Experimentation\.InProceedings of the Fourth International Conference on Automated Machine Learning,pp\. 21/1–25\.External Links:ISSN 2640\-3498Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1)\.
- Z\. Pan, Q\. Wu, H\. Jiang, M\. Xia, X\. Luo, J\. Zhang, Q\. Lin, V\. Rühle, Y\. Yang, C\. Lin, H\. V\. Zhao, L\. Qiu, and D\. Zhang \(2024\)LLMLingua\-2: Data Distillation for Efficient and Faithful Task\-Agnostic Prompt Compression\.InFindings of the Association for Computational Linguistics: ACL 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 963–981\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.57)Cited by:[Figure 5](https://arxiv.org/html/2608.03487#S4.F5),[Figure 5](https://arxiv.org/html/2608.03487#S4.F5.5.2)\.
- B\. Paria, K\. Kandasamy, and B\. Póczos \(2019\)A Flexible Framework for Multi\-Objective Bayesian Optimization using Random Scalarizations\.arXiv\.External Links:1805\.12168,[Document](https://dx.doi.org/10.48550/arXiv.1805.12168)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1)\.
- Z\. Qi, R\. Xu, Z\. Guo, C\. Wang, H\. Zhang, and W\. Xu \(2025\)Long$^2$RAG: Evaluating Long\-Context & Long\-Form Retrieval\-Augmented Generation with Key Point Recall\.arXiv\.External Links:2410\.23000,[Document](https://dx.doi.org/10.48550/arXiv.2410.23000)Cited by:[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p2.2),[§7\.2](https://arxiv.org/html/2608.03487#S7.SS2.p7.2)\.
- S\. Ray, R\. Pan, Z\. Gu, K\. Du, S\. Feng, G\. Ananthanarayanan, R\. Netravali, and J\. Jiang \(2025\)METIS: Fast Quality\-Aware RAG Systems with Configuration Adaptation\.arXiv\.External Links:2412\.10543,[Document](https://dx.doi.org/10.48550/arXiv.2412.10543)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p3.1),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p1.1),[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p4.1)\.
- N\. Reimers and I\. Gurevych \(2019\)Sentence\-BERT: Sentence Embeddings using Siamese BERT\-Networks\.arXiv\.External Links:1908\.10084,[Document](https://dx.doi.org/10.48550/arXiv.1908.10084)Cited by:[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p3.2)\.
- M\. Shoeybi, M\. Patwary, R\. Puri, P\. LeGresley, J\. Casper, and B\. Catanzaro \(2020\)Megatron\-LM: Training Multi\-Billion Parameter Language Models Using Model Parallelism\.arXiv\.External Links:1909\.08053,[Document](https://dx.doi.org/10.48550/arXiv.1909.08053)Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p3.1)\.
- K\. Shuster, S\. Poff, M\. Chen, D\. Kiela, and J\. Weston \(2021\)Retrieval Augmentation Reduces Hallucination in Conversation\.InFindings of the Association for Computational Linguistics: EMNLP 2021,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Punta Cana, Dominican Republic,pp\. 3784–3803\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.findings-emnlp.320)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p1.1)\.
- A\. Singh, A\. Ehtesham, S\. Kumar, T\. T\. Khoei, and A\. V\. Vasilakos \(2026\)Agentic Retrieval\-Augmented Generation: A Survey on Agentic RAG\.arXiv\.External Links:2501\.09136,[Document](https://dx.doi.org/10.48550/arXiv.2501.09136)Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p1.1)\.
- S\. Siriwardhana, R\. Weerasekera, E\. Wen, T\. Kaluarachchi, R\. Rana, and S\. Nanayakkara \(2023\)Improving the Domain Adaptation of Retrieval Augmented Generation \(RAG\) Models for Open Domain Question Answering\.Transactions of the Association for Computational Linguistics11,pp\. 1–17\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00530)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p1.1)\.
- K\. Song, X\. Tan, T\. Qin, J\. Lu, and T\. Liu \(2020\)MPNet: Masked and Permuted Pre\-training for Language Understanding\.arXiv\.External Links:2004\.09297,[Document](https://dx.doi.org/10.48550/arXiv.2004.09297)Cited by:[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p3.2)\.
- T\. Vu, M\. Iyyer, X\. Wang, N\. Constant, J\. Wei, J\. Wei, C\. Tar, Y\. Sung, D\. Zhou, Q\. Le, and T\. Luong \(2024\)FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation\.InFindings of the Association for Computational Linguistics: ACL 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 13697–13720\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.813)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p1.1)\.
- J\. Wang, X\. Yi, R\. Guo, H\. Jin, P\. Xu, S\. Li, X\. Wang, X\. Guo, C\. Li, X\. Xu, K\. Yu, Y\. Yuan, Y\. Zou, J\. Long, Y\. Cai, Z\. Li, Z\. Zhang, Y\. Mo, J\. Gu, R\. Jiang, Y\. Wei, and C\. Xie \(2021\)Milvus: A Purpose\-Built Vector Data Management System\.InProceedings of the 2021 International Conference on Management of Data,SIGMOD ’21,New York, NY, USA,pp\. 2614–2627\.External Links:[Document](https://dx.doi.org/10.1145/3448016.3457550),ISBN 978\-1\-4503\-8343\-1Cited by:[§2\.1](https://arxiv.org/html/2608.03487#S2.SS1.p3.1)\.
- S\. Watanabe \(2026\)Tree\-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance\.arXiv\.External Links:2304\.11127,[Document](https://dx.doi.org/10.48550/arXiv.2304.11127)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1)\.
- C\. Xu, Q\. Sun, K\. Zheng, X\. Geng, P\. Zhao, J\. Feng, C\. Tao, Q\. Lin, and D\. Jiang \(2025\)WizardLM: Empowering large pre\-trained language models to follow complex instructions\.arXiv\.External Links:2304\.12244,[Document](https://dx.doi.org/10.48550/arXiv.2304.12244)Cited by:[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p2.2),[§7\.2](https://arxiv.org/html/2608.03487#S7.SS2.p7.2)\.
- A\. B\. Yandex and V\. Lempitsky \(2016\)Efficient Indexing of Billion\-Scale Datasets of Deep Descriptors\.In2016 IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 2055–2063\.External Links:ISSN 1063\-6919,[Document](https://dx.doi.org/10.1109/CVPR.2016.226)Cited by:[§7\.4](https://arxiv.org/html/2608.03487#S7.SS4.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: Synergizing Reasoning and Acting in Language Models\.arXiv\.External Links:2210\.03629,[Document](https://dx.doi.org/10.48550/arXiv.2210.03629)Cited by:[§2\.3](https://arxiv.org/html/2608.03487#S2.SS3.p1.1)\.
- G\. Yu, J\. S\. Jeong, G\. Kim, S\. Kim, and B\. Chun \(2022\)Orca: A Distributed Serving System for Transformer\-Based Generative Models\.In16th USENIX Symposium on Operating Systems Design and Implementation \(OSDI 22\),pp\. 521–538\.External Links:ISBN 978\-1\-939133\-28\-1Cited by:[§2\.2](https://arxiv.org/html/2608.03487#S2.SS2.p3.1)\.
- Q\. Zhang and H\. Li \(2007\)MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition\.IEEE Transactions on Evolutionary Computation11\(6\),pp\. 712–731\.External Links:ISSN 1941\-0026,[Document](https://dx.doi.org/10.1109/TEVC.2007.892759)Cited by:[§2\.4](https://arxiv.org/html/2608.03487#S2.SS4.p2.1)\.
- T\. Zhang, K\. Qu, Z\. Li, J\. Wu, M\. Hutter, M\. Li, and F\. Shi \(2026\)Using large language models for embodied planning introduces systematic safety risks\.arXiv\.External Links:2604\.18463,[Document](https://dx.doi.org/10.48550/arXiv.2604.18463)Cited by:[§2\.3](https://arxiv.org/html/2608.03487#S2.SS3.p1.1)\.
- T\. Zhang, Z\. Jiang, S\. Bai, T\. Zhang, L\. Lin, Y\. Liu, and J\. Ren \(2024\)RAG4ITOps: A Supervised Fine\-Tunable and Comprehensive RAG Framework for IT Operations and Maintenance\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track,F\. Dernoncourt, D\. Preoţiuc\-Pietro, and A\. Shimorina \(Eds\.\),Miami, Florida, US,pp\. 738–754\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-industry.56)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p1.1)\.
- K\. Zhu, Y\. Luo, D\. Xu, Y\. Yan, Z\. Liu, S\. Yu, R\. Wang, S\. Wang, Y\. Li, N\. Zhang, X\. Han, Z\. Liu, and M\. Sun \(2025\)RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework\.arXiv\.External Links:2408\.01262,[Document](https://dx.doi.org/10.48550/arXiv.2408.01262)Cited by:[§1](https://arxiv.org/html/2608.03487#S1.p1.1),[§1](https://arxiv.org/html/2608.03487#S1.p9.1),[§7\.1](https://arxiv.org/html/2608.03487#S7.SS1.p2.2),[§7\.4](https://arxiv.org/html/2608.03487#S7.SS4.p1.1)\.