Opti-Q: A Constraint-Based Optimization Framework for Multi-LLM Question Planning

# Opti-Q: A Constraint-Based Optimization Framework for Multi-LLM Question Planning
Source: [https://arxiv.org/html/2607.22621](https://arxiv.org/html/2607.22621)
Aamir Hamid1, Bharg Barot1, Satvik Racharla1, Tim Finin1, Primal Pappachan2, and Roberto Yus1 1University of Maryland, Baltimore County, USA\.2Portland State University, USA\. \{ahamid2, bhargvb1, yv04378, finin, ryus\}@umbc\.edu, primal@pdx\.edu

###### Abstract

While large language models \(LLMs\) enable strong question answering \(QA\), budgeted deployment is complicated by nondeterminism and heterogeneous resource profiles \(cost, latency, and energy\)\. We presentOpti\-Q, a database\-inspired, cost\-based optimizer that implements a*plan\-before\-execute*paradigm for multi\-LLM orchestration\.Opti\-Qmodels LLM invocations as physical operators in an execution DAG and, for each question, searches for plans that optimize answer quality \(QoA\) while trading off financial cost, latency, and energy under user\-specified resource constraints\. Plans can include sequential operators that pass intermediate answers as context and parallel/blend operators that run models concurrently and merge their outputs\. To search this space without executing each candidate plan,Opti\-QusesPerfDB, a statistics catalog populated and refreshed from benchmarks and execution traces, to estimate the QoA and resource costs of both individual operators and composed subplans\. Using these estimates,Opti\-Qperforms Pareto\-frontier search and selects a final plan based on user preferences\. On MMLU\-Pro and SimpleQA under user\-specified budgets,Opti\-Qimproves average QoA by≈58%\\approx 58\\%and≈41%\\approx 41\\%over baselines at comparable cost, demonstrating that database\-style planning yields better quality–resource trade\-offs for multi\-LLM QA\.

## IIntroduction

Large Language Models \(LLMs\) such as GPT\-5\[[1](https://arxiv.org/html/2607.22621#bib.bib1)\], Llama 3\[[2](https://arxiv.org/html/2607.22621#bib.bib2)\], and DeepSeek\-R1\[[3](https://arxiv.org/html/2607.22621#bib.bib3)\]have transformed natural language processing \(NLP\)\[[4](https://arxiv.org/html/2607.22621#bib.bib4)\], driving advances in question\-answering \(QA\)\[[5](https://arxiv.org/html/2607.22621#bib.bib5)\], text summarization\[[6](https://arxiv.org/html/2607.22621#bib.bib6)\]and domain\-specific reasoning\[[7](https://arxiv.org/html/2607.22621#bib.bib7)\]\. Their success has fueled adoption across industries, from customer service to healthcare\[[8](https://arxiv.org/html/2607.22621#bib.bib8),[9](https://arxiv.org/html/2607.22621#bib.bib9)\]\. However, real\-world deployment remains challenging due to nondeterminism, leading to hallucinations\[[10](https://arxiv.org/html/2607.22621#bib.bib10)\], high computational costs\[[11](https://arxiv.org/html/2607.22621#bib.bib11)\], substantial energy consumption\[[12](https://arxiv.org/html/2607.22621#bib.bib12)\], and latency trade\-offs\[[13](https://arxiv.org/html/2607.22621#bib.bib13)\]\. Considerable effort has focused on these issues, including increasing accuracy and reliability through advanced prompting strategies \(e\.g\., Chain\-of\-Thought prompting\) or extensive fine\-tuning\[[14](https://arxiv.org/html/2607.22621#bib.bib14)\]\. However, a fundamental challenge persists: different LLMs exhibit varying performance across question/task types\[[15](https://arxiv.org/html/2607.22621#bib.bib15)\]\. Moreover, recent evidence suggests that a single high\-capability LLM is not always optimal; coordinated multi\-LLM collaboration can outperform the single “best” model on real\-world tasks\[[16](https://arxiv.org/html/2607.22621#bib.bib16)\]\.

Inspired by machine learning \(ML\) ensembles\[[17](https://arxiv.org/html/2607.22621#bib.bib17)\], recent work explores combining multiple LLMs to improve reliability and answer quality\[[18](https://arxiv.org/html/2607.22621#bib.bib18),[19](https://arxiv.org/html/2607.22621#bib.bib19)\], as cross\-model validation helps mitigate hallucinations and raise response quality\[[20](https://arxiv.org/html/2607.22621#bib.bib20)\]\. However, naive execution strategies \(e\.g\., always querying all models and combining \(blending\)\[[21](https://arxiv.org/html/2607.22621#bib.bib21)\]their outputs, or always choosing a fixed “strong” subset of models\) introduce new challenges: higher financial costs and latency, inefficient resource use, and sometimes lower answer quality\. Moreover, while many systems rely on dynamic sequential orchestration, where later model choices depend on earlier model outputs \(e\.g\., via LangChain\-style chains\)\[[22](https://arxiv.org/html/2607.22621#bib.bib22)\]or execution\-time optimization for cost/throughput\[[23](https://arxiv.org/html/2607.22621#bib.bib23)\]\. Such decisions can be myopic when made without planning for downstream consequences\. For example, in our experiments, a history question was best answered by a plan using Gemma\-3:27B, Qwen\-2\.5:14B, and Phi\-4:14B \(outperforming other plans by a 1\.24x to 1\.48× factor\), whereas a sports question was best handled by a simple parallel execution of Gemma\-3:27B and Qwen\-2\.5:14B\. These variations show that the optimal model combination and execution strategy are question\-dependent\. A robust multi\-LLM QA system must therefore determine the best execution plan dynamically for each question, while accounting for end\-to\-end cost, latency, energy, and quality\.

Database systems face a similar problem where the same declarative query can be executed by many alternative physical plans, and the optimizer uses a cost model and statistics to select a plan before execution\[[24](https://arxiv.org/html/2607.22621#bib.bib24)\]\. In Multi\-LLM QA, many alternative workflows may be possible for a user question, each trading off monetary cost, latency, energy, and answer quality\. We therefore cast multi\-LLM orchestration as a cost\-based \(and multi\-objective\) query\-planning problem, where the question serves as the query, each LLM invocation is a physical operator, and an end\-to\-end workflow is a physical plan whose selection is guided by a model “catalog” of historical performance statistics\. Unlike classical query optimization, multi\-LLM planning must account for stochastic and semantic operator outputs \(quality is uncertain and non\-monotonic\), output\-dependent downstream costs \(e\.g\., token lengths and content can amplify later latency/cost/energy\), and inherently multi\-dimensional objectives \(quality–cost–latency–energy\) under user constraints\. This necessitates new cost estimation models and multi\-objective plan selection beyond traditional single\-metric cost minimization\.

To tackle these challenges, we introduceOpti\-Q, a cost\-based optimizer for multi\-LLM QA that selects an execution plan*before*issuing any model calls, mirroring how DBMS optimizers choose physical query plans prior to execution\. As illustrated in Figure[1](https://arxiv.org/html/2607.22621#S3.F1), the plans generated byOpti\-Qinvolve combinations of parallel and sequential multi\-LLM operations\. To select among candidate plans, we propose a technique to estimate, before execution, a plan’s cost \(financial, latency, energy\) and benefit \(answer quality\) from historical performance statistics on LLMs and their combinations across question types\. Using these estimates,Opti\-Qidentifies plans representing optimal trade\-offs among competing objectives under optional user constraints via multi\-objective optimization \(MOO\)\. Given the potentially large plan space, which depends on the number of LLMs considered and the complexity of the plan structure,Opti\-Qemploys a “pluggable optimization engine” that supports diverse planning strategies, including dynamic programming \(DP\), Hill Climbing, and Non\-dominated Sorting Genetic Algorithm \(NSGA\-II\) to explore candidate plans and select one for execution\. This enablesOpti\-Qto dynamically balance performance and resource efficiency before committing to costly LLM invocations\. The main contributions of this work are:

- •A cost/benefit formulation of multi\-LLM QA planning to select a workflow under budget and resource constraints\.
- •A statistics\-driven optimizer that enumerates and prunes sequential/parallel/hybrid workflows and estimates quality and resource costs prior to execution\.
- •A system that integrates the optimizer with an execution engine for real\-time routing across open\-source LLMs\.

We evaluate our approach on two QA benchmarks, one open\-ended and one multiple\-choice, and compare it against four state\-of\-the\-art baselines\. In both,Opti\-Qachieves superior QoA\-cost efficiency, improving the average QoA by≈58%\\approx 58\\%and≈41%\\approx 41\\%over the strongest budget\-aware baseline at matched per\-question cost \(budget level b=3\)\. We further analyze the impact of LLM diversity, plan complexity, and budget levels on the trade\-offsOpti\-Qdiscovers\.

## IIRelated Work

Recent work at the intersection of data management and LLMs spans two largely orthogonal directions\. The first studies how LLMs can assist database query optimization \(e\.g\., improving cardinality estimation\[[25](https://arxiv.org/html/2607.22621#bib.bib25)\], cost modeling\[[26](https://arxiv.org/html/2607.22621#bib.bib26)\], or physical operator selection\[[27](https://arxiv.org/html/2607.22621#bib.bib27)\]\)\. This topic focuses on optimizing database queries over structured data which differs from our setting, where the challenge is to select and compose multiple LLM calls to answer a natural\-language question under quality–cost–latency–energy trade\-offs\. The second direction is more related to our work and treats LLM calls as expensive, probabilistic operators in query\-like workflows, asking how to optimize these AI\-powered workflows using system and optimizer principles\. Orthogonal to this, orchestration frameworks such as LangChain111https://github\.com/langchain\-ai/langchainand DSPy\[[28](https://arxiv.org/html/2607.22621#bib.bib28)\]support multi\-step flows but leave plan structure and physical choices largely developer\-scripted, making optimization implicit\.

Early approaches to LLM call orchestration adopt*cascading*, routing requests through models of increasing capacity\. FrugalGPT\[[29](https://arxiv.org/html/2607.22621#bib.bib29)\]typifies this paradigm by learning a budget\-aware cascade that reduces cost while maintaining accuracy\. However, cascades often impose a largely fixed structure and can accumulate sequential latency; moreover, their intermediate generations can inflate downstream token usage \(and thus cost/latency\), making greedy “start\-cheap” strategies globally suboptimal\. Other ensemble methods, such as LLM\-Blender\[[21](https://arxiv.org/html/2607.22621#bib.bib21)\]and LLM\-TOPLA\[[30](https://arxiv.org/html/2607.22621#bib.bib30)\], improve quality by fusing multiple outputs, often assuming that additional inference cost is acceptable\. Building on these insights, adaptive*question\-time*orchestration chooses models conditioned on the input\. ThriftLLM\[[31](https://arxiv.org/html/2607.22621#bib.bib31)\]learns per\-question ensemble selection within a budget, while Shekhar et al\.\[[32](https://arxiv.org/html/2607.22621#bib.bib32)\]predict quality pre\-inference to choose cost\-effective models under latency constraints for summarization\. These methods align computation with question difficulty, but they typically \(i\) optimize a single dominant objective and/or \(ii\) treat model invocations as largely independent decisions rather than as a structured plan with interactions among heterogeneous LLMs\.

Complementary recent work formalizes LLM calls as operators in declarative data systems, enabling optimizer\-style reasoning over AI pipelines\. Galois\[[33](https://arxiv.org/html/2607.22621#bib.bib33)\]emphasizes logical optimizations, such as pushing down filters to reduce expensive LLM calls, while frameworks such as PALIMPZEST\[[23](https://arxiv.org/html/2607.22621#bib.bib23)\], Abacus\[[34](https://arxiv.org/html/2607.22621#bib.bib34)\], and Stretto\[[35](https://arxiv.org/html/2607.22621#bib.bib35)\]develop cost\-based optimization, constrained objectives, and execution\-time trade\-offs\.Opti\-Qdiffers from these declarative systems along two key architectural dimensions\. First, prior systems primarily optimize data\-processing pipelines over records, documents, or multimodal corpora by selecting physical implementations for a given declarative program or workflow\.Opti\-Qinstead shifts the optimization boundary to online, per\-question planning, dynamically generating and selecting sequential, parallel, or hybrid execution graphs tailored to an individual incoming question\. Second, while several existing frameworks handle resource\-quality trade\-offs through scalarized objectives or fixed workflow templates,Opti\-Qtreats QoA, cost, latency, and energy as distinct first\-class objectives and explicitly explores Pareto trade\-offs at question time to support user\-specified priorities and resource constraints\.

## IIIOpti\-Q Multi\-LLM Planning

This section formalizes user questions, LLMs, and multi\-LLM question plans\. Then, it postulates the selection of an optimal plan for a given question under QoA, financial, latency, and energy constraints as a multi\-objective optimization \(MOO\) problem\. Finally, it sketches our approach\.

### III\-AModeling Multi\-LLM Planning

Question model\.We model each user question as a tupleQ=\(pt,T,Fmax,Lmax,Emax,QoAmin,W\),Q=\(p\_\{t\},\\,T,\\,F\_\{\\text\{max\}\},\\,L\_\{\\text\{max\}\},\\,E\_\{\\text\{max\}\},\\,\\mathrm\{QoA\}\_\{\\text\{min\}\},\\,W\),whereptp\_\{t\}is the textual prompt, explicitly formulated as a question \(e\.g\., “Who are currently the oldest and youngest presidents of an EU country?”\),TTrepresents its topic\(s\) \(e\.g\., “politics”\)\. The parametersFmaxF\_\{\\text\{max\}\},LmaxL\_\{\\text\{max\}\},EmaxE\_\{\\text\{max\}\}, andQ​o​AminQoA\_\{\\text\{min\}\}specify user constraints that represent the maximum allowable total financial cost \(USD/question\), maximum total latency \(seconds/question\), maximum energy usage \(J/question\), and minimum acceptable QoA222We use the generic termQoAto denote any metric to assess model performance depending on the task and dataset, such as accuracy, BLEU, F1, or embedding\-based similarity measures \(e\.g\., cosine similarity\)\.\(on a scale 0\-1, 1 being the highest\)\. We collectively refer to these as thebudgetℬ\\mathcal\{B\}\. The weight vectorW=\(wF,wL,wE,wQoA\)W=\(w\_\{F\},w\_\{L\},w\_\{E\},w\_\{\\mathrm\{QoA\}\}\), with∑iwi=1\\sum\_\{i\}w\_\{i\}=1, encodes relative importance across objectives\. Importantly,ℬ\\mathcal\{B\}defines*feasibility*\(candidate plans must satisfy the user\-constraints\), whereasWWis applied only after planning to select one plan from the Pareto\-optimal set;WWis not used to scalarize objectives during plan search\. In practice, users do not need to specify all constraints manually\.Opti\-Qcan expose presets such as Cost\-saving, Balanced, High\-quality, Low\-latency, and Energy\-aware, which translate internally into default bounds and objective weights\. We distinguish between two question types: \(1\) those withbinary correctness\(e\.g\., multiple\-choice or yes/no questions\), whereQ​o​A∈\{0,1\}QoA\\in\\\{0,1\\\}; and \(2\) those withopen\-ended or free\-textanswers, whereQ​o​A∈\[0,1\]QoA\\in\[0,1\]reflects graded partial correctness \(e\.g\., “John Smith III” vs\. “J\. Smith”\)\.Opti\-Q’s implementation provides estimators for QoA in both cases\.

LLM Model\.LetL=\{L1,L2,…,Ln\}L=\\\{L\_\{1\},L\_\{2\},\\dots,L\_\{n\}\\\}be the available LLMs\. EachLiL\_\{i\}is represented by the tuple\(fi,li,ei,𝑞𝑜𝑎i,𝑇𝑜𝑘i,Tiout\)\(f\_\{i\},l\_\{i\},e\_\{i\},\\mathit\{qoa\}\_\{i\},\\mathit\{Tok\}\_\{i\},T\_\{i\}^\{\\text\{out\}\}\), wherefif\_\{i\},lil\_\{i\}, andeie\_\{i\}denote the expected financial cost, latency, and energy consumption of invokingLiL\_\{i\}under an execution context;𝑇𝑜𝑘i​\(⋅\)\\mathit\{Tok\}\_\{i\}\(\\cdot\)is the model\-specific tokenizer that maps a promptptp\_\{t\}to its input\-token count𝑇𝑜𝑘i​\(pt\)\\mathit\{Tok\}\_\{i\}\(p\_\{t\}\)\(e\.g\., GPT\-4 vs\. Llama\-3 use different tokenizers\); andTioutT\_\{i\}^\{\\text\{out\}\}is the expected number of output tokens generated byLiL\_\{i\}under the same context\. We useξ\\xito denote the*execution context*\(e\.g\., topic, operator type, and model\)\. At planning time, Opti\-Q instantiates the context\-conditioned metrics𝑞𝑜𝑎i​\(ξ\)\\mathit\{qoa\}\_\{i\}\(\\xi\),fi​\(ξ\)f\_\{i\}\(\\xi\),li​\(ξ\)l\_\{i\}\(\\xi\),ei​\(ξ\)e\_\{i\}\(\\xi\), andTiout​\(ξ\)T\_\{i\}^\{\\text\{out\}\}\(\\xi\)333We may omit explicitly mentioningξ\\xiwhen it is clear from context\.using the corresponding empirical estimates retrieved fromPerfDB, rather than assuming a single global quality value per model\. These metrics characterize the expected cost and behavior of model invocations within a plan\. Each invocation cost includes a fixed and a variable component dependent on the number of input and output tokens underξ\\xi\(e\.g\.,fi​\(ξ,pt\)=fifix​\(ξ\)\+fivar​\(ξ\)⋅\(𝑇𝑜𝑘i​\(pt\)\+Tiout​\(ξ\)\)\.f\_\{i\}\(\\xi,p\_\{t\}\)=f\_\{i\}^\{\\text\{fix\}\}\(\\xi\)\+f\_\{i\}^\{\\text\{var\}\}\(\\xi\)\\cdot\\bigl\(\\mathit\{Tok\}\_\{i\}\(p\_\{t\}\)\+T\_\{i\}^\{\\text\{out\}\}\(\\xi\)\\bigr\)\.Here,fifix​\(ξ\)f\_\{i\}^\{\\text\{fix\}\}\(\\xi\)captures the minimum per\-invocation cost, regardless of the specific question\. To handle stochastic intermediate output lengths,PerfDBstores the expected output lengthTiout​\(ξ\)T\_\{i\}^\{\\text\{out\}\}\(\\xi\)estimated from historical traces under the same contextξ\\xi, which Opti\-Q uses to estimate token\-dependent plan costs during planning\.

Plan Model\.For a questionQQ, letΠ\\Pidenote the set of admissible plans\. A planπ∈Π\\pi\\in\\Pispecifies how one or more LLMs fromLLare invoked to processQQ\. Each plan comprises operations\{op1,…,opn\}\\\{\\text\{op\}\_\{1\},\\dots,\\text\{op\}\_\{n\}\\\}, where eachopi\\text\{op\}\_\{i\}represents either an LLM invocation or a composition of LLMs, and a partial order≺\\precdefining execution dependencies\. Ifopi≺opj\\text\{op\}\_\{i\}\\prec\\text\{op\}\_\{j\}, thenopj\\text\{op\}\_\{j\}consumes the output ofopi\\text\{op\}\_\{i\}\. Initial operations, those with no predecessors, consumeQQ; final operations produce the plan’s answer\. Operations without order constraints will execute in parallel, and all operation outputs must either feed into subsequent operations or produce final answers\. For example, a plan involving operationsop1,op2,op3\\text\{op\}\_\{1\},\\text\{op\}\_\{2\},\\text\{op\}\_\{3\}may specifyop1≺op2\\text\{op\}\_\{1\}\\prec\\text\{op\}\_\{2\}, where bothop1\\text\{op\}\_\{1\}andop3\\text\{op\}\_\{3\}execute in parallel andop2\\text\{op\}\_\{2\}consumesop1\\text\{op\}\_\{1\}’s output\. A*sequential operation*, denotedSeq​\(Q,L1,…,Lk\)=Lk​\(Q⊕pc​t​x⊕Lk−1​\(Q⊕pc​t​x⊕⋯⊕L1​\(Q\)\)\)\\text\{Seq\}\(Q,L\_\{1\},\\dots,L\_\{k\}\)=L\_\{k\}\\left\(Q\\oplus p\_\{ctx\}\\oplus L\_\{k\-1\}\\left\(Q\\oplus p\_\{ctx\}\\oplus\\dots\\oplus L\_\{1\}\(Q\)\\right\)\\right\), concatenates \(⊕\\oplus\) the question, a context promptp​tc​t​xpt\_\{ctx\}\(that instructs the LLM on how to use it as a context; see Section[V](https://arxiv.org/html/2607.22621#S5)\), andL1L\_\{1\}’s output for input toL2L\_\{2\}\. A*parallel operation*, denotedPar​\(Q,\{L1,…,Lk\}\)=L1​\(Q\)⊙L2​\(Q\)⊙⋯⊙Lk​\(Q\)\\text\{Par\}\(Q,\\\{L\_\{1\},\\dots,L\_\{k\}\\\}\)=L\_\{1\}\(Q\)\\odot L\_\{2\}\(Q\)\\odot\\dots\\odot L\_\{k\}\(Q\), posesQQto each model independently and uses structured output fusion \(⊙\\odot\) to combine their outputs\.⊙\\odotsignifies a “concatenation” in which the output of each model is prefixed with its identifier for traceability \(e\.g\., “LLM1: answer1; LLM2: answer2”\)\. A*blending operation*combines multiple model outputs into a single answer\[[21](https://arxiv.org/html/2607.22621#bib.bib21)\]\. This operation, denotedBlend​\(Q,Par​\(Q,…\),Lb\)=Lb​\(Q⊕p​tb​l​d⊕Par​\(Q,…\)\)\\text\{Blend\}\(Q,\\text\{Par\}\(Q,\\dots\),L\_\{b\}\)=L\_\{b\}\(Q\\ \\oplus pt\_\{bld\}\\ \\oplus\\text\{Par\}\(Q,\\dots\)\), uses a dedicated “blending” modelLbL\_\{b\}, which takes the original question, a blending promptpb​l​dp\_\{bld\}, and the concatenated outputs of the parallel operation as input \(Figure[1](https://arxiv.org/html/2607.22621#S3.F1)shows an example; more details on the prompt in Section[V](https://arxiv.org/html/2607.22621#S5)\)\. Blending operations may appear*only*after parallel operations, and each parallel operation must be immediately followed by exactly one blending operation\. Note that the same model may appear multiple times in both sequential and parallel operations, soSeq​\(Q,L1,L1\)\\text\{Seq\}\(Q,L\_\{1\},L\_\{1\}\)andPar​\(Q,\{L2,L2\}\)\\text\{Par\}\(Q,\\\{L\_\{2\},L\_\{2\}\\\}\)are valid\.

Multi\-LLM QA Planning Problem\.Letπ∈Π\\pi\\in\\Pibe a plan using a subset of LLMs ofLLto processQQ\. This plan induces total financial, latency, and energy consumption costs and obtains a QoA based on the chosen models and their composition\. Our goal is to find Pareto\-optimal plans balancing QoA, financial cost, latency, and energy under user constraints\. Formally, we frame this as a MOO problem:

Maximize\[QoA​\(π\),−Financial​\(π\),−Latency​\(π\),−Energy​\(π\)\]Subject to:\{Financial​\(π\)≤FmaxLatency​\(π\)≤LmaxEnergy​\(π\)≤EmaxQoA​\(π\)≥QoAmin\\begin\{aligned\} \\text\{Maximize\}\\quad&\\big\[\\mathrm\{QoA\}\(\\pi\),\-\\mathrm\{Financial\}\(\\pi\),\-\\mathrm\{Latency\}\(\\pi\),\-\\mathrm\{Energy\}\(\\pi\)\\big\]\\\\ \\text\{Subject to:\}\\quad&\\begin\{cases\}\\mathrm\{Financial\}\(\\pi\)\\leq F\_\{\\max\}\\\\ \\mathrm\{Latency\}\(\\pi\)\\leq L\_\{\\max\}\\\\ \\mathrm\{Energy\}\(\\pi\)\\leq E\_\{\\max\}\\\\ \\mathrm\{QoA\}\(\\pi\)\\geq\\mathrm\{QoA\}\_\{\\min\}\\end\{cases\}\\end\{aligned\}

### III\-BOverview ofOpti\-QApproach

For a given questionQQ, our approach selects the planπ∈Π\\pi\\in\\Pithat best balances QoA against financial, latency, and energy costs\. As shown in Figure[1](https://arxiv.org/html/2607.22621#S3.F1), consider the example question,“Select all that apply: Which of the following are part of the Kingdom of Denmark? A\) Faroe Islands B\) Greenland C\) Svalbard D\) Lofoten Islands E\) Aland Islands ”\. First,Opti\-Qperforms aquestion parsingphase to extract the prompt textp​tpt, and topic\(s\)TT\(detailed in Section[V](https://arxiv.org/html/2607.22621#S5)\), along with user\-defined constraints and priorities \(budgetℬ\\mathcal\{B\}and weight vectorWW\)\. We require the user to specifyℬ\\mathcal\{B\}; maximizing QoA “at any cost” is supported but not our focus\. If𝐖\\mathbf\{W\}is omitted, we default to uniform weights\.

![Refer to caption](https://arxiv.org/html/2607.22621v1/figures/FArch.png)Figure 1:Comparison of candidate multi\-LLM execution plans for a question\.Bounding the plan size\.Next,Opti\-Qdetermines the maximum number of operations allowed per plan,kk, based onℬ\\mathcal\{B\}\. For eachLiL\_\{i\},Opti\-Qcomputes the maximum model invocationsnin\_\{i\}allowed under each constraint \(dividing the relevant budget by model’s per\-constraint cost\) and taking the minimum across constraints\. For example, the budget in Figure[1](https://arxiv.org/html/2607.22621#S3.F1)constrains a resource\-heavy LLM,L1L\_\{1\}, to one call while allowing a lighterL2L\_\{2\}up to three calls\. Limits are not mutually exclusive: a plan may callL1L\_\{1\}once and still useL2L\_\{2\}one or more times within the remaining budget\.Opti\-Qsampleskkuniformly from\[m​i​ni​\(ni\),m​a​xi​\(ni\)\]\\left\[min\_\{i\}\(n\_\{i\}\),max\_\{i\}\(n\_\{i\}\)\\right\]; as shown in our evaluation and in traditional ML ensemble size selection methods\[[36](https://arxiv.org/html/2607.22621#bib.bib36)\], this randomized choice within the interval does not harm performance\.

Performance Statistics \(PerfDB\)\.Opti\-QusesPerfDB, a statistical catalog that stores quality and resource statistics for individual LLM invocations and composed plans\.PerfDBenables the planner to estimate plan metrics \(i\.e\., latency, financial cost, QoA, and energy\) without executing the plan\.PerfDBis organized into levels of compositional depth \(i\.e\., operator\-nesting depth in the plan representation\), which determines the granularity of stored statistics\. Level 0 contains statistics for single LLMs obtained from public benchmarks \(including ML\.Energy\[[37](https://arxiv.org/html/2607.22621#bib.bib37)\]and Artificial Analysis444https://artificialanalysis\.ai/\. Level 1 aggregates results of single operations with “depth 1”, e\.g\.,Seq​\(Q,L1,L2\)\\text\{Seq\}\(Q,L\_\{1\},L\_\{2\}\)orPar​\(Q,\{L1,L2,L3\}\)\\text\{Par\}\(Q,\\\{L\_\{1\},L\_\{2\},L\_\{3\}\\\}\)\. Level 2 and higher, store measurements for subsets of plans with “depth 2” or higher \(e\.g\.,Blend​\(Q,Seq​\(Q,L1,L2\),Par​\(Q,\{L3,L4\}\),Lb\)\\text\{Blend\}\(Q,\\text\{Seq\}\(Q,L\_\{1\},L\_\{2\}\),\\text\{Par\}\(Q,\\\{L\_\{3\},L\_\{4\}\\\}\),L\_\{b\}\)\)\. All entries are keyed by execution contextξ\\xi, including topic, operator type, model, prompt configuration, and serving backend, thus capturing both workload\-dependent behavior and backend\-specific latency, energy, and throughput profiles\.PerfDBis populated offline and incrementally after execution\. For each executed plan, OPTI\-Q appends a compact trace containing the execution context, plan structure, invocation configuration, token usage, and*observed*resource metrics\. Cost and latency are obtained from API responses and execution logs\. Energy is recorded when exposed by the backend, or*estimated*from token counts and prior measurements otherwise\. QoA is updated only when quality feedback is available, such as benchmark labels, known reference answers, analyst judgments, user feedback, or automatic quality estimators\. Thus,PerfDBstores*observed*values when available and*estimated*values otherwise\. Since each execution contributes one compact trace, storage grows linearly with the number of logged executions, andPerfDBdoes not need to be rebuilt from scratch\. New models, fine\-tuned checkpoints, prompt variants, and serving backends are represented as distinct configured LLM invocation instances with initially sparse statistics;Opti\-Qbootstraps these instances from metadata or similar invocations and refines their estimates as new benchmark runs and live traces accumulate\. Because LLM outputs and serving behavior may vary across runs,PerfDBalso stores variance estimates and confidence intervals, enabling risk\-aware planning when reliability is important\. Section VI analyzes how varyingPerfDBcoverage affects planning performance\.

Plan Generation\.In this phase,Opti\-Qgenerates plans with up tokkoperations\. Because the plan space grows exponentially in\|L\|\|L\|andkk\(as we will detail in the following section\), an exhaustive search is infeasible\.Opti\-Qtherefore supports a “pluggable optimization engine” that includes diverse planning strategies \(Section[IV](https://arxiv.org/html/2607.22621#S4)\): \(i\) Dynamic Programming \(DP\), an exact solver that can recover the Pareto frontier for small\|L\|\\left\|L\\right\|and k but suffers from state\-space explosion; \(ii\) stochastic Hill Climbing \(HC\), a lightweight greedy heuristic with low planning latency that can stagnate in local optima in our non\-convex multi\-objective landscape; and \(iii\) NSGA\-II\[[38](https://arxiv.org/html/2607.22621#bib.bib38)\], a multi\-objective evolutionary algorithm that balances exploration and exploitation\. This design supports exact optimization when feasible, lightweight heuristics otherwise, andNSGA\-IIas the default scalable Pareto\-front approximator\.

Plan Execution Phase\.To execute the selected plan, operations are carried out according to the plan topology, applying context propagation and blending where applicable\. For example, consider the execution of*Plannn*in Figure[1](https://arxiv.org/html/2607.22621#S3.F1), which includes a parallel call followed by blending, i\.e\.,Blend⁡\(Q,Par⁡\(Q,L3,L4\),Lb\)\\operatorname\{Blend\}\(Q,\\operatorname\{Par\}\(Q,L\_\{3\},L\_\{4\}\),L\_\{b\}\), and is selected in this case\. ModelsL3L\_\{3\}andL4L\_\{4\}run in parallel to answer the multi\-answer user prompt\. SupposeL3L\_\{3\}returns a partial but correct subset \(e\.g\., \(A\) Faroe Islands\), whileL4L\_\{4\}returns \(B\) Greenland but also includes a plausible yet incorrect extra option \(e\.g\., \(C\) Svalbard\)\. The blenderLbL\_\{b\}aggregates the candidate sets, cross\-checks them against the question constraints, removes unsupported options, and produces a consolidated final prediction \(A, B\) \(Faroe Islands; Greenland\)\. This simple example illustrates the strength of multi\-objective planning: a single\-model plan \(onlyL4L\_\{4\}\) may be cheaper but less robust, risking error propagation\. We quantify variability due to nondeterminism by repeated executions and confidence intervals in Section[VI](https://arxiv.org/html/2607.22621#S6)\.

## IVPlan Generation

To answer a question using multiple LLMs, we must \(i\) represent candidate execution plans in a compact form that supports efficient manipulation, \(ii\) estimate a plan’s cost–benefit*without*executing it, and \(iii\) search the resulting plan space to find high\-quality tradeoffs\.

### IV\-APlan Encoding

All planners in Opti\-Q operate over the same compact, yet expressive encoding candidate planπ\\pi\. We representπ\\pias a tuple\(connectivity\_map,𝐦\)\(\\textit\{connectivity\\\_map\},\\mathbf\{m\}\), whereconnectivity\_mapcanonically encodes the plan’s directed acyclic execution topology and𝐦\\mathbf\{m\}assigns one model to each of thekknodes\. Figure[2](https://arxiv.org/html/2607.22621#S4.F2)shows an example plan with its encoding\. The DAG topology overkkoperations is encoded by the upper\-triangular adjacency matrixM∈\{0,1\}k×kM\\in\\\{0,1\\\}^\{k\\times k\}, withMi​j=1M\_\{ij\}=1for\(1≤i<j≤k\)\(1\\leq i<j\\leq k\)iff there is a data\-flow edge fromLiL\_\{i\}toLjL\_\{j\}\. For computational efficiency, we flattenMMinto ak​\(k−1\)/2k\(k\-1\)/2\-bit vectorB=\(b12,b13,…,b\(k−1\)​k\)B=\(b\_\{12\},b\_\{13\},\\dots,b\_\{\(k\-1\)k\}\), where each bitbi​j∈\{0,1\}b\_\{ij\}\\in\\\{0,1\\\}indicates whetherLiL\_\{i\}feeds its output toLjL\_\{j\}\. Blending operations are implicit: a designated blending model \(LbL\_\{b\}\) is invoked to combine inputs if a nodeLjL\_\{j\}has an in\-degreedeg−⁡\(Lj\)\>1\\deg^\{\-\}\(L\_\{j\}\)\>1\. To ensure invariance under node relabeling \(structural isomorphism\), we enumerate all valid topological orderings of the DAG and select the lexicographically maximal bit vector,BmaxB\_\{\\text\{max\}\}, defining the canonical integer identifier asconnectivity\_map=integer⁡\(Bmax\)\\textit\{connectivity\\\_map\}=\\operatorname\{integer\}\(B\_\{\\text\{max\}\}\)\. This encoding method guarantees acyclicity by construction, facilitates efficient manipulation, and provides a unique, isomorphism\-aware representation of complex execution flows\. We enforce the constraint that the final node must have an in\-degree≥1\\geq 1, ensuring the plan produces a valid output\.

![Refer to caption](https://arxiv.org/html/2607.22621v1/figures/DAG_View.png)Figure 2:Sample question plan encoding\.Figure[2](https://arxiv.org/html/2607.22621#S4.F2)\(A\) illustrates an execution plan composed ofk=5k=5operations constructed from Sequential, Parallel, and Blending operations, while Figure[2](https://arxiv.org/html/2607.22621#S4.F2)\(B\) and \(C\) show its connectivity matrix and model assignments, respectively\. The plan begins with a parallel operation where modelsL1L\_\{1\}andL2L\_\{2\}process the questionQQ, defined aso​p1=Par​\(Q,\(L1,L2\)\)op\_\{1\}=\\text\{Par\}\(Q,\(L\_\{1\},L\_\{2\}\)\)\. Their outputs are then blended using modelLb1L\_\{b\_\{1\}\}, formingo​p2=Blend​\(Q,o​p1,Lb1\)op\_\{2\}=\\text\{Blend\}\(Q,op\_\{1\},L\_\{b\_\{1\}\}\)\. Concurrently, an independent parallel operation executes modelL3L\_\{3\}, defined aso​p3=Par​\(Q,\(L3\)\)op\_\{3\}=\\text\{Par\}\(Q,\(L\_\{3\}\)\)\. The results from the first blend \(o​p2op\_\{2\}\) and the execution ofL3L\_\{3\}\(o​p3op\_\{3\}\) are synthesized in a hierarchical blending step,op4=Blend\(Q,op2,op3,Lb2op\_\{4\}=\\text\{Blend\}\(Q,op\_\{2\},op\_\{3\},L\_\{b\_\{2\}\}\)\. Finally, in a sequential operation,o​p5=Seq​\(Q,o​p4,L4\)op\_\{5\}=\\text\{Seq\}\(Q,op\_\{4\},L\_\{4\}\), modelL4L\_\{4\}processes the original questionQQagain using the output ofo​p4op\_\{4\}as context to produce the final answer\. This encoding captures the execution structure compactly and canonically, requiring𝒪​\(k2\)\\mathcal\{O\}\(k^\{2\}\)space\.

### IV\-BCost\-Benefit Estimation

For each candidate plan, we estimate \(pre\-execution\) its expected QoA, financial cost, energy, and latency\.

Token Estimation\.Cost\-benefit metrics depend on the number of tokens processed and generated, which we must estimate pre\-execution\. For each modelLL, let𝑇𝑜𝑘L​\(⋅\)\\mathit\{Tok\}\_\{L\}\(\\cdot\)denote its tokenizer \(models may segment the same string differently\) and letTLoutT^\{\\mathrm\{out\}\}\_\{L\}be the model’s expected output length \(e\.g\., average response tokens from PerfDB\)\. We estimate two quantities for a planπ\\pi: total generated tokensTπoutT^\{\\mathrm\{out\}\}\_\{\\pi\}and total processed tokensTπprocT^\{\\mathrm\{proc\}\}\_\{\\pi\}\(input \+ output across model invocations\)\.

Sequential\.InSeq​\(Q,L1,L2\)\\mathrm\{Seq\}\(Q,L\_\{1\},L\_\{2\}\),L1L\_\{1\}consumes𝑇𝑜𝑘L1​\(Q\)\\mathit\{Tok\}\_\{L\_\{1\}\}\(Q\)tokens and producesTL1outT^\{\\mathrm\{out\}\}\_\{L\_\{1\}\}\. The second model consumes the original question plus a context promptp​tctxpt\_\{\\mathrm\{ctx\}\}and the*immediate predecessor’s*output, i\.e\.,𝑇𝑜𝑘L2​\(Q\+p​tctx\+TL1out\)\\mathit\{Tok\}\_\{L\_\{2\}\}\\\!\\bigl\(Q\+pt\_\{\\mathrm\{ctx\}\}\+T^\{\\mathrm\{out\}\}\_\{L\_\{1\}\}\\bigr\), and producesTL2outT^\{\\mathrm\{out\}\}\_\{L\_\{2\}\}\. We conservatively sum outputs,TSeqout=TL1out\+TL2outT^\{\\mathrm\{out\}\}\_\{\\mathrm\{Seq\}\}=T^\{\\mathrm\{out\}\}\_\{L\_\{1\}\}\+T^\{\\mathrm\{out\}\}\_\{L\_\{2\}\}, allowing for potential redundancy where later models restate prior text\. For a length\-kkchainSeq​\(Q,L1,…,Lk\)\\mathrm\{Seq\}\(Q,L\_\{1\},\\ldots,L\_\{k\}\)we avoid exponential growth by appending only the immediate predecessor’s output at each step:Tπproc=\(𝑇𝑜𝑘L1​\(Q\)\+TL1out\)\+∑i=2k\(𝑇𝑜𝑘Li​\(Q\+p​tctx\+TLi−1out\)\+TLiout\)\.T\_\{\\pi\}^\{\\mathrm\{proc\}\}=\\bigl\(\\mathit\{Tok\}\_\{L\_\{1\}\}\(Q\)\+T^\{\\mathrm\{out\}\}\_\{L\_\{1\}\}\\bigr\)\+\\sum\_\{i=2\}^\{k\}\\Bigl\(\\mathit\{Tok\}\_\{L\_\{i\}\}\\\!\\bigl\(Q\+pt\_\{\\mathrm\{ctx\}\}\+T^\{\\mathrm\{out\}\}\_\{L\_\{i\-1\}\}\\bigr\)\+T^\{\\mathrm\{out\}\}\_\{L\_\{i\}\}\\Bigr\)\.Equivalently, chains can be viewed as nested compositions, e\.g\.,Seq​\(Q,L1,L2,L3\)=Seq​\(Q,Seq​\(Q,L1,L2\),L3\)\\mathrm\{Seq\}\(Q,L\_\{1\},L\_\{2\},L\_\{3\}\)=\\mathrm\{Seq\}\(Q,\\mathrm\{Seq\}\(Q,L\_\{1\},L\_\{2\}\),L\_\{3\}\), withp​tctxpt\_\{\\mathrm\{ctx\}\}applied from the second step onward\.

Parallel \+ blending\.InBlend​\(Q,Par​\(Q,\(L1,…,Lk\)\),Lb\)\\mathrm\{Blend\}\\\!\\bigl\(Q,\\mathrm\{Par\}\(Q,\(L\_\{1\},\\ldots,L\_\{k\}\)\),L\_\{b\}\\bigr\), eachLiL\_\{i\}processes𝑇𝑜𝑘Li​\(Q\)\\mathit\{Tok\}\_\{L\_\{i\}\}\(Q\)input tokens and producesTLioutT^\{\\mathrm\{out\}\}\_\{L\_\{i\}\}output tokens\. The blending modelLbL\_\{b\}consumes the question, a blending promptp​tbldpt\_\{\\mathrm\{bld\}\}, and the concatenated outputs:𝑇𝑜𝑘Lb​\(Q\+p​tbld\+∑i=1kTLiout\)\\mathit\{Tok\}\_\{L\_\{b\}\}\\\!\\Bigl\(Q\+pt\_\{\\mathrm\{bld\}\}\+\\sum\_\{i=1\}^\{k\}T^\{\\mathrm\{out\}\}\_\{L\_\{i\}\}\\Bigr\)and producesTLboutT^\{\\mathrm\{out\}\}\_\{L\_\{b\}\}\. Thus,Tπproc=∑i=1k\(𝑇𝑜𝑘Li​\(Q\)\+TLiout\)\+𝑇𝑜𝑘Lb​\(Q\+p​tbld\+∑i=1kTLiout\)\+TLbout,Tπout=TLbout\.T^\{\\mathrm\{proc\}\}\_\{\\pi\}=\\sum\_\{i=1\}^\{k\}\\bigl\(\\mathit\{Tok\}\_\{L\_\{i\}\}\(Q\)\+T^\{\\mathrm\{out\}\}\_\{L\_\{i\}\}\\bigr\)\\;\+\\;\\mathit\{Tok\}\_\{L\_\{b\}\}\\\!\\Bigl\(Q\+pt\_\{\\mathrm\{bld\}\}\+\\sum\_\{i=1\}^\{k\}T^\{\\mathrm\{out\}\}\_\{L\_\{i\}\}\\Bigr\)\\;\+\\;T^\{\\mathrm\{out\}\}\_\{L\_\{b\}\},\\qquad T^\{\\mathrm\{out\}\}\_\{\\pi\}=T^\{\\mathrm\{out\}\}\_\{L\_\{b\}\}\.

General plans\.For an arbitrary \(nested\) planπ\\pi, we compute\(Tπproc,Tπout\)\(T^\{\\mathrm\{proc\}\}\_\{\\pi\},T^\{\\mathrm\{out\}\}\_\{\\pi\}\)by recursively applying the above rules bottom\-up over the plan tree/DAG and summing the processed tokens over all invoked models\. This yields a conservative, tokenizer\-aware estimate of token volume that propagates through both sequential and parallel compositions\. The extended version\[[39](https://arxiv.org/html/2607.22621#bib.bib39)\]includes a complete example computation for the plan in Figure[2](https://arxiv.org/html/2607.22621#S4.F2)\.

QoA Estimation\.Predicting a plan’s QoA before execution is difficult due to model nondeterminism and the combinatorial space of multi\-LLM plans\. We therefore estimate QoA usingPerfDB, a performance database populated from offline benchmarks and execution traces that stores topic\-conditioned QoA statistics for \(i\) atomic invocations \(e\.g\.,\(Q,Li\)\(Q,L\_\{i\}\)\) and \(ii\) common local compositions \(e\.g\.,S​e​q​\(Q,Li,Lj\)Seq\(Q,L\_\{i\},L\_\{j\}\)andB​l​e​n​d​\(Q,P​a​r​\(Q,\{L1,…,Lk\}\),Lb\)Blend\(Q,Par\(Q,\\\{L\_\{1\},\\ldots,L\_\{k\}\\\}\),L\_\{b\}\)\)\. The goal is not perfect calibration, but a conservative estimate that preserves*relative*differences between candidate plans, which is sufficient for Pareto\-based search\.

Topic conditioning and lookup\.Let𝒯=\{T1,…,Tm\}\\mathcal\{T\}=\\\{T\_\{1\},\\ldots,T\_\{m\}\\\}be the inferred topics ofQQ\. For any operator or substructureo​pop, we aggregate topic\-specific entries asQ​o​A​\(o​p\)=1m​∑i=1mQ​o​Ao​p,TiQoA\(op\)=\\frac\{1\}\{m\}\\sum\_\{i=1\}^\{m\}QoA\_\{op,T\_\{i\}\}\. To initialize estimation, we first attempt an exactPerfDBmatch for the whole plan \(or large substructures\)\. If not available, we perform a fuzzy match using a weighted similarity that combines structural overlap with a normalized edit\-distance over a canonicalized plan string; we accept a fuzzy match only if the score exceeds a fixed threshold \(e\.g\., 0\.75 in our experiments\)\. Importantly, all QoA values used during planning are retrieved fromPerfDBunder a single schema \(topic \+ operator/substructure key\), even when the retrieval itself is fuzzy\. The complete procedure appears in the extended version of the paper\[[39](https://arxiv.org/html/2607.22621#bib.bib39)\]\.

Sequential refinement\.For a sequential dependency whereo​pjop\_\{j\}consumes the output ofo​piop\_\{i\}\(e\.g\., a localSeq​\(⋅,Li,Lj\)\\text\{Seq\}\(\\cdot,L\_\{i\},L\_\{j\}\)\), we update the running estimate using a pairwise relative\-effect factor derived fromPerfDB:Q​o​Anext=clip\[0,1\]⁡\(Q​o​Acurr⋅Q​o​A​\(Q,Seq​\(Li,Lj\)\)max⁡\(ϵ,Q​o​A​\(Q,Lj\)\)\)\.QoA\_\{\\text\{next\}\}=\\operatorname\{clip\}\_\{\[0,1\]\}\\\!\\left\(QoA\_\{\\text\{curr\}\}\\cdot\\frac\{QoA\(Q,\\text\{Seq\}\(L\_\{i\},L\_\{j\}\)\)\}\{\\max\(\\epsilon,\\,QoA\(Q,L\_\{j\}\)\)\}\\right\)\.whereϵ\\epsilonis a small constant andclip\[0,1\]⁡\(⋅\)\\operatorname\{clip\}\_\{\[0,1\]\}\(\\cdot\)bounds the result to\[0,1\]\[0,1\]\. Intuitively, the ratio compares the composed step against the successor alone, capturing the marginal impact of insertingLiL\_\{i\}beforeLjL\_\{j\}under the same topic\-conditioned context\. We adopt a multiplicative relative\-effect formulation because composition effects in our traces scale with the successor’s baseline quality \(i\.e\., prepending a context\-providing model tends to improve a weak successor proportionally more than a strong one\) which an additive model does not capture\. Theclip\[0,1\]⁡\(⋅\)\\operatorname\{clip\}\_\{\[0,1\]\}\(\\cdot\)guards the small fraction of cases where sparse\-coverage ratios extrapolate beyond the valid range\.*Example:*ifQ​o​A​\(Q,Lj\)=0\.50QoA\(Q,L\_\{j\}\)=0\.50andQ​o​A​\(Q,Seq​\(Li,Lj\)\)=0\.60QoA\(Q,\\text\{Seq\}\(L\_\{i\},L\_\{j\}\)\)=0\.60, the factor is1\.21\.2; withQ​o​Acurr=0\.55QoA\_\{\\text\{curr\}\}=0\.55, we obtainQ​o​Anext≈clip\[0,1\]⁡\(0\.55×1\.2\)=0\.66QoA\_\{\\text\{next\}\}\\approx\\operatorname\{clip\}\_\{\[0,1\]\}\(0\.55\\times 1\.2\)=0\.66\.

Parallel \+ blending\.For a blend nodeo​pLbop\_\{L\_\{b\}\}, where the blending operation is defined asBlend​\(Q,Par​\(Q,\{L1,…,Lk\}\),Lb\)\\mathrm\{Blend\}\(Q,\\mathrm\{Par\}\(Q,\\\{L\_\{1\},\\ldots,L\_\{k\}\\\}\),L\_\{b\}\), we combine the incoming branch estimates usingPerfDBreferences for \(i\) the blend operator itself and \(ii\) each predecessor run standalone\. LetQ​o​Ao​pLbref=Q​o​A​\(Q,o​pLb\)QoA^\{\\text\{ref\}\}\_\{op\_\{L\_\{b\}\}\}=QoA\(Q,op\_\{L\_\{b\}\}\)andQ​o​ALsref=Q​o​A​\(Q,Ls\)QoA^\{\\text\{ref\}\}\_\{L\_\{s\}\}=QoA\(Q,L\_\{s\}\)\. Given the branch estimates computed in the current traversal,Q​o​ALsnewQoA^\{\\text\{new\}\}\_\{L\_\{s\}\}, we scale the reference blend quality by the average relative change of its inputs:

Q​o​Ao​pLbnew=clip\[0,1\]⁡\(Q​o​Ao​pLbref⋅\(1\+1k​∑s=1kQ​o​ALsnew−Q​o​ALsrefmax⁡\(ϵ,Q​o​ALsref\)\)\)\.QoA^\{\\text\{new\}\}\_\{op\_\{L\_\{b\}\}\}=\\operatorname\{clip\}\_\{\[0,1\]\}\\\!\\left\(QoA^\{\\text\{ref\}\}\_\{op\_\{L\_\{b\}\}\}\\cdot\\left\(1\+\\frac\{1\}\{k\}\\sum\_\{s=1\}^\{k\}\\frac\{QoA^\{\\text\{new\}\}\_\{L\_\{s\}\}\-QoA^\{\\text\{ref\}\}\_\{L\_\{s\}\}\}\{\\max\(\\epsilon,\\,QoA^\{\\text\{ref\}\}\_\{L\_\{s\}\}\)\}\\right\)\\right\)\.
This estimator is robust to a single weak branch \(average rather than product\) while still reflecting systematic improvements/degradations across branches\. Note that the averaging form estimates the expected blend quality and is deliberately agnostic to which branch is correct; it does not model the error\-correction mechanism illustrated in Figure[1](https://arxiv.org/html/2607.22621#S3.F1), which is realized at execution time\. This is a conservative choice that avoids over\-crediting blends on the basis of a single strong branch at planning time\.*Example:*supposek=2k=2,Q​o​Ao​pLbref=0\.60QoA^\{\\text\{ref\}\}\_\{op\_\{L\_\{b\}\}\}=0\.60, and referencesQ​o​AL1ref=0\.50QoA^\{\\text\{ref\}\}\_\{L\_\{1\}\}=0\.50,Q​o​AL2ref=0\.40QoA^\{\\text\{ref\}\}\_\{L\_\{2\}\}=0\.40\. If the current plan yieldsQ​o​AL1new=0\.60QoA^\{\\text\{new\}\}\_\{L\_\{1\}\}=0\.60andQ​o​AL2new=0\.44QoA^\{\\text\{new\}\}\_\{L\_\{2\}\}=0\.44, then the average relative change is12​\(0\.60−0\.500\.50\+0\.44−0\.400\.40\)=0\.15\\frac\{1\}\{2\}\\left\(\\frac\{0\.60\-0\.50\}\{0\.50\}\+\\frac\{0\.44\-0\.40\}\{0\.40\}\\right\)=0\.15, soQ​o​Ao​pLbnew=clip⁡\(0\.60×\(1\+0\.15\)\)=0\.69QoA^\{\\text\{new\}\}\_\{op\_\{L\_\{b\}\}\}=\\operatorname\{clip\}\(0\.60\\times\(1\+0\.15\)\)=0\.69\.

Plan\-level propagation and fallbacks\.We compute the plan QoA by traversing the plan DAG in a topological order that respects precedence constraints \(the complete algorithm is in the extended version\[[39](https://arxiv.org/html/2607.22621#bib.bib39)\]\)\. So, an operator is processed only after all prerequisites are estimated\. Along edges corresponding to sequential consumption we apply the sequential update rule; at blend nodes we wait for all incoming branches and apply the blending rule\. WhenPerfDBlacks a required composed entry \(e\.g\., missingQ​o​A​\(Q,Seq​\(Li,Lj\)\)QoA\(Q,\\text\{Seq\}\(L\_\{i\},L\_\{j\}\)\)\), we fall back to the closest available statistics \(e\.g\., use other known predecessors ofLjL\_\{j\}to estimate an average relative\-effect factor, or default to a neutral factor of11when no pairwise information exists\)\. For nested blend structures where direct references are sparse, we normalize the structure by flattening nested blends before performing lookups, then apply the same blend rule on the resulting fan\-in\. These fallbacks ensure estimation completes under incomplete coverage; in Section[VI](https://arxiv.org/html/2607.22621#S6)we quantify how estimation error varies withPerfDBcoverage\.

Financial Cost Estimation\.We estimate the financial cost of a plan as the sum of its operation\-level costs\[[29](https://arxiv.org/html/2607.22621#bib.bib29)\]\. For a single invocation of modelLiL\_\{i\}, we use a simple fixed\-plus\-variable pricing model: a fixed per\-call chargefifixf\_\{i\}^\{\\text\{fix\}\}and a per\-token ratefivarf\_\{i\}^\{\\text\{var\}\}applied to the total number of billed tokens \(prompt \+ generated\)\. Using our token estimator, the expected billed tokens foro​piop\_\{i\}equal the model\-specific tokenized prompt length plus the predicted output length:𝐹𝑖𝑛𝑎𝑛𝑐𝑖𝑎𝑙​\(Q,Li\)=fifix\+fivar⋅\(𝑇𝑜𝑘i​\(pt\)\+To​piout\)\\mathit\{Financial\}\(Q,L\_\{i\}\)=f\_\{i\}^\{\\text\{fix\}\}\+f\_\{i\}^\{\\text\{var\}\}\\cdot\\bigl\(\\mathit\{Tok\}\_\{i\}\(p\_\{t\}\)\+T^\{\\text\{out\}\}\_\{op\_\{i\}\}\\bigr\)\. For a parallel operatorPar​\(Q,\{L1,…,Ln\}\)\\text\{Par\}\(Q,\\\{L\_\{1\},\\ldots,L\_\{n\}\\\}\), invocations are independent, so costs add:𝐹𝑖𝑛𝑎𝑛𝑐𝑖𝑎𝑙​\(Par​\(Q,\{L1,…,Ln\}\)\)=∑i=1n𝐹𝑖𝑛𝑎𝑛𝑐𝑖𝑎𝑙​\(Q,Li\)\\mathit\{Financial\}\\\!\\left\(\\text\{Par\}\(Q,\\\{L\_\{1\},\\ldots,L\_\{n\}\\\}\)\\right\)=\\sum\_\{i=1\}^\{n\}\\mathit\{Financial\}\(Q,L\_\{i\}\)\. For a general planπ\\pi, we account for prompt growth in sequential and nested structures by charging each operation for its own prompt plus the outputs of its immediate predecessors:𝐹𝑖𝑛𝑎𝑛𝑐𝑖𝑎𝑙​\(π\)=∑i=1n\(fivar​\(𝑇𝑜𝑘i​\(pt\)\+∑o​pj∈pred​\(o​pi\)To​pjout\+To​piout\)\+fifix\)\\ \\mathit\{Financial\}\(\\pi\)=\\sum\_\{i=1\}^\{n\}\\left\(f\_\{i\}^\{\\text\{var\}\}\\left\(\\mathit\{Tok\}\_\{i\}\(p\_\{t\}\)\+\\textstyle\\sum\_\{op\_\{j\}\\in\\mathrm\{pred\}\(op\_\{i\}\)\}\\\!T^\{\\text\{out\}\}\_\{op\_\{j\}\}\+T^\{\\text\{out\}\}\_\{op\_\{i\}\}\\right\)\+f\_\{i\}^\{\\text\{fix\}\}\\right\)\. Here,pred​\(o​pi\)\\mathrm\{pred\}\(op\_\{i\}\)denotes the set of operators whose outputs are included ino​piop\_\{i\}’s prompt \(e\.g\., the immediate predecessor in a sequence, or all fan\-in branches for a blend node\)\.

Energy Consumption Estimation\.LLM energy consumption is approximately proportional to the number of tokens processed and generated\[[40](https://arxiv.org/html/2607.22621#bib.bib40)\]\. Leteie\_\{i\}\(J/token\\mathrm\{J/token\}\) denote the energy per token for operationo​piop\_\{i\}obtained fromPerfDBor external measurements\[[37](https://arxiv.org/html/2607.22621#bib.bib37)\]\)\. We estimate plan energy by reusing the same structural accounting as financial cost, but without a fixed per\-call term\. This keeps energy estimation consistent with how prompts and intermediate outputs propagate through the plan\.

Latency Estimation\.Plan latency depends on both the plan structure and the number of tokens processed and generated\[[32](https://arxiv.org/html/2607.22621#bib.bib32)\]\. For a single invocationo​piop\_\{i\}, we approximate latency as linear in token volume using a per\-token time coefficientlil\_\{i\}\(s/token\)L​a​t​e​n​c​y​\(Q,Li\)=li​\(𝑇𝑜𝑘i​\(pt\)\+To​piout\)Latency\(Q,L\_\{i\}\)=l\_\{i\}\(\\mathit\{Tok\}\_\{i\}\(p\_\{t\}\)\+T^\{\\text\{out\}\}\_\{op\_\{i\}\}\)\. For sequential \(or more generally, precedence\-constrained\) portions of a plan, latencies add along executed operations, using the same prompt\-growth accounting as above\. For a pure parallel operatorPar​\(Q,\{L1,…,Ln\}\)\\text\{Par\}\(Q,\\\{L\_\{1\},\\ldots,L\_\{n\}\\\}\), branches execute concurrently, so the end\-to\-end latency is dominated by the slowest branch\. For mixed plans with parallelism and joins, we apply these rules along the plan DAG: sum along each critical path and take the maximum over paths that run concurrently\.

### IV\-CPareto Search over Multi\-LLM Plans

The plan space is inherently combinatorial\. Under our operator\-DAG representation, each node selects one of\|L\|\|L\|models and each of the\(k2\)\\binom\{k\}\{2\}forward edges may be present or absent, yielding\|Π\|=\|L\|k⋅2\(k2\)=\|L\|k⋅2k​\(k−1\)2\|\\Pi\|=\|L\|^\{k\}\\cdot 2^\{\\binom\{k\}\{2\}\}=\|L\|^\{k\}\\cdot 2^\{\\frac\{k\(k\-1\)\}\{2\}\}, with the derivation provided in the extended version\[[39](https://arxiv.org/html/2607.22621#bib.bib39)\]\. This space becomes large even for modest settings \(e\.g\.,k=5k\{=\}5,\|L\|=10\|L\|\{=\}10gives105⋅210≈1\.024×10810^\{5\}\\cdot 2^\{10\}\\approx 1\.024\\times 10^\{8\}plans\)\. Moreover, optimizing overΠ\\Piis NP\-hard even for simplified variants of our objective structure\[[32](https://arxiv.org/html/2607.22621#bib.bib32)\]\. Consequently, exhaustive enumeration is infeasible except for small instances, and we instead rely on search procedures that can efficiently exploreΠ\\Piusing inexpensive, pre\-execution objective estimates\. In particular, we consider three widely used approaches: a genetic algorithm \(NSGA\-II\), bottom\-up dynamic programming \(DP\), and randomized hill climbing\. All three optimizers operate over the same plan encoding \(canonical DAG topology \+ model assignments\), use the samePerfDB\-based estimators for objective evaluation, and output a set of non\-dominated feasible plans that trade off QoA, cost, latency, and energy under user budgets without enumerating all ofΠ\\Pi\. We then select a single plan for execution by normalizing objectives and applying the user weightsWW\. Due to space constraints, we present only the core mechanics below; full plan\-generation details and algorithmic variants appear in the extended version\[[39](https://arxiv.org/html/2607.22621#bib.bib39)\]\.

Genetic Algorithm \(NSGA\-II\)\.NSGA\-II is a robust default for our discrete, non\-convex search space \(topology \+ model choices\)\. Each individual encodes \(i\) a canonicalized DAG topology \(connectivity\_map\) and \(ii\) a model\-assignment vector𝐦\\mathbf\{m\}\(one LLM per node\)\. Starting from an initial population of sizeNN, NSGA\-II iterates forGGgenerations using binary tournament selection \(non\-dominated rank, then crowding distance\)\. With crossover probabilityCrC\_\{r\}, we apply one\-point crossover independently to the topology and assignment segments, then apply targeted mutations that mirror common plan edits: \(i\) edge flips \(rewiring\), \(ii\) node insertions \(adding an operator\), and \(iii\) model reassignments \(changingLiL\_\{i\}on a non\-blending node\)\. After any structural edit we apply lightweight repair to enforce validity \(e\.g\., a valid sink/output\) and the blending constraint \(any node with in\-degree\>1\>1is assigned the blending model\), followed by canonicalization so isomorphic plans share a single encoding\. We rank the merged parent\+offspring pool with fast non\-dominated sorting and constraint handling \(feasible dominates infeasible; infeasible ranked by total violation\), then select the next generation by taking successive fronts with crowding\-distance tie\-breaking\. The first non\-dominated frontℱ1\\mathcal\{F\}\_\{1\}approximates the Pareto set; we choose a single execution plan by applyingWWto normalized objectives overℱ1\\mathcal\{F\}\_\{1\}\.

Dynamic Programming\.DP provides a systematic baseline by enumerating canonical plan structures and propagating feasible assignment sets per structure\. For plans withkkoperations, the DP state is indexed by the canonical structure identifierconnectivity\_map\. Each bucket stores multiple candidate assignments𝐦\\mathbf\{m\}\(one model per node\) together with plan\-level estimated costs computed viaPerfDB\-based estimators\. Blending is enforced by construction: nodes with in\-degree\>1\>1must use the designated blending model, while nodes with in\-degree≤1\\leq 1may use any base model\. To buildD​P​\[k\]DP\[k\]fromD​P​\[k−1\]DP\[k\{\-\}1\], we extend each canonical\(k−1\)\(k\{\-\}1\)\-node structure by adding a new node and enumerating predecessor subsetsS⊆\{0,…,k−2\}S\\subseteq\\\{0,\\ldots,k\{\-\}2\\\}, adding edges\(u→k−1\)\(u\\rightarrow k\{\-\}1\)for allu∈Su\\in S\. For each parent assignment, we enumerate admissible model choices for the new node, evaluate objectives, and canonicalize to merge isomorphic structures into same bucket\.

To control state growth, we use pruning\. In particular, we found a simple*parent\-specificΔ\\Delta\-QoA progress gate*to provide the best runtime–quality trade\-off: when expanding a parent into children, we retain a child only if𝑄𝑜𝐴​\(πchild\)≥𝑄𝑜𝐴​\(πparent\)\+Δ\\mathit\{QoA\}\(\\pi\_\{\\text\{child\}\}\)\\;\\geq\\;\\mathit\{QoA\}\(\\pi\_\{\\text\{parent\}\}\)\+\\Delta\. We useΔ=0\.05\\Delta=0\.05by default\. \(We also explored within\-bucket approximate dominance pruning via multiplicativeε\\varepsilon\-dominance; see the extended version\[[39](https://arxiv.org/html/2607.22621#bib.bib39)\]\.\)

Hill Climbing\.DP can become state\-intensive askkgrows, so we also implement a randomized local\-search optimizer that quickly finds high\-quality tradeoffs under a per\-question time budget, following\[[41](https://arxiv.org/html/2607.22621#bib.bib41)\]\. The method performs repeated random restarts: each restart samples a feasible planπ\\pi, then greedily hill\-climbs using*Pareto\-improving*moves until reaching a locally non\-dominated solution; collecting these local optima across restarts yields an approximate frontier\. We define the neighborhood ofπ\\pivia exhaustive single\-step mutations: \(i\)*model mutation*changes the base model at one non\-blending node, \(ii\)*edge flip*toggles one edge in the upper\-triangular encoding and keeps the result only if it remains a valid DAG, and \(iii\)*node addition*appends one new node \(up to the operation limitkk\) using an admissible extension that preserves a valid sink/output\. After any structural mutation we repair assignments to satisfy the blending constraint, canonicalize the topology, and remap𝐦\\mathbf\{m\}under the canonical permutation so isomorphic DAGs share a single encoding\. We discard neighbors whose estimated metrics violate the planning\-time budgets\. A neighbor is accepted if it dominates the current plan \(no worse on all objectives and strictly better on at least one\); we scan the neighborhood and take the first dominating move, repeating until no neighbor dominates the current plan\. As candidates are discovered we maintain a global frontier \(optionally applying the sameΔ\\Delta\-QoA filter from before when inserting plans\)\.

## VOpti\-Q Implementation

We implementOpti\-Qas a modular framework for flexible multi\-LLM QA planning under user\-defined constraints555Code, datasets, prompts, and results are available athttps://github\.com/Aamir7693/Opti\-Q\.git\.\. Our prototype integrates five widely used open\-source LLMs: Gemma\-3:27B\[[42](https://arxiv.org/html/2607.22621#bib.bib42)\], LLaMA3\-ChatQA \(8B\)\[[43](https://arxiv.org/html/2607.22621#bib.bib43)\], Qwen2\.5 \(14B\)\[[44](https://arxiv.org/html/2607.22621#bib.bib44)\], Phi\-4 \(14B\)\[[45](https://arxiv.org/html/2607.22621#bib.bib45)\], and Mistral \(7B\)\[[46](https://arxiv.org/html/2607.22621#bib.bib46)\]\. We selected these models for their complementary strengths across benchmarks\[[47](https://arxiv.org/html/2607.22621#bib.bib47),[15](https://arxiv.org/html/2607.22621#bib.bib15)\]\(e\.g\., strong reasoning vs\. conversational QA vs\. efficiency\)\. All models run locally via Ollama666https://ollama\.com, which also simplifies adding or replacing models\. For tuning system parameters, we construct a held\-out configuration set by sampling 100 SimpleQA and 100 MMLU\-Pro questions; this set is strictly disjoint from the test set in Section[VI](https://arxiv.org/html/2607.22621#S6)to prevent leakage\.

Blending Model\.Opti\-Qsupports pluggable blending models for combining outputs from parallel branches\. We evaluated both the GenFuser component from LLM\-Blender\[[21](https://arxiv.org/html/2607.22621#bib.bib21)\]and a general\-purpose LLM\-based blender\. In our experiments, Gemma\-3:27B produced higher validation QoA than GenFuser on the held\-out configuration set described above, so we use Gemma\-3:27B as the designated blending model\. The blender receives the original question, a blending prompt\(pb​l​d\)\(p\_\{bld\}\), and the concatenated branch outputs, i\.e\.,\(pb​l​d⊕Par​\(Q,…\)\)\(p\_\{bld\}\\oplus\\text\{Par\}\(Q,\\dots\)\), and produces a single consolidated answer\.

Question Processing andPerfDBInitialization\.Opti\-QinitializesPerfDBwith per\-model and per\-topic planning parameters, including model size and empirical estimates of cost, latency, energy, and QoA\. These priors are populated from public benchmarking sources, including ML\.ENERGY\[[37](https://arxiv.org/html/2607.22621#bib.bib37)\]and Artificial Analysis777https://artificialanalysis\.ai/, and are refined as additional execution traces are collected\.

At question time,Opti\-Qextracts topics for an incoming questionQQusing BERTopic\[[48](https://arxiv.org/html/2607.22621#bib.bib48)\]\. Concretely, we embedQQwith a pretrained transformer encoder, reduce dimensionality with UMAP\[[49](https://arxiv.org/html/2607.22621#bib.bib49)\], cluster with HDBSCAN\[[50](https://arxiv.org/html/2607.22621#bib.bib50)\], and derive representative topic keywords using c\-TF\-IDF\[[48](https://arxiv.org/html/2607.22621#bib.bib48)\]\. The inferred topics are then added to the question tuple used by the planners andPerfDBlookups\.

Prompt Design\.We consider four prompting strategies:Zero\-Shot \(ZS\)\[[51](https://arxiv.org/html/2607.22621#bib.bib51)\],Few\-Shot \(FS\)\[[52](https://arxiv.org/html/2607.22621#bib.bib52)\], andChain\-of\-Thought \(CoT\)\[[14](https://arxiv.org/html/2607.22621#bib.bib14)\], and abaselinestrategy \(B\) \(i\.e\., just instructions without examples or reasoning\)\. We also define two auxiliary prompts: acontext promptp​tc​t​xpt\_\{ctx\}for sequential operations \(ensuring the successor treats the predecessor output as evidence\) and ablending promptp​tb​l​dpt\_\{bld\}for parallel operations \(instructing the blender to reconcile redundant or conflicting answers\)\. Our prompting evaluation shows thatZSprovides the best trade\-off between efficiency and stability \(see the extended version\[[39](https://arxiv.org/html/2607.22621#bib.bib39)\]\)\. We therefore useZSin the experiments of Section[VI](https://arxiv.org/html/2607.22621#S6)\.

QoA Computation\.We measure QoA using semantic similarity rather than lexical overlap \(e\.g\., BLEU\[[53](https://arxiv.org/html/2607.22621#bib.bib53)\], ROUGE\[[54](https://arxiv.org/html/2607.22621#bib.bib54)\]\), since LLM outputs often vary stylistically while preserving meaning\. For free\-form QA, we embed generated and reference answers with the 384\-dimensionalall\-MiniLM\-L6\-v2sentence transformer888https://huggingface\.co/sentence\-transformers/all\-MiniLM\-L6\-v2, which has demonstrated strong performance on the Massive Text Embedding Benchmark \(MTEB\)999https://huggingface\.co/spaces/mteb/leaderboardand is widely adopted\. The QoA is then computed as the cosine similarity between embeddings\. For multiple\-choice QA, we score single\-answer questions \(e\.g\., MMLU\-Pro\) by normalized exact match \(binary in\{0,1\}\\\{0,1\\\}\)\. For multi\-answer \(“select all that apply”\) questions, we normalize the predicted and gold option sets and compute a set\-overlap score in\[0,1\]\[0,1\]\(e\.g\., Jaccard similarity\) so partially correct predictions receive partial credit\.

NSGA\-II Hyperparameter Optimization\.We tuned NSGA\-II hyperparameters via grid search on a held\-out stratified set \(200 questions\); we report the best setting and release configs/scripts\. The best\-performing configuration by mean scalarized score in the converged population wasP=200P=200,G=200G=200, and𝝁=\{0\.3,0\.1,0\.3\}\\boldsymbol\{\\mu\}=\\\{0\.3,0\.1,0\.3\\\}, which outperformed other settings by1\.2×1\.2\\times–12×12\\timesacross validation metrics, and we adopt this configuration for all subsequent experiments\.

## VIExperimental Evaluation

Evaluation setup\.Our evaluation targets two foundational QA paradigms using specialized benchmarks:SimpleQA, an open\-ended factual QA dataset containing 4K questions, andMMLU\-Pro, a domain\-specific dataset comprising 12K multiple choice questions\. Because exhaustive evaluation would entail \(≈\\approx3\.2M plan executions across five models\), we instead build a representative, computationally tractable testbed using stratified sampling: we select 10 diverse question types from each dataset \(e\.g\.,*History*,*Mathematics*,*Sports*\) and uniformly sample 10 questions per type, yielding 200 questions total \(100 per benchmark\) spanning factual, analytical, and reasoning\-intensive tasks\. Even under this stratified design, the evaluation remained computationally intensive, requiring roughly 1,000 GPU\-hours overall\. Experiments run on a dedicated high\-performance cluster equipped with2×2\\times24\-core Intel Xeon Gold 6240R \(2\.40 GHz, 165 W TDP\) CPUs and x4 NVIDIA RTX 6000 GPUs, with shared central storage exceeding 3 PB\. To account for variability in LLM outputs, we execute each question five times under fixed decoding configurations and report micro\-averaged QoA, latency, energy, and financial cost with 95% confidence intervals\.

Planner Comparison\.We compareOpti\-Q’s planner backends—NSGA\-II, exact DP, pruned DP, Hill Climbing, and pruned Hill Climbing—as the maximum number of operationskkincreases\. The full comparison table is provided in the extended version\[[39](https://arxiv.org/html/2607.22621#bib.bib39)\]\. For eachk∈\{1,…,5\}k\\in\\\{1,\\dots,5\\\}, we measure \(i\) planning time and \(ii\) Pareto\-front quality using hypervolume \(HV\) and inverted generational distance \(IGD\), computed relative to the exact DP frontier \(when tractable\)\. For fair comparison under matched computational budgets, the hill\-climbing planner is run with a per\-question timeout equal to the maximum wall\-clock time observed for NSGA\-II on the same question under the same operation limit\. Exact DP recovers the reference frontier but exhibits state\-space explosion askkgrows \(from near\-zero planning time at smallkkto over 200 s atk=5k\{=\}5\)\. Pruned DP substantially reduces planning time \(down to seconds atk=5k\{=\}5\) at the cost of lower frontier quality\. Hill Climbing, under the same time budget as NSGA\-II, degrades more sharply at largerkkdue to local optima; adding pruning further reduces frontier quality with little runtime benefit under timeout\-bounded search\. NSGA\-II provides the best scalability–quality trade\-off, maintaining near\-reference frontier quality while keeping planning time in the low tens of seconds atk=5k\{=\}5\(21 s\), motivating it as the default backend for larger plan spaces\. Unless noted otherwise, in subsequent experiments, we use pruned DP as the default DP\-based planner and Hill Climbing with strict Pareto dominance \(i\.e\., without theΔ\\Delta\-QoA filter\) as the default Hill Climbing variant, with per\-question timeout matched to NSGA\-II for the same question and operation limit, based on the above runtime–quality trade\-offs\.

Baselines and scope\.Opti\-Qis designed for broad open\-domain QA, while most existing cost\-aware multi\-LLM methods target classification tasks\. To enable a comparison on MMLU\-Pro, we treat each question as a classification and compareOpti\-QagainstThriftLLM\[[31](https://arxiv.org/html/2607.22621#bib.bib31)\],LLM\-Ensemble\[[55](https://arxiv.org/html/2607.22621#bib.bib55)\],FrugalGPT\[[29](https://arxiv.org/html/2607.22621#bib.bib29)\]andLLM\-Blender\[[21](https://arxiv.org/html/2607.22621#bib.bib21)\]\. BecauseLLM\-Ensemblelacks a budget constraint, we implement a budgeted variant that greedily selects top\-KKweighted models until the budget is met\[[55](https://arxiv.org/html/2607.22621#bib.bib55)\]\. ForLLM\-Blender, we adopt the latest*PairRanker*and*GenFuser*components with the author\-recommended settings\. On SimpleQA, we compareOpti\-QtoFrugalGPTandLLM\-Blenderusing the released configurations\. Unless noted, all methods \(Opti\-Qand baselines\) were evaluated under identical budget constraints\. We omitLLM\-TopLa\(repository errors\), Shekhar et al\. \(summarization task\), andPalimpzest\(declarative AI analytics pipelines over document corpora\) as they are not directly comparable under our QA\-oriented orchestration setting\.

### VI\-ABaseline Comparison

We evaluate all baseline systems under a unified cost–accuracy framework\. Each method is configured to adhere to the same strict per\-question budget asOpti\-Q\. However,LLM\-Blenderis not budget\-aware and invokes its full model set for every query, leading to different average costs across questions:≈4\.4×10−5\\approx 4\.4\{\\times\}10^\{\-5\}USD/query for theMMLU\-Proclassification benchmark and≈1\.6×10−5\\approx 1\.6\{\\times\}10^\{\-5\}USD/query for theSimpleQAopen\-ended benchmark\. Hence, it is shown in our figures for completeness but excluded from cost–accuracy comparisons, and compared againstOpti\-Qsolely on the basis of QoA\. Furthermore, we adapt FrugalGPT to operate under our strict per\-query budget constraint rather than its native expected\-cost scheduling mechanism to align its behavior with other baselines\. All baseline systems were re\-implemented using our selected model suite and benchmark datasets; we denote these adapted versions with an asterisk \(e\.g\.,FrugalGPT\*,ThriftLLM\*\)\. Hereafter, all baseline names refer to these re\-implemented variants\. The evaluation spans five budget levels,b∈\{1,…,5\}b\\in\\\{1,\\ldots,5\\\}, each scaling a baseline single\-model usage cost of1\.2×10−51\.2\\times 10^\{\-5\}\. This value represents the average per\-question execution cost across all candidate LLMs\. Thus, the maximum allowable financial cost per question is defined asFmax​\(b\)=b⋅1\.2×10−5F\_\{\\text\{max\}\}\(b\)=b\\cdot 1\.2\\times 10^\{\-5\}\.

#### Classification\-reasoning QA \(MMLU\-PRO\)

![Refer to caption](https://arxiv.org/html/2607.22621v1/figures/combined_baseline.png)Figure 3:QoA versus budget across two benchmarks: \(a\)MMLU\-Pro, where QoA is measured as the percentage of questions answered correctly, and \(b\)SimpleQA, where QoA is measured as the average quality per question\.We first evaluateOpti\-Qon theMMLU\-Probenchmark, a multiple\-choice QA task where each question has fixed answer options, and performance is measured by accuracy under varying budget constraints\. At the lowest budget \(b=1b\{=\}1\), as shown in Figure[3](https://arxiv.org/html/2607.22621#S6.F3)\(a\),ThriftLLM\*is competitive with theOpti\-Qplanner variants \(about 41%\), due to its routing mixture between Mistral and Qwen models and its per\-question ensemble aggregation, which balance cost and model capability\.Opti\-Q\(DP\) is slightly higher at approximately 42%\. As the budget increases \(b=2−3b\{=\}2\{\-\}3\), all baseline systems converge around 45–47% accuracy and subsequently plateau\. This stagnation arises because these methods execute all models in parallel and deterministically collapse their predictions into a single label via confidence ranking or majority voting, thereby discarding contextual and complementary evidence among models\. Consequently, additional model invocations increase the cost without improving the accuracy\. In contrast,Opti\-Qimproves substantially with budget\. Atb=2b\{=\}2,Opti\-Q\(DP\) performs best \(≈73%\\approx 73\\%\), while atb=3b\{=\}3,Opti\-Q\(NSGA\-II\) overtakes \(≈77%\\approx 77\\%versus≈75%\\approx 75\\%for DP and≈71%\\approx 71\\%for Hill Climbing\)\. Unlike deterministic aggregation strategies,Opti\-Qconstructs question\-specific sequential/parallel/hybrid execution plans that preserve inter\-model reasoning context and allocate budget adaptively across dependent and independent operations\. This enables continued gains beyond the baseline plateau\. Cost analysis shows that ThriftLLM\* and FrugalGPT\* consume, on average, 58–76% across budget levels, while LLM\-Ensemble\* consumes 53–99%\. In contrast,Opti\-Quses 64–88% of the available budget through selective plan allocation while achieving substantially higher accuracy\. Overall, these results demonstrate that deterministic parallel ensembles exhibit early efficiency but limited scalability, whereas adaptive planning inOpti\-Qbreaks this plateau by dynamically composing sequential, parallel, and hybrid plans to achieve superior cost–accuracy efficiency on classification\-style QA tasks\. Compared to LLM\-Blender,Opti\-Qattains substantially higher QoA \(0\.82 vs\. 0\.14\), underscoring the benefit of adaptive execution planning\.

#### Open\-Ended QA \(SimpleQA\)

We next evaluateOpti\-Qon theSimpleQAbenchmark \(Figure[3](https://arxiv.org/html/2607.22621#S6.F3)\(b\)\), an open\-ended QA task that tests the model’s ability to synthesize and articulate factual knowledge\. Performance is measured by QoA, range \[0,1\], computed as the cosine similarity between predicted and reference embeddings\. Among the baselines, FrugalGPT\* remains nearly flat at around 0\.46 QoA across budgets, consuming on average 58–76% of the budget \(mean cost7\.2×10−6−1\.4×10−57\.2\{\\times\}10^\{\-6\}\{\-\}1\.4\{\\times\}10^\{\-5\}\)\. LLM\-Blender\* \(shown for completeness but excluded from cost\-normalized comparisons due to fixed full\-ensemble invocation\) incurs a nearly fixed cost of≈1\.6×10−5\\approx 1\.6\{\\times\}10^\{\-5\}USD/query and remains substantially lower at about 0\.22 QoA\. In contrast, all threeOpti\-Qplanner variants outperform these baselines\. At the lowest budget \(b=1b\{=\}1\),Opti\-Q\(DP\) performs best \(≈0\.52\\approx 0\.52\), followed byOpti\-Q\(Hill Climbing\) \(≈0\.46\\approx 0\.46\) andOpti\-Q\(NSGA\-II\) \(≈0\.44\\approx 0\.44\)\. At moderate budgets \(b=2−3b\{=\}2\{\-\}3\), the three planners converge to similar QoA \(≈0\.56\\approx 0\.56–0\.610\.61\)\. At higher budgets \(b=4−5b\{=\}4\{\-\}5\),Opti\-Q\(NSGA\-II\) achieves the best QoA, improving to about 0\.65 while utilizing 64–88% of the available budget, whereas DP and Hill Climbing plateau near 0\.55–0\.58\.

Evaluating the raw trade\-offs highlights the diverging cost\-benefit profiles between the two benchmarks\. For SimpleQA, scaling fromk=1k=1tok=5k=5exhibits moderate returns: a23\.6%23\.6\\%relative QoA gain drives cost, energy, and latency up by roughly6\.5×6\.5\\times,6×6\\times, and3\.8×3\.8\\times, respectively\. Conversely, MMLU\-Pro demonstrates a much stronger return on resource investment at higher k\. For this dataset, a substantial54\.7%54\.7\\%QoA improvement is achieved alongside4×4\\timesto5\.3×5\.3\\timesincreases across the resource metrics, justifying the expanded plan space\. Crucially, askkgrows, the planner increasingly favors concurrent execution\. Hybrid and parallel plans account for≈36%\\approx 36\\%,77%77\\%, and82%82\\%of all executions atk=3,4k=3,4, and55, respectively \(leaving sequential plans at just≈14%\\approx 14\\%byk=5k=5\)\. This architectural shift explains the higher resource consumption at largerkk:k=3k=3captures most benefits for semantic QA, while classification tasks can justify largerkkfor higher QoA\.

### VI\-BRobustness to Limited Historical Data inPerfDB

We assess howOpti\-Qbehaves whenPerfDBhas limited historical coverage\. This experiment directly studies cold\-start and sparse\-statistics settings by varying the amount of empirical execution metadata available to the planner\. We define five coverage levels \(0–44\)\. Level 0 contains only static model priors, representing a cold\-start setting with no empirical execution history\. Levels 1–4 progressively enrichPerfDBwith empirical traces collected from*disjoint training questions*that share the same topical distribution as the evaluation set\. This setup enables the planner to generalize from related but unseen instances while avoiding data leakage\. The cumulative numbers of recorded executions available inPerfDBat Levels 1–4 are135135,985985,1,7351\{,\}735, and2,3602\{,\}360, respectively\. These counts denote execution traces used to populatePerfDB, not evaluation questions\. For each question, the planner can compose up to five operations \(k=5k\{=\}5\), as identified in the previous experiment\.

Due to space constraints, the main paper reports NSGA\-II results forMMLU\-ProandSimpleQA\(Figures[4](https://arxiv.org/html/2607.22621#S6.F4)and[4](https://arxiv.org/html/2607.22621#S6.F4)\); the extended version\[[39](https://arxiv.org/html/2607.22621#bib.bib39)\]includes the corresponding level\-sweep analysis for DP and Hill Climbing, which show the same qualitative trend\. Figures[4](https://arxiv.org/html/2607.22621#S6.F4)and[4](https://arxiv.org/html/2607.22621#S6.F4)compare predicted and actual plan\-level cost, energy, latency, and QoA acrossPerfDBcoverage levels, while Table[I](https://arxiv.org/html/2607.22621#S6.T1)summarizes the resulting QoA\-bucket shifts\.

![Refer to caption](https://arxiv.org/html/2607.22621v1/figures/levels_fig/nsga_Levels_comp_mmlu.png)

\\phantomcaption

![Refer to caption](https://arxiv.org/html/2607.22621v1/figures/levels_fig/nsga_Levels_comp_simpleqa.png)

\\phantomcaption

Figure 4:Actual vs\. predicted plan\-level cost, energy, latency, and QoA acrossPerfDBcoverage levels \(0–4\) forNSGA\-IIonMMLU\-Pro\(a\) andSimpleQA\(b\)\. As historical coverage increases, resource estimates become better calibrated and QoA estimates provide a stronger selection signal for planning\.RicherPerfDBcoverage consistently improves plan quality\. ForMMLU\-Pro, QoA increases monotonically from0\.400\.40at Level 0 to0\.670\.67at Level 4 \(\+0\.27\+0\.27,≈66\.7%\\approx 66\.7\\%relative improvement\), while resource usage stabilizes after an early Level 1 calibration increase\. Mean cost rises from5\.9×10−55\.9\{\\times\}10^\{\-5\}at Level 0 to1\.14×10−41\.14\{\\times\}10^\{\-4\}at Level 1, then settles in the7\.7×10−57\.7\{\\times\}10^\{\-5\}–8\.4×10−58\.4\{\\times\}10^\{\-5\}range at Levels 2–4; energy and latency show the same pattern, peaking at Level 1 \(134\.8134\.8J,129\.7129\.7s\) and then remaining broadly stable \(roughly106106–116116J and109109–124124s\)\. ForSimpleQA, QoA increases from0\.250\.25to0\.480\.48\(\+0\.24\+0\.24,≈96\.7%\\approx 96\.7\\%relative improvement\), while resource usage decreases substantially: cost drops from3\.9×10−53\.9\{\\times\}10^\{\-5\}to2\.2×10−52\.2\{\\times\}10^\{\-5\}, energy from62\.162\.1J to22\.122\.1J, and latency from74\.474\.4s to18\.118\.1s \(Levels 0 to 4\)\. Thus, empirical history enables the planner to select plans that are not only higher quality but also more efficient, especially for open\-ended QA where cold\-start priors are conservative\.

Prediction error decreases sharply asPerfDBcoverage increases\. ForMMLU\-Pro, MAE drops from Level 0 to Level 4 for cost \(1\.37→0\.33×10−41\.37\{\\to\}0\.33\{\\times\}10^\{\-4\}USD\), energy \(173\.8→41\.5173\.8\{\\to\}41\.5J\), latency \(257\.0→48\.8257\.0\{\\to\}48\.8s\), and QoA \(0\.58→0\.340\.58\{\\to\}0\.34\)\. The trend is even stronger forSimpleQA: cost MAE decreases from2\.58→0\.32×10−42\.58\{\\to\}0\.32\{\\times\}10^\{\-4\}USD, energy from307\.3→41\.6307\.3\{\\to\}41\.6J, latency from474\.0→39\.6474\.0\{\\to\}39\.6s, and QoA from0\.72→0\.270\.72\{\\to\}0\.27\. Thus, empirical traces rapidly correct cold\-start bias in resource estimates, while QoA remains harder to calibrate but still improves with coverage\.

TABLE I:Plan QoA distribution \(%\) across QoA buckets as a function of operation limitKKandPerfDBcoverage level on MMLU\-Pro and SimpleQA\. LargerKKand higher levels generally shift mass toward higher\-QoA buckets\.ParamValMMLU\-Pro QoA bucket \(%\)SimpleQA QoA bucket \(%\)0–0\.50\.5–0\.80\.8–1\.00–0\.50\.5–0\.80\.8–1\.0Operation limitKKKK157\.550\.0042\.4571\.6320\.005\.12KK256\.790\.0043\.2186\.369\.631\.87KK346\.220\.0053\.7871\.7420\.226\.01KK440\.000\.0060\.0065\.2224\.518\.48KK534\.580\.0065\.4257\.2327\.6112\.86PerfDBcoverage levelLevel059\.670\.0040\.3387\.860\.4311\.71Level149\.400\.0050\.6080\.4013\.606\.00Level245\.200\.0054\.8076\.2017\.406\.40Level336\.200\.0063\.8059\.0027\.4013\.60Level432\.800\.0067\.2056\.0028\.0016\.00Table[I](https://arxiv.org/html/2607.22621#S6.T1)shows that improved calibration translates into better plan selection\. ForMMLU\-Pro, the high\-QoA bucket \(0\.80\.8–1\.01\.0\) grows from40\.33%40\.33\\%at Level 0 to67\.20%67\.20\\%at Level 4, while the low\-QoA bucket drops from59\.67%59\.67\\%to32\.80%32\.80\\%\. The middle bucket remains empty because exact\-match multiple\-choice scoring produces effectively binary QoA outcomes\. ForSimpleQA, the shift is more gradual: the low\-QoA bucket decreases from87\.86%87\.86\\%to56\.00%56\.00\\%, the middle bucket expands from0\.43%0\.43\\%to28\.00%28\.00\\%, and the high\-QoA bucket increases from11\.71%11\.71\\%to16\.00%16\.00\\%\. This reflects the open\-ended nature ofSimpleQA, where richer history often moves plans from poor to moderate semantic quality before reaching the highest QoA band\. Overall, these results show thatPerfDBprovides an actionable selection signal for multi\-objective planning even when absolute QoA magnitudes remain imperfect\.

### VI\-CImpact of Budget

We evaluate execution under planning\-time financial and latency constraints, leavingEmaxE\_\{\\max\}andQ​o​AminQoA\_\{\\min\}unconstrained101010We budget only*Financial*and*Latency*—the primary levers for fast\-and\-cheap interactive QA; QoA is maximized \(not budgeted\) and Energy closely tracks latency\. Focusing on\(Fmax,Lmax\)\(F\_\{\\max\},L\_\{\\max\}\)captures the user\-facing trade\-off while keeping the analysis simple\.\. A planπ\\piis considered feasible at planning time when its estimated financial cost and latency satisfyFinancial​\(π\)≤Fmax\\mathrm\{Financial\}\(\\pi\)\\leq F\_\{\\max\}andLatency​\(π\)≤Lmax\\mathrm\{Latency\}\(\\pi\)\\leq L\_\{\\max\}; the planner may use up toK=5K\{=\}5operations\. Unless noted otherwise, we use NSGA\-II as the planning backend in this experiment, since it provides the best scalability–quality trade\-off in our planner comparison\. Budgets are instantiated at five levels using the model\-averaged anchors\($​0\.000012,19\.47​s\)\(\\mathdollar 0\.000012,19\.47\\,\\mathrm\{s\}\), and scaled as\(Fmax​\(b\),Lmax​\(b\)\)=b⋅\($​0\.000012,19\.47​s\)\(F\_\{\\max\}\(b\),L\_\{\\max\}\(b\)\)=b\\cdot\(\\mathdollar 0\.000012,19\.47\\,\\mathrm\{s\}\)forb∈\{1,…,5\}b\\in\\\{1,\\ldots,5\\\}\. The anchor tuple specifies the mean observed financial cost and latency across the five distinct models, establishing a standardized baseline for resource allocation\. Budgets are then generated by scaling this composite anchor, ensuring that each constraint is systematically derived from the aggregated typical resource usage of the system\. In this setup,Opti\-Qruns NSGA\-II with feasibility dominance and prunes candidates using plan\-time estimates of cost and QoA\. For each selected plan, we label it as*budget\-adherent*if budget constraints are met after its execution and as an*overrun*otherwise\. For each of the five budgets, we report the*budget\-adherence breakdown*\(see Table[II](https://arxiv.org/html/2607.22621#S6.T2)\) as well as the QoA on budget\-adherent runs \(see Figure[5](https://arxiv.org/html/2607.22621#S6.F5)\)\.

TABLE II:Budget\-adherence breakdown per budget level\.bbBudget\-adherence\(%\)Overrun:Fin\. \(%\)Overrun:Lat\. \(%\)Overrun:Both \(%\)196\.62\.800\.6291\.42\.63\.32\.7395\.81\.51\.21\.5488\.852\.33\.9588\.64\.32\.54\.6The*adherence rate*remains high across budgets \(96\.6% atb=1b\{=\}1to 88\.6% atb=5b\{=\}5\)\. Overruns are predominantly cost\-driven \(1\.5–5\.0%\), with latency\-only overruns at 0\.0–3\.3%, and joint cost\+latency overruns at 0\.6–4\.7%\. Mean QoA on budget\-adherent selections shows a clear knee atb=3b\{=\}3\(0\.45\), with only marginal changes thereafter \(0\.41 atb=4b\{=\}4, 0\.44 atb=5b\{=\}5\), indicating diminishing returns beyond moderate budgets\. Dispersion increases with budget \(IQR0\.31→0\.520\.31\{\\to\}0\.52fromb=1→5b\{=\}1\{\\to\}5\), reflecting higher upside but greater variability under looser constraints\. Budget utilization is conservative: budget\-adherent plans consume 0\.32 – 0\.38 of the financial cost budget and 0\.25–0\.30 of the latency budget, indicating that the planner typically meets the targets without saturating them\. Overall, the planner maintains high budget adherence across all levels, with overruns mainly cost\-driven\. QoA increases sharply at moderate budgets \(b=3b=3\) and then plateaus, while dispersion increases under looser budgets, indicating a trade\-off between upside and variability\.

![Refer to caption](https://arxiv.org/html/2607.22621v1/figures/fig_budget_qoa.png)Figure 5:QoA vs\. budget \(*budget\-adherent plans only*\)\.
### VI\-DQuality–Cost Trade\-offs Against Commercial API Models

We evaluate whetherOpti\-Qremains useful when compared against high\-capability state\-of\-the\-art commercial models accessed through their respective provider APIs \(i\.e\., Claude Opus 4\.6, GPT 5\.4, and Gemini 3\.5 Flash\)\.111111API model versions and prices were taken from provider documentation at evaluation time: Anthropichttps://docs\.anthropic\.com/, OpenAIhttps://platform\.openai\.com/docs/models, and Google Geminihttps://ai\.google\.dev/gemini\-api/docs\.Each commercial model is invoked five times per question using the same benchmark prompt and fixed decoding configuration; we report average QoA and cost across runs\. Financial cost is computed from provider\-reported token usage and public pricing\. We also include the best local single model per benchmark to isolate the benefit of planning over simply choosing the strongest local model\. Since commercial APIs run on provider\-managed backends with hidden hardware, batching, and serving policies, their latency and energy are not directly comparable to our local models served through Ollama\. We therefore compare only consistently measurable metrics across both settings: QoA and financial cost\. All methods use the same sampledMMLU\-ProandSimpleQAquestions and the QoA metrics defined earlier\.

Table[III](https://arxiv.org/html/2607.22621#S6.T3)reports the results of the quality\-cost comparison\. Note thatOpti\-Qprovides a substantially cheaper quality–cost operating point than strong commercial API models\. Local models \(including the ones inOpti\-Q\) where run using Ollama in our server\. Hence, the cost is0\. We report in parenthesis also the cost of running the local models in an external server\. Even in that case, the cost compared to commercial LLMs is minimal\. W\.r\.t\. quality, onSimpleQA,Opti\-Qachieves higher QoA than Claude Opus 4\.6 and GPT 5\.4 while costing, if executed in an external server,37\.8×37\.8\{\\times\}and14\.5×14\.5\{\\times\}less, respectively\. Gemini 3\.5 Flash outperformsOpti\-Q\(\+0\.140\.14\) but it is95\.2×95\.2\{\\times\}more expensive\. OnMMLU\-Prowe observe similar performance\. Hence,Opti\-Qis complementary to commercial API models\. When maximum raw QoA is required, a high\-capability single\-call model may be preferable, but under cost\-sensitive deployment, optimizer\-guided local composition provides a substantially cheaper operating point with competitive \(if not higher\) quality\.

TABLE III:Quality–cost comparison against strong commercial API LLM baselines\.DatasetMethodQoACost\(10−410^\{\-4\}USD\)SimpleQAClaude Opus 4\.60\.5979\.59GPT 5\.40\.6053\.68Gemini 3\.5 Flash0\.79024\.18Best local single \(*LLaMA3\-ChatQA \(8B\)*\)0\.3540\(0\.0710\.071\)Opti\-Q\(NSGA\-II\)0\.650\(0\.2540\.254\)MMLU\-ProClaude Opus 4\.60\.49818\.72GPT 5\.40\.538\.64Gemini 3\.5 Flash0\.87139\.85Best local single \(*Phi\-4\-14B*\)0\.740\(0\.1210\.121\)Opti\-Q\(NSGA\-II\)0\.820\(0\.9090\.909\)
### VI\-EAdditional Sensitivity Analyses

Due to space constraints, we report two additional sensitivity analyses in the extended version\[[39](https://arxiv.org/html/2607.22621#bib.bib39)\]\. First, we evaluate the impact of model diversity and find that performance gains saturate at moderate diversity: the largest marginal QoA improvement occurs fromd=2d=2tod=3d=3distinct models \(Δ​QoA=0\.038\\Delta\\text\{QoA\}=0\.038\), whereas the gain fromd=4d=4tod=5d=5drops to just0\.0080\.008\. Second, we evaluate the impact of the number of operations and find that increasing the number of operations fromk=1k=1tok=5k=5improves QoA by23\.6%23\.6\\%on SimpleQA and54\.7%54\.7\\%on MMLU\-Pro; atk=5k=5, parallel and hybrid plans account for82%82\\%of the plans selected by Opti\-Q\.

## VIIConclusion and Future Work

We presentedOpti\-Q, a database\-inspired optimizer for per\-question multi\-LLM QA planning\.Opti\-Qrepresents LLM invocations as physical operators in a DAG, estimates QoA and resource usage usingPerfDB, and searches for Pareto\-efficient sequential, parallel, and hybrid plans under user\-specified constraints\. Across MMLU\-Pro and SimpleQA,Opti\-Qachieves higher QoA at comparable cost than strong routing, cascading, and ensembling baselines, demonstrating that plan\-before\-execute optimization can improve quality–resource trade\-offs in multi\-LLM QA\. These results show that structured, statistics\-driven planning provides a practical foundation for adaptive LLM orchestration as model choices, budgets, and deployment backends continue to diversify\.

A natural next step is to extend the same plan\-before\-execute abstraction beyond QA to richer RAG and agentic workflows\. Because such pipelines can be expressed as DAGs over a broader operator vocabulary \(e\.g\., retrieval, reranking, verification, and tool execution alongside generation and blending\) each carrying its own quality and resource profile inPerfDB,Opti\-Q’s planning machinery should transfer with little conceptual change\. Realizing this will require characterizing these new operators’ cost–benefit behavior and validating that the estimators remain reliable as plans grow longer and more heterogeneous, which we leave to future work\.

## References

- \[1\]OpenAI, “Introducing gpt\-5,”https://openai\.com/index/introducing\-gpt\-5/, 2025, accessed: 2026\-02\-20\.
- \[2\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan*et al\.*, “The llama 3 herd of models,”*arXiv preprint arXiv:2407\.21783*, 2024\.
- \[3\]D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi*et al\.*, “Deepseek\-r1: Incentivizing reasoning capability in llms via reinforcement learning,”*arXiv preprint arXiv:2501\.12948*, 2025\.
- \[4\]J\. Devlin, M\.\-W\. Chang, K\. Lee, and K\. Toutanova, “Bert: Pre\-training of deep bidirectional transformers for language understanding,” 2019\. \[Online\]\. Available:https://arxiv\.org/abs/1810\.04805
- \[5\]D\. Chen, A\. Fisch, J\. Weston, and A\. Bordes, “Reading wikipedia to answer open\-domain questions,” 2017\. \[Online\]\. Available:https://arxiv\.org/abs/1704\.00051
- \[6\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. tau Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela, “Retrieval\-augmented generation for knowledge\-intensive nlp tasks,” 2021\. \[Online\]\. Available:https://arxiv\.org/abs/2005\.11401
- \[7\]V\. Karpukhin, B\. Oğuz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. tau Yih, “Dense passage retrieval for open\-domain question answering,” 2020\. \[Online\]\. Available:https://arxiv\.org/abs/2004\.04906
- \[8\]J\. Gao, M\. Galley, and L\. Li, “Neural approaches to conversational ai: Question answering, task\-oriented dialogues and social chatbots,” 2019\.
- \[9\]T\. W\. Bickmore and R\. W\. Picard, “Establishing and maintaining long\-term human\-computer relationships,”*ACM Trans\. Comput\. Hum\. Interact\.*, vol\. 12, pp\. 293–327, 2005\. \[Online\]\. Available:https://api\.semanticscholar\.org/CorpusID:946518
- \[10\]A\. Hamid, H\. R\. Samidi, T\. Finin, P\. Pappachan, and R\. Yus, “Genaipabench: A benchmark for generative ai\-based privacy assistants,”*arXiv preprint arXiv:2309\.05138*, vol\. n/a, no\. n/a, p\. n/a, 2023\.
- \[11\]B\. Cottier, R\. Rahman, L\. Fattorini, N\. Maslej, T\. Besiroglu, and D\. Owen, “The rising costs of training frontier ai models,”*arXiv preprint arXiv:2405\.21015*, vol\. n/a, no\. n/a, p\. n/a, 2024\.
- \[12\]E\. Strubell, A\. Ganesh, and A\. McCallum, “Energy and policy considerations for deep learning in NLP,” in*Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, A\. Korhonen, D\. Traum, and L\. Màrquez, Eds\. Florence, Italy: Association for Computational Linguistics, Jul\. 2019, pp\. 3645–3650\. \[Online\]\. Available:https://aclanthology\.org/P19\-1355
- \[13\]D\. Patterson, J\. Gonzalez, U\. Hölzle, Q\. Le, C\. Liang, L\.\-M\. Munguia, D\. Rothchild, D\. So, M\. Texier, and J\. Dean, “The carbon footprint of machine learning training will plateau, then shrink,” 2022\. \[Online\]\. Available:https://arxiv\.org/abs/2204\.05149
- \[14\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou*et al\.*, “Chain\-of\-thought prompting elicits reasoning in large language models,”*Advances in neural information processing systems*, vol\. 35, pp\. 24 824–24 837, 2022\.
- \[15\]J\. Wei, N\. Karina, H\. W\. Chung, Y\. J\. Jiao, S\. Papay, A\. Glaese, J\. Schulman, and W\. Fedus, “Measuring short\-form factuality in large language models,” 2024\. \[Online\]\. Available:https://arxiv\.org/abs/2411\.04368
- \[16\]S\. Feng, W\. Ding, A\. Liu, Z\. Wang, W\. Shi, Y\. Wang, Z\. Shen, X\. Han, H\. Lang, C\.\-Y\. Lee, T\. Pfister, Y\. Choi, and Y\. Tsvetkov, “When one llm drools, multi\-llm collaboration rules,” 2025\. \[Online\]\. Available:https://arxiv\.org/abs/2502\.04506
- \[17\]T\. G\. Dietterich, “Ensemble methods in machine learning,” in*Proceedings of the First International Workshop on Multiple Classifier Systems*, ser\. MCS ’00\. Berlin, Heidelberg: Springer\-Verlag, 2000, p\. 1–15\.
- \[18\]X\. Amatriain, “Prompt design and engineering: Introduction and advanced methods,” 2024\. \[Online\]\. Available:https://arxiv\.org/abs/2401\.14423
- \[19\]S\. Suzuoki and K\. Hatano, “Reducing hallucinations in large language models: A consensus voting approach using mixture of experts,” 2024\.
- \[20\]P\. Dey, S\. Merugu, and S\. Kaveri, “Uncertainty\-aware fusion: An ensemble framework for mitigating hallucinations in large language models,”*arXiv preprint arXiv:2503\.05757*, vol\. n/a, no\. n/a, p\. n/a, 2025\.
- \[21\]D\. Jiang, X\. Ren, and B\. Y\. Lin, “Llm\-blender: Ensembling large language models with pairwise ranking and generative fusion,” 2023\. \[Online\]\. Available:https://arxiv\.org/abs/2306\.02561
- \[22\]LangChain,*LangChain Documentation*, 2024, accessed: 2024\-02\-05\. \[Online\]\. Available:https://python\.langchain\.com/docs/introduction/
- \[23\]C\. Liu, M\. Russo, M\. Cafarella, L\. Cao, P\. B\. Chen, Z\. Chen, M\. Franklin, T\. Kraska, S\. Madden, R\. Shahout, and G\. Vitagliano, “Palimpzest: Optimizing AI\-powered analytics with declarative query processing,” in*Proceedings of the Conference on Innovative Database Research \(CIDR\)*, 2025\.
- \[24\]R\. Marcus, P\. Negi, H\. Mao, N\. Tatbul, M\. Alizadeh, and T\. Kraska, “Bao: Making learned query optimization practical,” in*Proceedings of the 2021 International Conference on Management of Data*, ser\. SIGMOD ’21\. New York, NY, USA: Association for Computing Machinery, 2021, p\. 1275–1288\. \[Online\]\. Available:https://doi\.org/10\.1145/3448016\.3452838
- \[25\]Y\. Han, Z\. Wu, P\. Wu, R\. Zhu, J\. Yang, L\. W\. Tan, K\. Zeng, G\. Cong, Y\. Qin, A\. Pfadler, Z\. Qian, J\. Zhou, J\. Li, and B\. Cui, “Cardinality estimation in dbms: a comprehensive benchmark evaluation,”*Proc\. VLDB Endow\.*, vol\. 15, no\. 4, p\. 752–765, Dec\. 2021\. \[Online\]\. Available:https://doi\.org/10\.14778/3503585\.3503586
- \[26\]Z\. Yao, H\. Li, J\. Zhang, C\. Li, and H\. Chen, “A query optimization method utilizing large language models,”*arXiv preprint arXiv:2503\.06902*, 2025\.
- \[27\]R\. Zhu, W\. Chen, B\. Ding, X\. Chen, A\. Pfadler, Z\. Wu, and J\. Zhou, “Lero: A learning\-to\-rank query optimizer,”*Proc\. VLDB Endow\.*, vol\. 16, no\. 6, p\. 1466–1479, Feb\. 2023\. \[Online\]\. Available:https://doi\.org/10\.14778/3583140\.3583160
- \[28\]O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. Vardhamanan, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam, H\. Miller, M\. Zaharia, and C\. Potts, “Dspy: Compiling declarative language model calls into self\-improving pipelines,” 2024\.
- \[29\]L\. Chen, M\. Zaharia, and J\. Zou, “Frugalgpt: How to use large language models while reducing cost and improving performance,” 2023\. \[Online\]\. Available:https://arxiv\.org/abs/2305\.05176
- \[30\]S\. F\. Tekin, F\. Ilhan, T\. Huang, S\. Hu, and L\. Liu, “Llm\-topla: Efficient llm ensemble by maximising diversity,” 2024\. \[Online\]\. Available:https://arxiv\.org/abs/2410\.03953
- \[31\]K\. Huang, Y\. Shi, D\. Ding, Y\. Li, Y\. Fei, L\. Lakshmanan, and X\. Xiao, “Thriftllm: On cost\-effective selection of large language models for classification queries,”*Proc\. VLDB Endow\.*, vol\. 18, no\. 11, p\. 4410–4423, Jul\. 2025\. \[Online\]\. Available:https://doi\.org/10\.14778/3749646\.3749702
- \[32\]S\. Shekhar, T\. Dubey, K\. Mukherjee, A\. Saxena, A\. Tyagi, and N\. Kotla, “Towards optimizing the costs of llm usage,” 2024\. \[Online\]\. Available:https://arxiv\.org/abs/2402\.01742
- \[33\]M\. Saeed, N\. D\. Cao, and P\. Papotti, “Querying large language models with sql,” in*International Conference on Extending Database Technology*, 2023\. \[Online\]\. Available:https://api\.semanticscholar\.org/CorpusID:257913347
- \[34\]M\. Russo, S\. Sudhir, G\. Vitagliano, C\. Liu, T\. Kraska, S\. Madden, and M\. J\. Cafarella, “Abacus: A cost\-based optimizer for semantic operator systems,”*ArXiv*, vol\. abs/2505\.14661, 2025\. \[Online\]\. Available:https://api\.semanticscholar\.org/CorpusID:278768629
- \[35\]G\. Sanmartino, M\. Urban, P\. Papotti, and C\. Binnig, “The stretto execution engine for llm\-augmented data systems,” 2026\. \[Online\]\. Available:https://arxiv\.org/abs/2602\.04430
- \[36\]M\. Durasevic, F\. J\. Gil\-Gala, and D\. Jakobović, “Does size matter? on the influence of ensemble size on constructing ensembles of dispatching rules,” in*Proceedings of the Companion Conference on Genetic and Evolutionary Computation*, ser\. GECCO ’23 Companion\. New York, NY, USA: Association for Computing Machinery, 2023, p\. 559–562\. \[Online\]\. Available:https://doi\.org/10\.1145/3583133\.3590562
- \[37\]J\.\-W\. Chung, J\. Liu, Z\. Wu, Y\. Xia, and M\. Chowdhury, “ML\.ENERGY leaderboard,”https://ml\.energy/leaderboard, 2023\.
- \[38\]K\. Deb, A\. Pratap, S\. Agarwal, and T\. Meyarivan, “A fast and elitist multiobjective genetic algorithm: Nsga\-ii,”*IEEE Transactions on Evolutionary Computation*, vol\. 6, no\. 2, pp\. 182–197, 2002\.
- \[39\]A\. Hamid and coauthors, “Opti\-q extended version,”https://github\.com/Aamir7693/Opti\-Q/blob/main/extended\_version/extended\_version\.pdf, 2026, extended version\.
- \[40\]J\. You, J\. Chung, and M\. Chowdhury, “Zeus: Understanding and optimizing gpu energy consumption of dnn training,”*ArXiv*, vol\. abs/2208\.06102, no\. n/a, p\. n/a, 2022\. \[Online\]\. Available:https://api\.semanticscholar\.org/CorpusID:251554526
- \[41\]I\. Trummer and C\. Koch, “A fast randomized algorithm for multi\-objective query optimization,” in*Proceedings of the 2016 International Conference on Management of Data*, 2016, pp\. 1737–1752\.
- \[42\]Gemma Team, “Gemma,” Kaggle Model Artifact, 2024\. \[Online\]\. Available:https://www\.kaggle\.com/m/3301
- \[43\]Z\. Liu, W\. Ping, R\. Roy, P\. Xu, C\. Lee, M\. Shoeybi, and B\. Catanzaro, “Chatqa: Surpassing gpt\-4 on conversational qa and rag,”*arXiv preprint arXiv:2401\.10225*, vol\. n/a, no\. n/a, p\. n/a, 2024\.
- \[44\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv*et al\.*, “Qwen3 technical report,”*arXiv preprint arXiv:2505\.09388*, 2025\.
- \[45\]M\. Abdin, J\. Aneja, H\. Behl, S\. Bubeck, R\. Eldan, S\. Gunasekar, M\. Harrison, R\. J\. Hewett, M\. Javaheripi, P\. Kauffmann, J\. R\. Lee, Y\. T\. Lee, Y\. Li, W\. Liu, C\. C\. T\. Mendes, A\. Nguyen, E\. Price, G\. de Rosa, O\. Saarikivi, A\. Salim, S\. Shah, X\. Wang, R\. Ward, Y\. Wu, D\. Yu, C\. Zhang, and Y\. Zhang, “Phi\-4 technical report,” 2024\. \[Online\]\. Available:https://arxiv\.org/abs/2412\.08905
- \[46\]A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\.\-A\. Lachaux, P\. Stock, T\. L\. Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. E\. Sayed, “Mistral 7b,” 2023\. \[Online\]\. Available:https://arxiv\.org/abs/2310\.06825
- \[47\]F\. Ye, M\. Yang, J\. Pang, L\. Wang, D\. Wong, E\. Yilmaz, S\. Shi, and Z\. Tu, “Benchmarking llms via uncertainty quantification,”*Advances in Neural Information Processing Systems*, vol\. 37, pp\. 15 356–15 385, 2024\.
- \[48\]M\. Grootendorst, “Bertopic: Neural topic modeling with a class\-based tf\-idf procedure,”*arXiv preprint arXiv:2203\.05794*, vol\. n/a, no\. n/a, p\. n/a, 2022\.
- \[49\]L\. McInnes, J\. Healy, and J\. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,” 2020\. \[Online\]\. Available:https://arxiv\.org/abs/1802\.03426
- \[50\]L\. McInnes, J\. Healy, S\. Astels*et al\.*, “hdbscan: Hierarchical density based clustering\.”*J\. Open Source Softw\.*, vol\. 2, no\. 11, p\. 205, 2017\.
- \[51\]T\. Kojima, S\. S\. Gu, M\. Reid, Y\. Matsuo, and Y\. Iwasawa, “Large language models are zero\-shot reasoners,”*Advances in neural information processing systems*, vol\. 35, pp\. 22 199–22 213, 2022\.
- \[52\]T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell*et al\.*, “Language models are few\-shot learners,”*Advances in neural information processing systems*, vol\. 33, pp\. 1877–1901, 2020\.
- \[53\]K\. Papineni, S\. Roukos, T\. Ward, and W\.\-J\. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in*Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics*, P\. Isabelle, E\. Charniak, and D\. Lin, Eds\. Philadelphia, Pennsylvania, USA: Association for Computational Linguistics, Jul\. 2002, pp\. 311–318\. \[Online\]\. Available:https://aclanthology\.org/P02\-1040/
- \[54\]C\.\-Y\. Lin, “ROUGE: A package for automatic evaluation of summaries,” in*Text Summarization Branches Out*\. Barcelona, Spain: Association for Computational Linguistics, Jul\. 2004, pp\. 74–81\. \[Online\]\. Available:https://aclanthology\.org/W04\-1013/
- \[55\]Z\. Chen, J\. Li, P\. Chen, Z\. Li, K\. Sun, Y\. Luo, Q\. Mao, D\. Yang, H\. Sun, and P\. S\. Yu, “Harnessing multiple large language models: A survey on llm ensemble,” 2025\. \[Online\]\. Available:https://arxiv\.org/abs/2502\.18036