Toward Reliable Design of LLM-Enabled Agentic Workflows: Optimizing Latency-Reliability-Cost Tradeoffs

# Toward Reliable Design of LLM-Enabled Agentic Workflows: Optimizing Latency-Reliability-Cost Tradeoffs
Source: [https://arxiv.org/html/2605.23929](https://arxiv.org/html/2605.23929)
###### Abstract

Modern AI systems increasingly rely on workflows composed of multiple interacting agents, some powered by large language models \(LLMs\) and others by conventional computational modules\. This paper analyzes the fundamental tradeoffs between latency, reliability, and cost in LLM\-enabled agentic workflows\. We introduce performance models for both LLM and non\-LLM agents that capture the relationship between computational effort and output quality, incorporating the impact of reasoning and output tokens for LLM agents using a parametric exponential reliability function\. Then, we study the design of sequential workflows under latency and cost constraints\. Main results include a water\-filling token allocation policy and characterizations of optimal workflow reliability in terms of shadow prices\.

## 1Introduction

With recent advances in artificial intelligence \(AI\), particularly large language models \(LLMs\), modern systems increasingly rely on workflows composed of multiple interacting agents\. In these systems, agents collaborate to solve complex tasks by exchanging intermediate information and progressively refining their outputs\[[1](https://arxiv.org/html/2605.23929#bib.bib2),[6](https://arxiv.org/html/2605.23929#bib.bib7)\]\. These agents may include LLM\-based components as well as conventional computational modules, such as database queries, optimization solvers, and verification tools\. As these workflows grow in scale and complexity, their deployment also becomes increasingly resource\-intensive\. This raises fundamental challenges in how to configure LLM\-enabled systems efficiently and reliably\. In practice, system design is often guided by heuristics or costly trial\-and\-error\. Hence, there is a need for mathematical and principle frameworks to analyze and optimize agentic LLM workflows\.

From a systems perspective, workflow performance can be characterized by three key quantities: latency, reliability, and cost\. Optimizing these performance metrics requires understanding how individual agent characteristics combine to determine overall workflow behavior\. This study develops performance models for LLM\-enabled agentic workflows and derives optimal resource allocation policies under latency and cost constraints\. We introduce parametric models that capture the relationship between computational effort and output quality for both LLM and non\-LLM agents\. Then, we focus on sequential workflows, formulate the design problem as maximizing reliability subject to resource budgets, and derive closed\-form allocation policies using convex optimization method\[[2](https://arxiv.org/html/2605.23929#bib.bib3)\]\. Our results provide designers with principled methods for configuring LLM\-based systems to achieve desired performance tradeoffs\.

## 2LLM Agentic Workflows

An LLM agentic workflow can be represented as a directed computation graph where nodes correspond to agents and edges represent information flow\. Tasks enter the workflow as input messages, propagate through agents, and eventually produce outputs\.

### 2\.1LLM Agent Performance Metrics

In practice, the execution of an LLM agent involves two distinct computational stages\. The agent first performs internal reasoning to construct a solution to the task\. Then, it generates a sequence of tokens that communicates the solution to downstream agents or to the user\. Both stages consume computational resources and therefore influence latency, reliability, and cost\. Considering an LLM agent indexed byjj, we define two key resource variables:XjX\_\{j\}denotes the number of “reasoning tokens” generated internally during the reasoning process, andLjL\_\{j\}denotes the number of “output tokens” for the final response\. Both quantities are measured in units of tokens\. The variableXjX\_\{j\}represents the internal computational effort, whileLjL\_\{j\}determines the length of the communicated response\.

Latency:The latency of an LLM agent depends on network conditions, system scheduling, and variability in model execution\. We model the latency of agentjjas a random variableTjT\_\{j\}measured in units of time \(e\.g\., seconds\)\. The latency consists of two components: infrastructure delay and model inference\. The infrastructure delay is denoted byτj\\tau\_\{j\}, which captures network transmission and orchestration overhead\. This quantity is treated as a random variable with meanτ¯j=𝔼​\[τj\]\\bar\{\\tau\}\_\{j\}=\\mathbb\{E\}\[\\tau\_\{j\}\]\. During model inference, processing rates differ in stages\[[4](https://arxiv.org/html/2605.23929#bib.bib6)\]\. We denote byλth\\lambda\_\{\\text\{th\}\}the average processing rate for reasoning tokens and byλgen\\lambda\_\{\\text\{gen\}\}the average generation rate for output tokens\. The total latency of agentjjisTj=τj\+Xjλth\+LjλgenT\_\{j\}=\\tau\_\{j\}\+\\frac\{X\_\{j\}\}\{\\lambda\_\{\\text\{th\}\}\}\+\\frac\{L\_\{j\}\}\{\\lambda\_\{\\text\{gen\}\}\}, where the first term captures the stochastic infrastructure delay, and the remaining terms represent deterministic computation time\. Taking expectations yields the mean latency

T¯j=τ¯j\+\(Xj/λth\)\+\(Lj/λgen\),\\bar\{T\}\_\{j\}=\\bar\{\\tau\}\_\{j\}\+\(X\_\{j\}/\\lambda\_\{\\text\{th\}\}\)\+\(L\_\{j\}/\\lambda\_\{\\text\{gen\}\}\),\(1\)which we use as the latency metric in subsequent analysis\.

Reliability:The reliability of agentjjis defined as the probability that the output produced by the agent is acceptable\. We define the reliability functionρj:ℝ\+2→\[0,1\]\\rho\_\{j\}:\\mathbb\{R\}^\{2\}\_\{\+\}\\to\[0,1\]byρj​\(Xj,Lj\)=Pr⁡\(output of​j​is acceptable\|Xj,Lj\)\\rho\_\{j\}\(X\_\{j\},L\_\{j\}\)=\\Pr\(\\text\{output of \}j\\text\{ is acceptable\}\|X\_\{j\},L\_\{j\}\)\. This function captures how the quality of the agent’s output depends on the computational resources allocated to reasoning and generation\. Typically, increasing the reasoning effortXjX\_\{j\}improves the quality of the internal solution; increasing the output lengthLjL\_\{j\}may enhance the clarity, explanation, or verification of the result\[[5](https://arxiv.org/html/2605.23929#bib.bib5)\]\. To capture diminishing marginal returns in both dimensions, we adopt the following parametric form:

ρj​\(Xj,Lj\)=\(1−e−αj​Xj\)​\(1−e−βj​Lj\),\\rho\_\{j\}\(X\_\{j\},L\_\{j\}\)=\\big\(1\-e^\{\-\\alpha\_\{j\}X\_\{j\}\}\\big\)\\big\(1\-e^\{\-\\beta\_\{j\}L\_\{j\}\}\\big\),\(2\)whereαj\>0\\alpha\_\{j\}\>0andβj\>0\\beta\_\{j\}\>0are agent\-specific parameters in units of\(tokens\)−1\(\\text\{tokens\}\)^\{\-1\}\. This choice ensures that the arguments of the exponential functions are dimensionless and thatρj​\(Xj,Lj\)∈\[0,1\]\\rho\_\{j\}\(X\_\{j\},L\_\{j\}\)\\in\[0,1\]\.

Cost:There are two distinct cost notions that arise in LLM\-based systems\. First, the user\-visible cost is associated with the number of output tokens generated\. We denote byctok\>0c\_\{\\text\{tok\}\}\>0the price per output token, measured in units of currency per token\. The resulting user cost of agentjjisCju​\(Lj\)=ctok​LjC^\{u\}\_\{j\}\(L\_\{j\}\)=c\_\{\\text\{tok\}\}L\_\{j\}\. Second, the system provider incurs computational cost when processing tokens\. Since both reasoning tokens and output tokens must be processed by the model, the total token workload equalsXj\+LjX\_\{j\}\+L\_\{j\}\. Letccomp\>0c\_\{\\text\{comp\}\}\>0denote the computational cost per processed token \(in units of currency per token\)\. The corresponding computational cost isCjc​\(Xj,Lj\)=ccomp​\(Xj\+Lj\)C^\{c\}\_\{j\}\(X\_\{j\},L\_\{j\}\)=c\_\{\\text\{comp\}\}\(X\_\{j\}\+L\_\{j\}\)\.

### 2\.2Non\-LLM Agent Performance Metrics

Non\-LLM agents typically perform deterministic or algorithmic computations on structured inputs and do not generate token sequences\. In many cases, the latencyTiT\_\{i\}of agentiican be modeled using queueing\-theoretic abstractions\. For instance, if the service time follows an exponential distribution with rateλi\\lambda\_\{i\}\(measured in\(time\)−1\(\\text\{time\}\)^\{\-1\}\), the expected latency is𝔼​\[Ti\]=1/λi\\mathbb\{E\}\[T\_\{i\}\]=1/\\lambda\_\{i\}\. The reliability of the agent is defined as the probability that the agent produces a correct output given a valid input, denoted byρi∈\[0,1\]\\rho\_\{i\}\\in\[0,1\]\. The pair\(Ti,ρi\)\(T\_\{i\},\\rho\_\{i\}\)therefore provides a compact representation of the latency and reliability characteristics of a non\-LLM agent\.

### 2\.3Workflow Aggregation

Agentic workflows combine heterogeneous agents in different structural configurations\. Letkkdenote an arbitrary agent with latencyTkT\_\{k\}and reliabilityρk∈\[0,1\]\\rho\_\{k\}\\in\[0,1\]\. The performance of a workflow depends not only on the characteristics of individual agents but also on the structural configuration through which agents interact\. We consider three common configurations: sequential composition, parallel composition, and feedback \(iterative\) loops\.

Sequential:In a sequential configuration, the output of one agent becomes the input to the next\. Consider a pipeline ofnnagents indexed byk=1,…,nk=1,\\ldots,n\. Since each stage must complete before the next stage begins, the expected workflow latency is𝔼​\[Tseq\]=∑k=1n𝔼​\[Tk\]\\mathbb\{E\}\[T\_\{\\text\{seq\}\}\]=\\sum\_\{k=1\}^\{n\}\\mathbb\{E\}\[T\_\{k\}\]\. The workflow produces a correct result only if all agents succeed, yielding the reliabilityRseq=∏k=1nρkR\_\{\\text\{seq\}\}=\\prod\_\{k=1\}^\{n\}\\rho\_\{k\}\.

Parallel:In a parallel configuration, multiple agents process the same input simultaneously\. The latency of the parallel stage is determined by the critical path\. For two agentsiiandjj, the expected latency is𝔼​\[Ti∥j\]=𝔼​\[max⁡\(Ti,Tj\)\]\\mathbb\{E\}\[T\_\{i\\\|j\}\]=\\mathbb\{E\}\[\\max\(T\_\{i\},T\_\{j\}\)\]\. Reliability, however, depends on how the outputs are aggregated\. If both outputs are required for the workflow to succeed, the reliability isRi∥j=ρi​ρjR\_\{i\\\|j\}=\\rho\_\{i\}\\rho\_\{j\}\. In contrast, if the workflow succeeds whenever at least one branch produces a correct result, the reliability becomesRi∥j=1−\(1−ρi\)​\(1−ρj\)R\_\{i\\\|j\}=1\-\(1\-\\rho\_\{i\}\)\(1\-\\rho\_\{j\}\)\. These two cases correspond to conjunctive and redundant parallel execution patterns, respectively\.

Feedback:Many agentic workflows employ iterative reasoning in which agents repeatedly exchange information to refine a solution\. Such structures arise in debate systems, verification loops, and iterative plannings\. Consider a feedback loop in which agentsiiandjjexecute in the sequencei→ji\\to jforKKiterations\. Each iteration has latencyTi\+TjT\_\{i\}\+T\_\{j\}, so the expected latency afterKKiterations is𝔼​\[Tfb\]=K​\(𝔼​\[Ti\]\+𝔼​\[Tj\]\)\\mathbb\{E\}\[T\_\{\\text\{fb\}\}\]=K\(\\mathbb\{E\}\[T\_\{i\}\]\+\\mathbb\{E\}\[T\_\{j\}\]\)\. If both agents must produce correct outputs in each iteration, the reliability becomesRfb=\(ρi​ρj\)KR\_\{\\text\{fb\}\}=\(\\rho\_\{i\}\\rho\_\{j\}\)^\{K\}\.

## 3Performance Optimization

We now turn to the design problem of optimizing agentic workflows\. In this work, we focus on the case of sequential composition\. It is worth noting that studying sequential workflows is not a loss of generality: more complex architectures constructed from parallel or feedback structures can be viewed as sequential compositions of composite modules, where each module corresponds to a sub\-workflow whose latency and reliability are determined by the aggregation rules introduced in Section[2\.3](https://arxiv.org/html/2605.23929#S2.SS3)\.

Consider a sequential workflow consisting of agents indexed byk=1,…,nk=1,\\ldots,n\. Let𝒜LLM\\mathcal\{A\}\_\{\\text\{LLM\}\}denote the set of LLM agents and𝒜NLLM\\mathcal\{A\}\_\{\\text\{NLLM\}\}denote the set of non\-LLM agents\. For non\-LLM agentsi∈𝒜NLLMi\\in\\mathcal\{A\}\_\{\\text\{NLLM\}\}, the expected latency𝔼​\[Ti\]\\mathbb\{E\}\[T\_\{i\}\]and reliabilityρi\\rho\_\{i\}are fixed parameters determined by the system implementation\. For LLM agentsj∈𝒜LLMj\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}, the response lengthLjL\_\{j\}is a decision variable chosen by the system designer\. For LLM agents, expected latency𝔼​\[Tj​\(Lj\)\]\\mathbb\{E\}\[T\_\{j\}\(L\_\{j\}\)\]follows \([1](https://arxiv.org/html/2605.23929#S2.E1)\) and the reliabilityρj​\(Lj\)\\rho\_\{j\}\(L\_\{j\}\)follows \([2](https://arxiv.org/html/2605.23929#S2.E2)\)\.

Since agents operate sequentially, the expected latency of the workflow equals the sum of individual expected latencies,𝔼​\[Twf​\(L\)\]=∑i∈𝒜NLLM𝔼​\[Ti\]\+∑j∈𝒜LLM𝔼​\[Tj​\(Lj\)\]\\mathbb\{E\}\[T\_\{\\text\{wf\}\}\(L\)\]=\\sum\_\{i\\in\\mathcal\{A\}\_\{\\text\{NLLM\}\}\}\\mathbb\{E\}\[T\_\{i\}\]\+\\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\mathbb\{E\}\[T\_\{j\}\(L\_\{j\}\)\]\. Then, assuming independent failures across agents, the workflow reliability equals the product of the reliabilities of all agents in the pipeline\. For non\-LLM agentsi∈𝒜NLLMi\\in\\mathcal\{A\}\_\{\\text\{NLLM\}\}, the reliabilityρi∈\[0,1\]\\rho\_\{i\}\\in\[0,1\]is a fixed parameter determined by the system implementation\. For LLM agentsj∈𝒜LLMj\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}, reliability depends on both the reasoning effortXjX\_\{j\}and the output lengthLjL\_\{j\}\. The overall workflow reliability is thereforeRwf​\(X,L\)=∏i∈𝒜NLLMρi​∏j∈𝒜LLMρj​\(Xj,Lj\)R\_\{\\text\{wf\}\}\(X,L\)=\\prod\_\{i\\in\\mathcal\{A\}\_\{\\text\{NLLM\}\}\}\\rho\_\{i\}\\prod\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\rho\_\{j\}\(X\_\{j\},L\_\{j\}\)\.

In the subsequent analysis, we treat the reasoning allocationsXjX\_\{j\}as fixed parameters determined by the internal reasoning policy of each agent\. We therefore introduce the reduced reliability functionρ^j​\(Lj\):=ρj​\(Xj,Lj\)\\hat\{\\rho\}\_\{j\}\(L\_\{j\}\):=\\rho\_\{j\}\(X\_\{j\},L\_\{j\}\), which depends only on the response lengthLjL\_\{j\}\. Under this assumption, the workflow reliability becomesRwf​\(L\)=∏i∈𝒜NLLMρi​∏j∈𝒜LLMρ^j​\(Lj\)R\_\{\\text\{wf\}\}\(L\)=\\prod\_\{i\\in\\mathcal\{A\}\_\{\\text\{NLLM\}\}\}\\rho\_\{i\}\\prod\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\hat\{\\rho\}\_\{j\}\(L\_\{j\}\)\.

### 3\.1Latency\-Cost\-Constrained Design

Suppose that the workflow must satisfy both a latency constraintTTand a user cost budgetCC\. The system designer selects the response lengthsLjL\_\{j\}for the LLM agents to maximize the reliability of the workflow\. The resulting design problem can be written as

maxLj,j∈𝒜LLM\\displaystyle\\max\_\{L\_\{j\},j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}∏j∈𝒜LLMρ^j​\(Lj\)\\displaystyle\\prod\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\hat\{\\rho\}\_\{j\}\(L\_\{j\}\)\(3\)s\.t\.∑j∈𝒜LLMLjλgen≤T−Tfixed,∑j∈𝒜LLMCju​\(Lj\)≤C,\\displaystyle\\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\frac\{L\_\{j\}\}\{\\lambda\_\{\\text\{gen\}\}\}\\leq T\-T\_\{\\text\{fixed\}\},\\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}C^\{u\}\_\{j\}\(L\_\{j\}\)\\leq C,\(4\)Lj≥0,j∈𝒜LLM,\\displaystyle L\_\{j\}\\geq 0,\\quad j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\},\(5\)whereTfixed=∑i∈𝒜NLLM𝔼​\[Ti\]\+∑j∈𝒜LLM\(τ¯j\+Xjλth\)T\_\{\\text\{fixed\}\}=\\sum\_\{i\\in\\mathcal\{A\}\_\{\\text\{NLLM\}\}\}\\mathbb\{E\}\[T\_\{i\}\]\+\\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\big\(\\bar\{\\tau\}\_\{j\}\+\\frac\{X\_\{j\}\}\{\\lambda\_\{\\text\{th\}\}\}\\big\)represents the latency that does not depend on the response\. Since the logarithm is a strictly increasing function, maximizing the product in \([3](https://arxiv.org/html/2605.23929#S3.E3)\) is equivalent to maximizing the sum of logarithms\. We have

maxLj,j∈𝒜LLM\\displaystyle\\max\_\{L\_\{j\},j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}∑j∈𝒜LLMlog⁡ρ^j​\(Lj\)s\.t\.constraints\([4](https://arxiv.org/html/2605.23929#S3.E4)\),\([5](https://arxiv.org/html/2605.23929#S3.E5)\)\\displaystyle\\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\log\\hat\{\\rho\}\_\{j\}\(L\_\{j\}\)\\quad\\text\{s\.t\.\}\\text\{ constraints \}\\eqref\{eq:TC\_constraints\},\\eqref\{eq:L\_constraints\}\(6\)

### 3\.2Optimal Token Allocation

We now solve the log\-equivalent problem \([6](https://arxiv.org/html/2605.23929#S3.E6)\)\. We denote byBT=T−TfixedB\_\{T\}=T\-T\_\{\\text\{fixed\}\}the latency budget available for output generation\. Since the reasoning allocationsXjX\_\{j\}are treated as fixed parameters, the reduced reliability function isρ^j​\(Lj\)=\(1−e−αj​Xj\)​\(1−e−βj​Lj\)\\hat\{\\rho\}\_\{j\}\(L\_\{j\}\)=\\big\(1\-e^\{\-\\alpha\_\{j\}X\_\{j\}\}\\big\)\\big\(1\-e^\{\-\\beta\_\{j\}L\_\{j\}\}\\big\)\. Substituting the reliability model in \([2](https://arxiv.org/html/2605.23929#S2.E2)\), the objective function in \([6](https://arxiv.org/html/2605.23929#S3.E6)\) becomes∑j∈𝒜LLMlog⁡\(1−e−αj​Xj\)\+∑j∈𝒜LLMlog⁡\(1−e−βj​Lj\)\\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\log\(1\-e^\{\-\\alpha\_\{j\}X\_\{j\}\}\)\+\\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\log\(1\-e^\{\-\\beta\_\{j\}L\_\{j\}\}\)\. SinceXjX\_\{j\}is fixed, the first term is constant with respect to the decision variablesLjL\_\{j\}\. Consequently, the optimization over response lengths is equivalent to maximizing only the second term\.

maxLj,j∈𝒜LLM\\displaystyle\\max\_\{L\_\{j\},j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}∑j∈𝒜LLMlog⁡\(1−e−βj​Lj\)\\displaystyle\\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\log\\big\(1\-e^\{\-\\beta\_\{j\}L\_\{j\}\}\\big\)\(7\)s\.t\.∑j∈𝒜LLMLjλgen≤BT,∑j∈𝒜LLMctok​Lj≤C,Lj≥0\.\\displaystyle\\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\frac\{L\_\{j\}\}\{\\lambda\_\{\\text\{gen\}\}\}\\leq B\_\{T\},\\ \\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}c\_\{\\text\{tok\}\}L\_\{j\}\\leq C,\\ L\_\{j\}\\geq 0\.
###### Proposition 1\(Water\-filling token allocation\)

Consider problem \([7](https://arxiv.org/html/2605.23929#S3.E7)\) and assumeβj\>0,∀j∈𝒜LLM\\beta\_\{j\}\>0,\\forall j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\. Define the effective token budget asB=min⁡\{λgen​BT,C/ctok\}B=\\min\\left\\\{\\lambda\_\{\\text\{gen\}\}B\_\{T\},\\,C/c\_\{\\text\{tok\}\}\\right\\\}\. Then the optimal response lengths satisfy the water\-filling ruleLj∗=\[1βj​log⁡\(1\+βjθ\)\]\+L^\{\*\}\_\{j\}=\\big\[\\frac\{1\}\{\\beta\_\{j\}\}\\log\\big\(1\+\\frac\{\\beta\_\{j\}\}\{\\theta\}\\big\)\\big\]\_\{\+\}, whereθ≥0\\theta\\geq 0is the Lagrange multiplier associated with the effective budget constraint and is chosen so that∑j∈𝒜LLMLj∗=B\\sum\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}L\_\{j\}^\{\*\}=B\. The optimal solutionL∗L^\{\*\}is unique\.

###### Proof 3\.1\.

The constraints in \([7](https://arxiv.org/html/2605.23929#S3.E7)\)imply that the feasible set of \([7](https://arxiv.org/html/2605.23929#S3.E7)\) is equivalent to\{Lj≥0:∑jLj≤B\}\\\{L\_\{j\}\\geq 0:\\sum\_\{j\}L\_\{j\}\\leq B\\\}\. Then, we apply the Karush\-Kuhn\-Tucker \(KKT\) conditions by introducing a multiplierθ≥0\\theta\\geq 0for∑jLj≤B\\sum\_\{j\}L\_\{j\}\\leq Band multipliersνj≥0\\nu\_\{j\}\\geq 0for nonnegativity constraints\. The Lagrangian isℒ​\(L,θ,ν\)=∑jlog⁡\(1−e−βj​Lj\)−θ​\(∑jLj−BL\)\+∑jνj​Lj\\mathcal\{L\}\(L,\\theta,\\nu\)=\\sum\_\{j\}\\log\(1\-e^\{\-\\beta\_\{j\}L\_\{j\}\}\)\-\\theta\(\\sum\_\{j\}L\_\{j\}\-B\_\{L\}\)\+\\sum\_\{j\}\\nu\_\{j\}L\_\{j\}\. The KKT stationarity condition gives∂ℒ/∂Lj=\(βj​e−βj​Lj\)/\(1−e−βj​Lj\)−θ\+νj=0\\partial\\mathcal\{L\}/\\partial L\_\{j\}=\(\\beta\_\{j\}e^\{\-\\beta\_\{j\}L\_\{j\}\}\)/\(1\-e^\{\-\\beta\_\{j\}L\_\{j\}\}\)\-\\theta\+\\nu\_\{j\}=0\. Consider an agent withLj\>0L\_\{j\}\>0\. Complementary slackness impliesνj=0\\nu\_\{j\}=0, yieldingβj​e−βj​Lj/\(1−e−βj​Lj\)=θ\\beta\_\{j\}e^\{\-\\beta\_\{j\}L\_\{j\}\}/\(1\-e^\{\-\\beta\_\{j\}L\_\{j\}\}\)=\\theta\. Rearranging this equation givesLj=1βj​log⁡\(1\+βj/θ\)L\_\{j\}=\\frac\{1\}\{\\beta\_\{j\}\}\\log\(1\+\\beta\_\{j\}/\\theta\)\. If this value is negative, the nonnegativity constraint becomes active andLj∗=0L\_\{j\}^\{\*\}=0\. Combining these two cases yields the water\-filling rule\. As the objective function is strictly concave and the feasible set is convex, KKT conditions are necessary and sufficient for optimality\. The solution is unique\.

###### Corollary 3\.3\(Optimal workflow reliability\)\.

LetL∗L^\{\*\}be the optimal token allocation characterized in Proposition[1](https://arxiv.org/html/2605.23929#Thmprop1)\. The corresponding optimal workflow reliability is thusRwf∗=∏j∈𝒜LLMρj​\(Xj,Lj∗\)=∏j∈𝒜LLM\(1−e−αj​Xj\)​βjβj\+θR\_\{\\text\{wf\}\}^\{\*\}=\\prod\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\rho\_\{j\}\(X\_\{j\},L\_\{j\}^\{\*\}\)=\\prod\_\{j\\in\\mathcal\{A\}\_\{\\text\{LLM\}\}\}\\big\(1\-e^\{\-\\alpha\_\{j\}X\_\{j\}\}\\big\)\\frac\{\\beta\_\{j\}\}\{\\beta\_\{j\}\+\\theta\}, whereθ\\thetais the effective shadow token price\.

###### Proof 3\.4\.

The proof follows the model in \([2](https://arxiv.org/html/2605.23929#S2.E2)\) and the optimal allocation in Proposition[1](https://arxiv.org/html/2605.23929#Thmprop1)that givese−βj​Lj∗=θ/\(βj\+θ\)e^\{\-\\beta\_\{j\}L\_\{j\}^\{\*\}\}=\\theta/\(\\beta\_\{j\}\+\\theta\)\.

## 4Numerical Experiments

![Refer to caption](https://arxiv.org/html/2605.23929v1/allocation.png)Figure 1:Output token allocations under different strategies\.![Refer to caption](https://arxiv.org/html/2605.23929v1/reliability.png)Figure 2:Workflow reliability under different token budgets\.We illustrate the water\-filling allocation policy of Proposition[1](https://arxiv.org/html/2605.23929#Thmprop1)through a numerical example\. Consider a sequential workflow comprisingn=5n=5LLM agents with heterogeneous reliability parametersβj∈\{0\.001,0\.002,0\.0005,0\.003,0\.0015\}\\beta\_\{j\}\\in\\\{0\.001,0\.002,0\.0005,0\.003,0\.0015\\\}\. The total output token budget isB=12000B=12000tokens\. Figure[1](https://arxiv.org/html/2605.23929#S4.F1)compares the optimal water\-filling allocation against three baseline strategies: uniform \(equal tokens to all agents\), proportional \(tokens∝βj\\propto\\beta\_\{j\}\), and inverse proportional \(tokens∝1/βj\\propto 1/\\beta\_\{j\}\)\. The water\-filling rule allocates more tokens to agents with lower values ofβj\\beta\_\{j\}\. This result reflects that agents with higherβj\\beta\_\{j\}quickly reach high reliability with fewer tokens, while agents with lowerβj\\beta\_\{j\}require more tokens to achieve comparable levels\. Figure[2](https://arxiv.org/html/2605.23929#S4.F2)shows the reliability of the workflow under different token budgets and strategies; the optimal allocation rule consistently dominates all baseline strategies\.

## 5Conclusion

This work proposed a framework for analyzing and optimizing LLM\-enabled agentic workflows under latency and cost constraints\. We introduced performance models quantifying trade\-offs among reliability, latency, and cost, and derived closed\-form optimal token allocation policies for sequential workflows\. The optimal allocation equalizes marginal gains in log reliability across agents, with the allocation determined by a shadow price that reflects the scarcity of the token budget\. Future work includes joint optimization of reasoning and response length, as well as adaptive allocation policies that dynamically adjust token budgets based on intermediate results\.

## References

- \[1\]D\. B\. Acharya, K\. Kuppan, and B\. Divya\(2025\)Agentic ai: autonomous intelligence for complex goals—a comprehensive survey\.IEEE Access13,pp\. 18912–18936\.Cited by:[§1](https://arxiv.org/html/2605.23929#S1.p1.1)\.
- \[2\]S\. Boyd and L\. Vandenberghe\(2004\)Convex optimization\.Cambridge university press\.Cited by:[§1](https://arxiv.org/html/2605.23929#S1.p2.1)\.
- \[3\]J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. Amodei\(2020\)Scaling laws for neural language models\.arXiv preprint arXiv:2001\.08361\.Cited by:[Remark 1](https://arxiv.org/html/2605.23929#Thmremark1.p1.10.10)\.
- \[4\]W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. Gonzalez, H\. Zhang, and I\. Stoica\(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the 29th symposium on operating systems principles,pp\. 611–626\.Cited by:[§2\.1](https://arxiv.org/html/2605.23929#S2.SS1.p2.8)\.
- \[5\]X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou\(2022\)Self\-consistency improves chain of thought reasoning in language models\.arXiv preprint arXiv:2203\.11171\.Cited by:[§2\.1](https://arxiv.org/html/2605.23929#S2.SS1.p3.5)\.
- \[6\]Y\. Yang and Q\. Zhu\(2026\)Internet of agentic ai: incentive\-compatible distributed teaming and workflow\.arXiv preprint arXiv:2602\.03145\.Cited by:[§1](https://arxiv.org/html/2605.23929#S1.p1.1)\.