Cost-Optimal Foundation Model Deployment Portfolio for Transportation Management

# Cost-Optimal Foundation Model Deployment Portfolio for Transportation Management
Source: [https://arxiv.org/html/2607.13239](https://arxiv.org/html/2607.13239)
Xi Cheng1,∗, Ke Liu2,∗, Siyuan Feng3, Jane Lin4, H\. Oliver Gao1∗Equal contribution\. Contact: xc557@cornell\.edu\.1Cornell University, Ithaca, NY, USA\.2University of California, Berkeley, CA, USA\.3The Hong Kong Polytechnic University, Hong Kong SAR\.4University of Illinois Chicago, Chicago, IL, USA\.

###### Abstract

Foundation models, including large language models \(LLMs\) and vision\-language models \(VLMs\), are increasingly used for transportation management center \(TMC\) tasks such as anomaly detection, incident reporting, and traveler information\. Deploying multiple such models across TMC functions raises a portfolio question: which model should serve each function, in which deployment mode, and under what shared hardware budget? We formulate this as the Foundation Model Deployment Portfolio \(FMDP\) problem, a mixed\-integer program minimizing total cost of ownership \(TCO\) subject to per\-function quality, latency, and safety constraints over shared GPU capacity\. We prove the problem NP\-hard by reduction from the 0\-1 knapsack problem and propose a polynomial\-time greedy heuristic\. In an illustrative case study with five TMC functions and 19 candidate \(model, mode\) pairs, FMDP identifies a mixed portfolio costing $34/mo \(97% below the cheapest feasible all\-closed\-API baseline\) by routing four functions to open\-source APIs and the one function whose quality floor no open\-source model meets to a closed API\. Break\-even analysis shows that on\-premise GPU investment becomes reasonable only above approximately 309 vision queries/hour or if API prices double\.

## IIntroduction

Transportation management centers \(TMCs\) operate continuously, monitoring hundreds of CCTV feeds, detecting and verifying incidents, coordinating response with police and fire agencies, adjusting signal timing and ramp metering, and pushing traveler information to dynamic message signs and 511 traveler information services\[[11](https://arxiv.org/html/2607.13239#bib.bib2)\]\. A recent Federal Highway Administration \(FHWA\) briefing on next\-generation Traffic Management Systems recommends that agencies “assess the balance between on\-premises and cloud\-enabled capabilities”, weighing AI/ML computing needs against the cost of GPUs\[[21](https://arxiv.org/html/2607.13239#bib.bib1)\]\. To date, we are aware of no formal framework for this assessment\.

Many TMC functions already rely on AI models\. Classical detectors like YOLO\[[23](https://arxiv.org/html/2607.13239#bib.bib40)\]handle vehicle counting and basic anomaly detections, but foundation models \(FMs\) can support broader TMC tasks: VLMs interpret camera scenes for rare\-event detection\[[40](https://arxiv.org/html/2607.13239#bib.bib3)\], LLMs predict traffic states from heterogeneous sensor data\[[17](https://arxiv.org/html/2607.13239#bib.bib4)\], draft incident reports\[[6](https://arxiv.org/html/2607.13239#bib.bib5)\], and advise on signal timing\[[26](https://arxiv.org/html/2607.13239#bib.bib6)\]\. Several recent surveys document these applications\[[38](https://arxiv.org/html/2607.13239#bib.bib7),[42](https://arxiv.org/html/2607.13239#bib.bib8),[10](https://arxiv.org/html/2607.13239#bib.bib9)\]\.

As capable models become widely available, a practical planning question emerges:*which model should serve each function, and how should it be deployed?*Agencies can now choose closed\-source cloud APIs \(e\.g\., GPT\-4o at $2\.50/M tokens\[[30](https://arxiv.org/html/2607.13239#bib.bib10)\], Gemini Flash at $0\.075/M\[[13](https://arxiv.org/html/2607.13239#bib.bib11)\]\), open\-source models on hosted inference platforms \(e\.g\., Llama\-3\.1\-8B\[[14](https://arxiv.org/html/2607.13239#bib.bib12)\]at $0\.18/M via Together\.ai\[[37](https://arxiv.org/html/2607.13239#bib.bib42)\]\), or on\-premise \(self\-hosted\) GPU\. These options differ substantially in cost, latency, quality, and data sovereignty\. Moreover, the decisions are coupled: when multiple functions share a limited on\-premise GPU budget, loading a large vision model for detection can consume available memory to force the report generator onto a more expensive cloud API\.

Existing work on LLM cost optimization operates at the per\-query level\. FrugalGPT\[[4](https://arxiv.org/html/2607.13239#bib.bib13)\]cascades models within a single task; RouteLLM\[[29](https://arxiv.org/html/2607.13239#bib.bib14)\]and ThriftLLM\[[19](https://arxiv.org/html/2607.13239#bib.bib15)\]route individual queries to cheaper alternatives; and INFaaS\[[33](https://arxiv.org/html/2607.13239#bib.bib16)\]automates model\-variant selection at runtime\. None of these methods address the multi\-function setting in which functions compete for shared infrastructure, and none guarantee the operational requirements a TMC imposes\. A low\-cost model is not useful if it misreads a sensor or produces an incoherent incident report, so deployment must respect a minimum*quality*\. It must also meet*latency*targets; FHWA, for instance, names rapid incident detection a primary Traffic Incident Management performance goal\[[21](https://arxiv.org/html/2607.13239#bib.bib1)\]\. And for safety\-critical functions such as wrong\-way\-driver detection, a*safety*bound must hold the false\-negative rate \(FNR\) below an agency\-specified threshold\. To our knowledge, FM deployment for TMC operations has not been formulated as a portfolio optimization problem under constraints of the quality, latency, safety, and costs\. Our contributions are threefold\.

1. 1\.We formulate the Foundation Model Deployment Portfolio \(FMDP\) problem as a constrained mixed\-integer program that minimizes monthly total cost of ownership \(TCO\) across functions sharing GPU capacity, subject to quality, latency, and safety constraints\.
2. 2\.We prove the FMDP is NP\-hard via reduction from the 0\-1 knapsack problem and propose a greedy heuristic\.
3. 3\.We present an illustrative TMC case study showing 97% cost reduction over the cheapest feasible all\-closed\-API baseline and derive per\-function API vs\. on\-premise break\-even formulas\.

## IIRelated Work

Foundation models in transportation\.Several surveys discuss FM applications in intelligent transportation: Wandelt et al\.\[[38](https://arxiv.org/html/2607.13239#bib.bib7)\]review LLMs across ITS tasks; Zhang et al\.\[[42](https://arxiv.org/html/2607.13239#bib.bib8)\]focus on LLMs for mobility forecasting; Fan et al\.\[[10](https://arxiv.org/html/2607.13239#bib.bib9)\]address multimodal perception and decision\-making on complex roads\. VLMs have been applied to video anomaly detection\[[40](https://arxiv.org/html/2607.13239#bib.bib3),[39](https://arxiv.org/html/2607.13239#bib.bib17)\], LLMs to traffic state prediction\[[17](https://arxiv.org/html/2607.13239#bib.bib4)\]and signal control\[[26](https://arxiv.org/html/2607.13239#bib.bib6)\], and LLM\-based agents to incident analysis\[[6](https://arxiv.org/html/2607.13239#bib.bib5)\]\.

Cost\-efficient model selection and routing\.FrugalGPT\[[4](https://arxiv.org/html/2607.13239#bib.bib13)\]pioneered LLM cascading, reporting up to 98% cost reduction by querying models from cheapest to most expensive until a quality bar is met\. ThriftLLM\[[19](https://arxiv.org/html/2607.13239#bib.bib15)\]formulates ensemble selection with approximation guarantees; RouteLLM\[[29](https://arxiv.org/html/2607.13239#bib.bib14)\]trains preference\-based routers; Hybrid LLM\[[9](https://arxiv.org/html/2607.13239#bib.bib18)\]routes between cloud and on\-device models; Dekoninck et al\.\[[7](https://arxiv.org/html/2607.13239#bib.bib19)\]derive provably optimal routing; and FORC\[[35](https://arxiv.org/html/2607.13239#bib.bib20)\]applies integer programming to per\-task assignment\. All target a single function and do not model shared GPU capacity\.

Deployment economics and security\.Zhang et al\.\[[41](https://arxiv.org/html/2607.13239#bib.bib21)\]analyze cloud vs on\-premise deployment as a pricing game between LLM providers, finding that privacy concerns shape equilibrium strategies\. Huang et al\.\[[18](https://arxiv.org/html/2607.13239#bib.bib22)\]propose SOLID, a semi\-open on\-premise framework securing selected model layers against distillation while preserving fine\-tuning flexibility\. Both study a single model’s deployment choice\.

Resource allocation and model serving\.From an operations research point of view, FMDP combines a multiple\-choice assignment structure with knapsack\-type GPU\-capacity constraints and a fixed\-charge model\-sharing structure, which relate FMDP to the Generalized Assignment Problem\[[12](https://arxiv.org/html/2607.13239#bib.bib23),[32](https://arxiv.org/html/2607.13239#bib.bib24)\]\. INFaaS\[[33](https://arxiv.org/html/2607.13239#bib.bib16)\]automates the selection of models in heterogeneous hardware, and Mélange\[[15](https://arxiv.org/html/2607.13239#bib.bib25)\]formulates the composition of the GPU fleet as a bin packing\. These systems operate at per\-query or serving\-system granularity, whereas FMDP addresses deployment planning across functions\.

## IIIProblem Formulation

### III\-APreliminaries

Consider a TMC operatingFFfunctionsℱ=\{1,…,F\}\\mathcal\{F\}=\\\{1,\\ldots,F\\\}over a planning horizon ofHHhours \(e\.g\.,H=720H=720for one month\)\. Each functionffprocesses queries at rateλf\\lambda\_\{f\}\(queries/hour\) and must satisfy: \(i\) a minimum quality scoreqfmin∈\[0,1\]q\_\{f\}^\{\\min\}\\in\[0,1\]; \(ii\) a maximum per\-query latencyLfL\_\{f\}\(seconds\); and \(iii\) for safety\-critical functionsf∈ℱs⊆ℱf\\in\\mathcal\{F\}\_\{s\}\\subseteq\\mathcal\{F\}, a domain\-specific safety constraintsf​m≤Sfmaxs\_\{fm\}\\leq S\_\{f\}^\{\\max\}\. Whatsf​ms\_\{fm\}measures depends on the task\. For video anomaly detection, a natural choice is FNR\[[40](https://arxiv.org/html/2607.13239#bib.bib3)\]; for text generation, it could be a hallucination rate\[[20](https://arxiv.org/html/2607.13239#bib.bib26)\]\. The formulation can use any task\-specific safety metric that is measured at the model level and constrained by an upper bound\. Table[I](https://arxiv.org/html/2607.13239#S3.T1)summarizes the notations\.

TABLE I:Table of NotationsLetℳ\\mathcal\{M\}be the set of candidate models and𝒟\\mathcal\{D\}the set of deployment modes\. We distinguish four modes by how inference is accessed and where it runs \(Table[II](https://arxiv.org/html/2607.13239#S3.T2)\)\. Pricing varies enormously\[[31](https://arxiv.org/html/2607.13239#bib.bib27),[16](https://arxiv.org/html/2607.13239#bib.bib28),[22](https://arxiv.org/html/2607.13239#bib.bib29),[8](https://arxiv.org/html/2607.13239#bib.bib30),[36](https://arxiv.org/html/2607.13239#bib.bib31)\]: among closed\-source APIs alone, GPT\-4o charges $2\.50/M input tokens while Gemini\-2\.5\-Flash charges $0\.075/M, a 33×\\timesgap\. Open\-source hosted APIs are cheaper and allow fine\-tuning\. Cloud GPU rental charges by the hour regardless of utilization; the agency can load any open\-weight model using an inference engine \( e\.g\., vLLM\[[25](https://arxiv.org/html/2607.13239#bib.bib32)\]or SGLang\[[44](https://arxiv.org/html/2607.13239#bib.bib33)\]\)\. On\-premise GPUs incur amortized purchase and energy costs but provide full data control\.

TABLE II:Deployment Modes and PricingModeModel / PlatformPricingTrade\-offClosed APIGPT\-4o\[[30](https://arxiv.org/html/2607.13239#bib.bib10)\]$2\.50/$10\.00Highest qualityClaude Haiku 4\.5\[[1](https://arxiv.org/html/2607.13239#bib.bib34)\]$1\.00/$5\.00Mid\-tier qualityGPT\-4o\-mini\[[30](https://arxiv.org/html/2607.13239#bib.bib10)\]$0\.15/$0\.60Budget tierGemini\-2\.5\-Flash\[[13](https://arxiv.org/html/2607.13239#bib.bib11)\]$0\.075/$0\.30Lowest closedOpen APILlama\-3\.1\-8B\[[14](https://arxiv.org/html/2607.13239#bib.bib12)\]$0\.18/$0\.18Fine\-tunableQwen2\.5\-VL\-7B\[[3](https://arxiv.org/html/2607.13239#bib.bib35)\]$0\.20/$0\.20VisionCloud GPURTX 4090\[[34](https://arxiv.org/html/2607.13239#bib.bib36)\]$0\.35/hrFull controlA100 80GB\[[34](https://arxiv.org/html/2607.13239#bib.bib36)\]$1\.29/hrFull controlOn\-premiseRTX 4090\[[28](https://arxiv.org/html/2607.13239#bib.bib37)\]$44/mo\+energySovereigntyA100 80GB\[[27](https://arxiv.org/html/2607.13239#bib.bib38)\]$417/mo\+energySovereigntyAPI: input/output per M tokens \(Open\-source models hosted inference APIs at representative prices\)\. On\-premise: MSRP amortized over 36 months\. Cloud GPU: on\-demand hourly rate\. All prices retrieved Jan\. 2026; subject to change\.To compute the cost model, we reduce the four modes into two cost families:*per\-query variable cost*\(both API types\) and*fixed infrastructure plus variable energy*\(cloud rental and on\-premise\)\. We write𝒟=\{closed api,open api,gpu\-rent,gpu\-own\}\\mathcal\{D\}=\\\{\\textsf\{closed api\},\\textsf\{open api\},\\textsf\{gpu\-rent\},\\textsf\{gpu\-own\}\\\}\. Let𝒜f⊆ℳ×𝒟\\mathcal\{A\}\_\{f\}\\subseteq\\mathcal\{M\}\\times\\mathcal\{D\}denote the feasible candidate set for functionff\. Each pair\(m,d\)\(m,d\)is characterized by qualityqf​mq\_\{fm\}, safety metricsf​ms\_\{fm\}, latencyℓf​m​d\\ell\_\{fmd\}\(forf∈ℱsf\\in\\mathcal\{F\}\_\{s\}\), and memorymemm\\mathrm\{mem\}\_\{m\}\. For GPU modes, the required device count isnm​d=⌈memm/capd⌉n\_\{md\}=\\lceil\\mathrm\{mem\}\_\{m\}/\\mathrm\{cap\}\_\{d\}\\rceil\.

### III\-BCost Model

Monthly cost consists of three components: variable API charges, fixed hardware costs, and, for owned hardware, variable energy cost\.

API cost\(d=\(closed/open\) apid=\\textsf\{\(closed/open\) api\}\)\. Monthly cost is proportional to query volume and token length:

Cf​mapi=λf⋅H⋅\(τfin⋅pmin\+τfout⋅pmout\)C^\{\\text\{api\}\}\_\{fm\}=\\lambda\_\{f\}\\cdot H\\cdot\\bigl\(\\tau^\{\\textsf\{in\}\}\_\{f\}\\cdot p^\{\\textsf\{in\}\}\_\{m\}\+\\tau^\{\\textsf\{out\}\}\_\{f\}\\cdot p^\{\\textsf\{out\}\}\_\{m\}\\bigr\)\(1\)whereτfin,τfout\\tau^\{\\textsf\{in\}\}\_\{f\},\\tau^\{\\textsf\{out\}\}\_\{f\}are average input/output tokens per query andpmin,pmoutp^\{\\textsf\{in\}\}\_\{m\},p^\{\\textsf\{out\}\}\_\{m\}are the model provider’s per\-token prices\.

GPU hardware cost \(shared\)\.Ford∈\{gpu\-rent,gpu\-own\}d\\in\\\{\\textsf\{gpu\-rent\},\\textsf\{gpu\-own\}\\\}, hosting modelmmrequiresnm​dn\_\{md\}devices\. Monthly infrastructure cost reads:

Cm​dhw=\{Rd⋅nm​d⋅Hd=gpu\-rentPd⋅nm​d/Tdd=gpu\-ownC^\{\\text\{hw\}\}\_\{md\}=\\begin\{cases\}R\_\{d\}\\cdot n\_\{md\}\\cdot H&d=\\textsf\{gpu\-rent\}\\\\\[4\.0pt\] P\_\{d\}\\cdot n\_\{md\}\\,/\\,T\_\{d\}&d=\\textsf\{gpu\-own\}\\end\{cases\}\(2\)Importantly,Cm​dhwC^\{\\text\{hw\}\}\_\{md\}is charged once per model loaded\.

Energy cost\(d=gpu\-ownd=\\textsf\{gpu\-own\}only\)\. Operational energy per function is given by:

Cf​m​denergy=TDPd⋅nm​d⋅μf​m​d⋅H⋅peC^\{\\text\{energy\}\}\_\{fmd\}=\\text\{TDP\}\_\{d\}\\cdot n\_\{md\}\\cdot\\mu\_\{fmd\}\\cdot H\\cdot p\_\{e\}\(3\)whereμf​m​d=min⁡\(λf​ℓf​m​d/3600,1\)\\mu\_\{fmd\}=\\min\(\\lambda\_\{f\}\\ell\_\{fmd\}/3600,1\)is the fraction of time the GPU actively processes queries for functionff, assuming sequential inference\.

### III\-CFMDP Formulation

We formulate FMDP as a mixed\-integer program\. Binary variablexf​m​dx\_\{fmd\}assigns functionffto model–deployment pair\(m,d\)\(m,d\), andym​dy\_\{md\}indicates whether modelmmis loaded on GPU infrastructuredd\.

minx,y\\displaystyle\\min\_\{x,y\}\\;\\;∑f∑\(m,d\)∈𝒜fd=\(closed/open\) apiCf​mapi​xf​m​d⏟API token cost\+∑m∑d≠apiCm​dhw​ym​d⏟fixed hardware cost\\displaystyle\\underbrace\{\\sum\_\{f\}\\sum\_\{\\begin\{subarray\}\{c\}\(m,d\)\\in\\mathcal\{A\}\_\{f\}\\\\ d=\\textsf\{\(closed/open\) api\}\\end\{subarray\}\}\\\!\\\!\\\!C^\{\\text\{api\}\}\_\{fm\}\\,x\_\{fmd\}\}\_\{\\text\{API token cost\}\}\\;\+\\;\\underbrace\{\\sum\_\{m\}\\sum\_\{d\\neq\\textsf\{api\}\}\\\!\\\!\\\!C^\{\\text\{hw\}\}\_\{md\}\\,y\_\{md\}\}\_\{\\text\{fixed hardware cost\}\}\+∑f∑\(m,d\)∈𝒜fd=gpu\-ownCf​m​denergy​xf​m​d⏟variable energy cost\\displaystyle\\;\+\\;\\underbrace\{\\sum\_\{f\}\\sum\_\{\\begin\{subarray\}\{c\}\(m,d\)\\in\\mathcal\{A\}\_\{f\}\\\\ d=\\textsf\{gpu\-own\}\\end\{subarray\}\}\\\!\\\!\\\!C^\{\\text\{energy\}\}\_\{fmd\}\\,x\_\{fmd\}\}\_\{\\text\{variable energy cost\}\}\(4\)s\.t\.∑\(m,d\)∈𝒜fxf​m​d=1∀f\\displaystyle\\textstyle\\sum\_\{\(m,d\)\\in\\mathcal\{A\}\_\{f\}\}x\_\{fmd\}=1\\quad\\forall\\,f\(5\)∑\(m,d\)∈𝒜fqf​m⋅xf​m​d≥qfmin∀f\\displaystyle\\textstyle\\sum\_\{\(m,d\)\\in\\mathcal\{A\}\_\{f\}\}q\_\{fm\}\\cdot x\_\{fmd\}\\geq q\_\{f\}^\{\\min\}\\quad\\forall\\,f\(6\)∑\(m,d\)∈𝒜fsf​m⋅xf​m​d≤Sfmax∀f∈ℱs\\displaystyle\\textstyle\\sum\_\{\(m,d\)\\in\\mathcal\{A\}\_\{f\}\}s\_\{fm\}\\cdot x\_\{fmd\}\\leq S\_\{f\}^\{\\max\}\\quad\\forall\\,f\\\!\\in\\\!\\mathcal\{F\}\_\{s\}\(7\)∑\(m,d\)∈𝒜fℓf​m​d⋅xf​m​d≤Lf∀f\\displaystyle\\textstyle\\sum\_\{\(m,d\)\\in\\mathcal\{A\}\_\{f\}\}\\ell\_\{fmd\}\\cdot x\_\{fmd\}\\leq L\_\{f\}\\quad\\forall\\,f\(8\)ym​d≥xf​m​d∀f,\(m,d\),d≠api\\displaystyle y\_\{md\}\\geq x\_\{fmd\}\\quad\\forall\\,f,\\;\(m,d\),\\;d\\\!\\neq\\\!\\textsf\{api\}\(9\)∑mnm​d⋅ym​d≤Gd∀d≠api\\displaystyle\\textstyle\\sum\_\{m\}n\_\{md\}\\cdot y\_\{md\}\\leq G\_\{d\}\\quad\\forall\\,d\\\!\\neq\\\!\\textsf\{api\}\(10\)xf​m​d,ym​d∈\{0,1\}\\displaystyle x\_\{fmd\},\\;y\_\{md\}\\in\\\{0,1\\\}\(11\)
Constraint \([5](https://arxiv.org/html/2607.13239#S3.E5)\) assigns exactly one model–mode pair per function\. Constraints \([6](https://arxiv.org/html/2607.13239#S3.E6)\)–\([8](https://arxiv.org/html/2607.13239#S3.E8)\) enforce quality floors, safety bounds and latency ceilings\. Since exactly onexf​m​d=1x\_\{fmd\}=1per function, these reduce to checking whether the selected pair meets each threshold; infeasible pairs can be pre\-filtered from𝒜f\\mathcal\{A\}\_\{f\}\. We keep the linear\-sum form because it produces tighter LP relaxation bounds \(Section[IV](https://arxiv.org/html/2607.13239#S4)\)\. Constraint \([9](https://arxiv.org/html/2607.13239#S3.E9)\) links the two variable families: using modelmmon a GPU for any function forcesym​d=1y\_\{md\}=1\. Constraint \([10](https://arxiv.org/html/2607.13239#S3.E10)\) ensures that GPU usage in each mode does not exceedGdG\_\{d\}\.

## IVComplexity and Solution Methods

### IV\-ANP\-Hardness

###### Proposition 1\.

The FMDP is NP\-hard\.

###### Proof\.

We reduce from the 0–1 Knapsack Problem \(KP\)\[[24](https://arxiv.org/html/2607.13239#bib.bib44)\], in which itemsj=1,…,Jj=1,\\ldots,Jwith weightswjw\_\{j\}, valuesvjv\_\{j\}, and a capacityBBare given\. Construct an FMDP instance with one functionfjf\_\{j\}per item and budgetGgpu\-own=BG\_\{\\textsf\{gpu\-own\}\}=B; the mode set𝒟\\mathcal\{D\}is unchanged, and modes other thangpu\-ownandopen apisimply have no feasible candidates\. Set the electricity pricepe=0p\_\{e\}=0\. Each functionfjf\_\{j\}has two feasible candidates\. The first is a distinct local modelmjm\_\{j\}withmemmj=wj⋅capgpu\-own\\mathrm\{mem\}\_\{m\_\{j\}\}=w\_\{j\}\\cdot\\mathrm\{cap\}\_\{\\textsf\{gpu\-own\}\}, so thatnmj,gpu\-own=wjn\_\{m\_\{j\},\\textsf\{gpu\-own\}\}=w\_\{j\}and, by Eq\. \([2](https://arxiv.org/html/2607.13239#S3.E2)\), its hardware cost isκ​wj\\kappa w\_\{j\}withκ=Pgpu\-own/Tgpu\-own\\kappa=P\_\{\\textsf\{gpu\-own\}\}/T\_\{\\textsf\{gpu\-own\}\}\. The second is a dedicated API modelaja\_\{j\}whose per\-token price is chosen through Eq\. \([1](https://arxiv.org/html/2607.13239#S3.E1)\) so that its monthly cost equalsκ​wj\+vj\\kappa w\_\{j\}\+v\_\{j\}\. All quality, latency, and safety constraints are set non\-binding\. Because eachmjm\_\{j\}serves onlyfjf\_\{j\}, an assignment is feasible iff the GPU\-assigned subsetSSsatisfies∑j∈Swj≤B\\sum\_\{j\\in S\}w\_\{j\}\\leq B, and its total cost isκ​∑jwj\+∑j∉Svj\\kappa\\sum\_\{j\}w\_\{j\}\+\\sum\_\{j\\notin S\}v\_\{j\}\. Minimizing cost is therefore equivalent to maximizing∑j∈Svj\\sum\_\{j\\in S\}v\_\{j\}subject to the knapsack constraint, so solving FMDP optimally solves KP\. ∎

### IV\-BLP Relaxation Bound

Relaxing the binary variables to\[0,1\]\[0,1\]gives a polynomial\-time LP lower bound\. Without the GPU\-capacity constraint \([10](https://arxiv.org/html/2607.13239#S3.E10)\) and the linking constraints \([9](https://arxiv.org/html/2607.13239#S3.E9)\), the problem decomposes into independent per\-function selections whose LP relaxation is integral\. With shared GPU capacity and fixed\-charge loading variables, tightness is no longer guaranteed; we therefore use the LP optimum as a lower bound to certify heuristic solution quality\.

### IV\-CGreedy Heuristic

Algorithm[1](https://arxiv.org/html/2607.13239#alg1)is a two\-phase greedy heuristic cheap enough to re\-run whenever prices change\.Phase 1scans the functions in arbitrary order and assigns each to its cheapest feasible \(model, mode\) pair given the models already loaded; a model loaded once serves later functions at no extra hardware cost, matching the fixed\-charge structure of \([4](https://arxiv.org/html/2607.13239#S3.E4)\)\.Phase 2runs only if the GPU budgetGdG\_\{d\}is exceeded\. It first computes the cheapest feasible API fallback cost for each GPU\-assigned functionc^f\\hat\{c\}\_\{f\}, then repeatedly switches the function with the smallest repair ratiorf=Δf/gfr\_\{f\}=\\Delta\_\{f\}/g\_\{f\}to its fallback, whereΔf=c^f−costf​m​d\\Delta\_\{f\}=\\hat\{c\}\_\{f\}\-\\mathrm\{cost\}\_\{fmd\}is the resulting cost increase andgfg\_\{f\}is the number of GPU devices released\. A function sharing its model with others frees no devices, so we setgf=0g\_\{f\}=0andrf=\+∞r\_\{f\}=\+\\inftyrather than divide by zero\. If allrf=\+∞r\_\{f\}=\+\\infty, the heuristic switches the function with the smallestΔf\\Delta\_\{f\}, reducing the number of users of at least one loaded model and eventually releasing capacity\. The loop ends once the budget is met\. The Worst\-case complexity isO​\(F⋅\|𝒜\|max\+F2\)O\(F\\cdot\|\\mathcal\{A\}\|\_\{\\max\}\+F^\{2\}\): candidate lists are scanned once in Phase 1 and once forc^f\\hat\{c\}\_\{f\}, and Phase 2 performs repair iterations of at mostFF, up toFFfunctions\.

Algorithm 1Two\-Phase Greedy Heuristic for FMDP1:Input:functions

ℱ\\mathcal\{F\}, catalogs

\{𝒜f\}\\\{\\mathcal\{A\}\_\{f\}\\\}, GPU budget

GdG\_\{d\}
2:Output:assignment

σ\\sigma; loaded GPU models

𝒴⊆ℳ\\mathcal\{Y\}\\subseteq\\mathcal\{M\}, initially

∅\\emptyset
3:// Phase 1: minimum\-cost assignment

4:foreach

f∈ℱf\\in\\mathcal\{F\}do

5:

σ​\(f\)←arg⁡min\(m,d\)∈𝒜f⁡costf​m​d​\(𝒴\)\\sigma\(f\)\\leftarrow\\arg\\min\_\{\(m,d\)\\in\\mathcal\{A\}\_\{f\}\}\\mathrm\{cost\}\_\{fmd\}\(\\mathcal\{Y\}\);if

d=gpud=\\mathrm\{gpu\}then

𝒴←𝒴∪\{m\}\\mathcal\{Y\}\\leftarrow\\mathcal\{Y\}\\cup\\\{m\\\}
6:endfor

7:

Gused←∑m∈𝒴nm,gpuG\_\{\\text\{used\}\}\\leftarrow\\sum\_\{m\\in\\mathcal\{Y\}\}n\_\{m,\\text\{gpu\}\};if

Gused≤GdG\_\{\\text\{used\}\}\\leq G\_\{d\}then return

σ\\sigma
8:// Phase 2: capacity repair

9:

c^f←min\(m′,d′\)∈𝒜f,d′=api⁡costf​m′​d′\\hat\{c\}\_\{f\}\\leftarrow\\min\_\{\(m^\{\\prime\},d^\{\\prime\}\)\\in\\mathcal\{A\}\_\{f\},\\,d^\{\\prime\}=\\mathrm\{api\}\}\\mathrm\{cost\}\_\{fm^\{\\prime\}d^\{\\prime\}\}for each GPU\-assigned

ff
10:while

Gused\>GdG\_\{\\text\{used\}\}\>G\_\{d\}do

11:foreach

ffwith

σ​\(f\)=\(m,gpu\)\\sigma\(f\)=\(m,\\mathrm\{gpu\}\)do

12:

Δf←c^f−costf​m​d\\Delta\_\{f\}\\leftarrow\\hat\{c\}\_\{f\}\-\\mathrm\{cost\}\_\{fmd\};

gf←nm,gpu⋅𝟏​\[f​sole user of​m\]g\_\{f\}\\leftarrow n\_\{m,\\mathrm\{gpu\}\}\\cdot\\mathbf\{1\}\[f\\text\{ sole user of \}m\]
13:

rf←Δf/gfr\_\{f\}\\leftarrow\\Delta\_\{f\}/g\_\{f\}if

gf\>0g\_\{f\}\>0else

\+∞\+\\infty
14:endfor

15:Switch

f∗f^\{\*\}with smallest

rfr\_\{f\}\(smallest

Δf\\Delta\_\{f\}if all

rf=\+∞r\_\{f\}\{=\}\+\\infty\); update

σ\\sigma,

𝒴\\mathcal\{Y\},

GusedG\_\{\\text\{used\}\}
16:endwhile

17:return

σ\\sigma

## VIllustrative Case Study

We use an illustrative TMC case study to demonstrate the FMDP\. Scenario details are demonstrated as follows\. In practice, agencies would replace the illustrative entries in Table[IV](https://arxiv.org/html/2607.13239#S5.T4)with measurements from their own models and workloads\.

### V\-AScenario Design

We consider a TMC withℱ=5\\mathcal\{F\}\{=\}5functions and illustrative requirements \(Table[III](https://arxiv.org/html/2607.13239#S5.T3)\), reflecting the operational scope described in the NextGen TMS briefing\[[21](https://arxiv.org/html/2607.13239#bib.bib1)\]: video anomaly detection \(f1\), traffic state estimation \(f2\), incident report generation \(f3\), signal timing advisory \(f4\), and traveler information \(f5\)\. Function f1 is safety\-critical: it monitors CCTV feeds for wrong\-way drivers, pedestrians on freeways, and road safety, and we assume a FNR bound ofsf​1≤0\.10s\_\{f1\}\\leq 0\.10\. Query rates range widely, from 200 queries/hour for f1 \(one frame every 18 s\) down to just 3 incidents/hour for f3\.

TABLE III:Illustrative TMC FunctionsWe assemble a set of candidates per function from the deployment modes in Table[II](https://arxiv.org/html/2607.13239#S3.T2); Table[IV](https://arxiv.org/html/2607.13239#S5.T4)lists the full catalog\. Quality and FNR scores are illustrative but align with published performance tiers: frontier closed\-source VLMs outperform open\-source VLMs, which in turn outperform classical detectors\[[40](https://arxiv.org/html/2607.13239#bib.bib3),[39](https://arxiv.org/html/2607.13239#bib.bib17),[5](https://arxiv.org/html/2607.13239#bib.bib39),[23](https://arxiv.org/html/2607.13239#bib.bib40)\]; text\-function quality \(f2–f5\) follows the same ordering, consistent with LLM\-as\-judge evaluations\[[43](https://arxiv.org/html/2607.13239#bib.bib41)\]and public leaderboards \(e\.g\.,\[[2](https://arxiv.org/html/2607.13239#bib.bib43)\]\)\. We assume that the relative performance of these models on TMC tasks mirrors their ranking on public benchmarks\.

API costs are computed from Eq\. \([1](https://arxiv.org/html/2607.13239#S3.E1)\)\. Because providers use different tokenizers, the same prompt results in different token counts and different per\-query costscqc\_\{q\}\. For vision queries, GPT\-4o encodes a 720p CCTV frame as∼\{\\sim\}1,105 image tokens\[[30](https://arxiv.org/html/2607.13239#bib.bib10)\]; Qwen2\.5\-VL tiles the same frame into∼\{\\sim\}1,000 tokens\[[3](https://arxiv.org/html/2607.13239#bib.bib35)\]\. On\-premise costs follow Eqs\. \([2](https://arxiv.org/html/2607.13239#S3.E2)\)–\([3](https://arxiv.org/html/2607.13239#S3.E3)\)\. We assume two agency\-owned RTX 4090 GPUs \($1,600 MSRP\[[28](https://arxiv.org/html/2607.13239#bib.bib37)\], 36\-month amortization, 450 W TDP, $0\.12/kWh\), a workstation\-class configuration without data\-center infrastructure, for 48 GB total on\-premise VRAM\.

TABLE IV:Model Catalog and Monthly TCOModelDeploymentqqFNR$/monthf1 \(200 q/h\)GPT\-4o \(vision\)Closed API\.85\.041,102Qwen2\.5\-VL\-7B \(vision\)Open API\.78\.0829InternVL2\-8B \(vision\)On\-Premise\(16G\)\.78\.0846YOLOv8\-L \(vision\)On\-Premise\(<\\\!<\\\!1G\)\.72\.11<<1f2 \(120 q/h\)GPT\-4o\-miniClosed API\.82–9Gemini\-2\.5\-FlashClosed API\.79–3Llama\-3\.1\-8BOpen API\.77–2Qwen2\.5\-7BOn\-Premise\(15G\)\.77–45f3 \(3 q/h\)GPT\-4oClosed API\.90–9Claude Haiku 4\.5Closed API\.84–4GPT\-4o\-miniClosed API\.82–1Llama\-3\.1\-8BOpen API\.78–<<1Rule\-based–\.60–0f4 \(60 q/h\)GPT\-4o\-miniClosed API\.80–6Gemini\-2\.5\-FlashClosed API\.77–2Qwen2\.5\-7BOpen API\.75–1f5 \(10 q/h\)GPT\-4oClosed API\.88–40GPT\-4o\-miniClosed API\.80–2Llama\-3\.1\-8BOpen API\.76–1qq/FNR: Illustrative values informed by published performance tiers \(\[[40](https://arxiv.org/html/2607.13239#bib.bib3),[43](https://arxiv.org/html/2607.13239#bib.bib41)\]\)\. API TCO==λf⋅H⋅cq\\lambda\_\{f\}\\\!\\cdot\\\!H\\\!\\cdot\\\!c\_\{q\}per Eq\. \([1](https://arxiv.org/html/2607.13239#S3.E1)\) with official pricing\[[30](https://arxiv.org/html/2607.13239#bib.bib10),[1](https://arxiv.org/html/2607.13239#bib.bib34),[13](https://arxiv.org/html/2607.13239#bib.bib11)\]\(Table[II](https://arxiv.org/html/2607.13239#S3.T2)\); On\-Premise from Eqs\. \([2](https://arxiv.org/html/2607.13239#S3.E2)\)–\([3](https://arxiv.org/html/2607.13239#S3.E3)\)\.
### V\-BResults and Analysis

We implement Algorithm[1](https://arxiv.org/html/2607.13239#alg1)in Python on an Apple MacBook Pro \(M4, 24 GB RAM\)\. Experiment 1 presents the FMDP portfolio, while Experiment 2 studies API on\-premise break\-even points\.

Experiment 1: Optimal portfolio vs\. baselines\.We evaluate five single\-tier baselines \(Table[V](https://arxiv.org/html/2607.13239#S5.T5)\), each forcing every function into one deployment mode:*All\-Closed\-Frontier*picks the highest\-quality closed model per function;*All\-Closed\-Budget*picks the cheapest feasible closed model;*All\-Open\-Source\-API*uses hosted open\-source models throughout;*All\-On\-Premise*runs everything locally; and*Classical\-Only*relies on rule\-based or classical\-ML methods\.

FMDP assigns f1, f2, f4, and f5 to hosted open\-source APIs and routes only f3 to GPT\-4o\-mini, because no open\-source option meets its quality floor\. The resulting portfolio costs $34/mo with zero GPU usage\. At current open\-source API prices \($0\.20/M tokens for vision\), f1’s 144,000 monthly queries cost even less via API than on\-premise hardware \($29 vs\. $46/mo\)\.

Three baselines are infeasible\. All\-On\-Premise requires 63 GB of GPU memory, exceeding the 48 GB budget\. Classical\-Only and All\-Open\-Source\-API fail the quality threshold for f3 \(q=\.60q\{=\}\.60and\.78\.78vs\.qmin=\.80q^\{\\min\}\{=\}\.80\)\. Thus, the all\-open\-source portfolio nearly matches FMDP in cost but does not satisfy all constraints\. FMDP is feasible by assigning f3 to GPT\-4o\-mini at a marginal cost of $1/mo, while keeping the other functions on open\-source APIs\. The two feasible baselines are closed\-source and exceed $1,100/mo, driven by f1’s sustained vision queries via GPT\-4o\. FMDP’s $34/mo portfolio is thus, 97% below the cheapest feasible baseline\.

TABLE V:Deployment Strategy ComparisonStrategyTCOQ¯\\bar\{Q\}GPUFeasible?\($/mo\)\(GB\)All\-Closed\-Frontier1,166\.850✓All\-Closed\-Budget1,110\.810✓All\-Open\-Source\-API34\.770×\\timesAll\-On\-Premise226\.7763×\\timesClassical\-Only<<1\.630×\\timesFMDP Optimal34\.780✓Frontier: highest\-qqclosed model per function\. Budget: cheapest feasible closed model\. Open\-Source: Qwen2\.5\-VL \(vision\), Llama \(text\); failsqminq^\{\\min\}on f3\. On\-Premise: needs 63 GB\>\>48 GB budget\. Classical: failsqminq^\{\\min\}on f3\.Q¯=1\|ℱ\|​∑fqf​m\\bar\{Q\}\{=\}\\tfrac\{1\}\{\|\\mathcal\{F\}\|\}\\sum\_\{f\}q\_\{fm\}\.Experiment 2: API vs\. on\-premise break\-even points\.On\-premise deployment becomes cheaper than a cloud API once the query rate exceeds

λ∗=Cm​dhwH⋅cq\\lambda^\{\*\}\\;=\\;\\frac\{C^\{\\text\{hw\}\}\_\{md\}\}\{H\\cdot c\_\{q\}\}\(12\)wherecqc\_\{q\}is the per\-query API cost andCm​dhwC^\{\\text\{hw\}\}\_\{md\}is the monthly hardware amortization\. Table[VI](https://arxiv.org/html/2607.13239#S5.T6)evaluates this for a single RTX 4090 at $44/mo\.

At $0\.20/M tokens, the open\-source vision API \(Qwen2\.5\-VL\) breaks even at 309 q/h, above f1’s 200 q/h, so FMDP keeps f1 on the API\. Adding cameras or raising the frame rate beyond one frame every 12 s would cross this threshold and shift f1 on\-premise\. GPT\-4o, on the contrary, breaks even at only 8 q/h due to its higher per\-query cost \($0\.0077\): for any sustained vision workload beyond that rate, owning a GPU beats paying closed\-API vision prices\. Text workloads are far below their break\-even points, 617 q/h for GPT\-4o\-mini and 2,684 q/h for Llama\-3\.1\-8B\. Therefore, no text function in this scenario justifies local hardware\. These thresholds move with API pricing, which changes frequently: doubling the vision price to $0\.40/M halvesλ∗\\lambda^\{\*\}to 154 q/h, below f1’s rate, and the optimal assignment shifts on\-premise\.

TABLE VI:API vs\. On\-Premise Break\-Even PointsCloud APIcqc\_\{q\}\($/q\)λ∗\\lambda^\{\*\}\(q/h\)ImplicationGPT\-4o \(vision\)0\.00778on\-premQwen2\.5\-VL \(vision\)0\.00020308\.6use APIGPT\-4o\-mini \(text\)0\.00010617\.3use APILlama\-3\.1\-8B \(text\)0\.0000232,683\.8use APIλ∗\\lambda^\{\*\}: query rate above which on\-premise is cheaper, per Eq\. \([12](https://arxiv.org/html/2607.13239#S5.E12)\) withChw=$​44C^\{\\text\{hw\}\}\{=\}\\mathdollar 44/mo \(one RTX 4090\)\. “On\-prem”:λ∗<λf​1=200\\lambda^\{\*\}<\\lambda\_\{f1\}\{=\}200\. “Use API”:λ∗\>maxf⁡λf\\lambda^\{\*\}\>\\max\_\{f\}\\lambda\_\{f\}\.

## VIDiscussion

Although the case study is small, it illustrates why FM deployment for TMCs should be treated as a portfolio problem rather than a set of independent model choices\. No single deployment tier can be both feasible and cost\-minimal\. The all\-open\-source API strategy nearly matches FMDP’s monthly cost but fails the quality requirement for incident report generation, while the cheapest feasible all\-closed\-source strategy exceeds $1,100/mo because sustained vision API calls dominate cost\. FMDP identifies the narrow point where closed\-source capability is needed and keeps the remaining functions on lower\-cost hosted open\-source APIs\. Break\-even analysis shows that deployment decisions are sensitive to query volume and per\-query price\. At January 2026 prices, the open\-source vision API remains cheaper than local deployment at 200 q/h, but this would reverse if camera sampling rates, the number of streams, or API prices increase\. Text functions have much higher break\-even rates, thus on\-premise GPU deployment is not cost effective for the small volume reporting and advisory workloads in our case study\.

## VIIConclusion

This paper introduced FMDP, a mixed\-integer programming framework to jointly select foundation models and deployment modes across TMC functions subject to the constraints of quality, latency, safety, and shared GPU\-capacity\. We proved FMDP is NP\-hard and proposed a two\-phase greedy heuristic\. In an illustrative five\-function case study, FMDP achieves a $34/mo portfolio, 97% below the cheapest feasible all\-closed\-source baseline, by using hosted open\-source APIs where possible and a closed\-source API only where needed\. The break\-even analysis further identifies when on\-premise GPU deployment becomes cost\-effective\.

Limitations and future work\.The case study demonstrates the planning methodology rather than realistic operational performance: its quality, latency, and safety parameters are illustrative and should be replaced with agency\-specific measurements prior to operational use\. The current formulation assigns one model to each function and does not consider model cascading, price and demand uncertainty, or inter\-function dependencies\. Future work will extend FMDP to model cascading, multi\-period planning, and validation with operational TMC data\.

## References

- \[1\]\(2025\)API pricing\.Note:[https://claude\.com/pricing\#api](https://claude.com/pricing#api)Accessed: Jan\. 2026Cited by:[TABLE II](https://arxiv.org/html/2607.13239#S3.T2.5.3.3.1),[TABLE IV](https://arxiv.org/html/2607.13239#S5.T4.7.7.3.3.3.2)\.
- \[2\]Artificial Analysis\(2025\)LLM leaderboard\.Note:[https://artificialanalysis\.ai/leaderboards/models](https://artificialanalysis.ai/leaderboards/models)Accessed: Jan\. 2026Cited by:[§V\-A](https://arxiv.org/html/2607.13239#S5.SS1.p2.1)\.
- \[3\]S\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge, S\. Song, K\. Dang, P\. Wang, S\. Wang, J\. Tang, H\. Zhong, Y\. Zhu, M\. Yang, Z\. Li, J\. Wan, P\. Wang, W\. Ding, Z\. Fu, Y\. Xu, J\. Ye, X\. Zhang, T\. Xie, Z\. Cheng, H\. Zhang, Z\. Yang, H\. Xu, and J\. Lin\(2025\)Qwen2\.5\-vl technical report\.External Links:2502\.13923,[Link](https://arxiv.org/abs/2502.13923)Cited by:[TABLE II](https://arxiv.org/html/2607.13239#S3.T2.5.7.7.1),[§V\-A](https://arxiv.org/html/2607.13239#S5.SS1.p3.3)\.
- \[4\]L\. Chen, M\. Zaharia, and J\. Zou\(2023\)Frugalgpt: how to use large language models while reducing cost and improving performance\.arXiv preprint arXiv:2305\.05176\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p4.1),[§II](https://arxiv.org/html/2607.13239#S2.p2.1)\.
- \[5\]Z\. Chen, J\. Wu, W\. Wang, W\. Su, G\. Chen, S\. Xing, M\. Zhong, Q\. Zhang, X\. Zhu, L\. Lu,et al\.\(2024\)Internvl: scaling up vision foundation models and aligning for generic visual\-linguistic tasks\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 24185–24198\.Cited by:[§V\-A](https://arxiv.org/html/2607.13239#S5.SS1.p2.1)\.
- \[6\]L\. Da, K\. Liou, T\. Chen, X\. Zhou, X\. Luo, Y\. Yang, and H\. Wei\(2024\)Open\-ti: open traffic intelligence with augmented language model\.International Journal of Machine Learning and Cybernetics15\(10\),pp\. 4761–4786\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p2.1),[§II](https://arxiv.org/html/2607.13239#S2.p1.1)\.
- \[7\]J\. Dekoninck, M\. Baader, and M\. Vechev\(2025\)A unified approach to routing and cascading for llms\.InICML 2025,Cited by:[§II](https://arxiv.org/html/2607.13239#S2.p2.1)\.
- \[8\]DetectX\(2025\)Cost comparison: API vs self\-hosting for open\-weight LLMs\.Note:DetectX BlogAccessed: Jan\. 2026External Links:[Link](https://www.detectx.com.au/cost-comparison-api-vs-self-hosting-for-open-weight-llms/)Cited by:[§III\-A](https://arxiv.org/html/2607.13239#S3.SS1.p2.3)\.
- \[9\]D\. Ding, A\. Mallick, C\. Wang, R\. Sim, S\. Mukherjee, V\. Rühle, L\. V\. Lakshmanan, and A\. H\. Awadallah\(2024\)Hybrid llm: cost\-efficient and quality\-aware query routing\.InThe Twelfth International Conference on Learning Representations,Cited by:[§II](https://arxiv.org/html/2607.13239#S2.p2.1)\.
- \[10\]L\. Fan, Y\. Wang, H\. Zhang, C\. Zeng, Y\. Li, C\. Gou, and H\. Yu\(2024\)Multimodal perception and decision\-making systems for complex roads based on foundation models\.IEEE Transactions on Systems, Man, and Cybernetics: Systems54\(11\),pp\. 6561–6569\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p2.1),[§II](https://arxiv.org/html/2607.13239#S2.p1.1)\.
- \[11\]Federal Highway Administration\(2024\)Next generation of traffic management systems and centers: a primer\.Technical reportU\.S\. Department of Transportation\.Note:[https://highways\.dot\.gov/media/49321](https://highways.dot.gov/media/49321)Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p1.1)\.
- \[12\]M\. L\. Fisher, R\. Jaikumar, and L\. N\. Van Wassenhove\(1986\)A multiplier adjustment method for the generalized assignment problem\.Management science32\(9\),pp\. 1095–1103\.Cited by:[§II](https://arxiv.org/html/2607.13239#S2.p4.1)\.
- \[13\]Google\(2025\)Gemini API pricing\.Note:[https://ai\.google\.dev/pricing](https://ai.google.dev/pricing)Accessed: Jan\. 2026Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p3.1),[TABLE II](https://arxiv.org/html/2607.13239#S3.T2.5.5.5.1),[TABLE IV](https://arxiv.org/html/2607.13239#S5.T4.7.7.3.3.3.2)\.
- \[14\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p3.1),[TABLE II](https://arxiv.org/html/2607.13239#S3.T2.5.6.6.2)\.
- \[15\]T\. Griggs, X\. Liu, J\. Yu, D\. Kim, W\. Chiang, A\. Cheung, and I\. Stoica\(2024\)M\\\\backslash’elange: cost efficient large language model serving by exploiting gpu heterogeneity\.arXiv preprint arXiv:2404\.14527\.Cited by:[§II](https://arxiv.org/html/2607.13239#S2.p4.1)\.
- \[16\]O\. Gubanova\(2025\)LLM total cost of ownership 2025: build vs\. buy math\.Note:Ptolemay BlogAccessed: Jan\. 2026External Links:[Link](https://www.ptolemay.com/post/llm-total-cost-of-ownership)Cited by:[§III\-A](https://arxiv.org/html/2607.13239#S3.SS1.p2.3)\.
- \[17\]X\. Guo, Q\. Zhang, J\. Jiang, M\. Peng, M\. Zhu, and H\. F\. Yang\(2024\)Towards explainable traffic flow prediction with large language models\.Communications in Transportation Research4,pp\. 100150\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p2.1),[§II](https://arxiv.org/html/2607.13239#S2.p1.1)\.
- \[18\]H\. Huang, Y\. Li, B\. Jiang, B\. Jiang, L\. Liu, Z\. Liu, R\. Sun, and S\. Liang\(2025\)A middle path for on\-premises llm deployment: preserving privacy without sacrificing model confidentiality\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 8332–8370\.Cited by:[§II](https://arxiv.org/html/2607.13239#S2.p3.1)\.
- \[19\]K\. Huang, Y\. Shi, D\. Ding, Y\. Li, Y\. Fei, L\. Lakshmanan, and X\. Xiao\(2025\)ThriftLLM: on cost\-effective selection of large language models for classification queries\.Proceedings of the VLDB Endowment18\(11\),pp\. 4410–4423\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p4.1),[§II](https://arxiv.org/html/2607.13239#S2.p2.1)\.
- \[20\]L\. Huang, W\. Yu, W\. Ma, W\. Zhong, Z\. Feng, H\. Wang, Q\. Chen, W\. Peng, X\. Feng, B\. Qin,et al\.\(2025\)A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions\.ACM Transactions on Information Systems43\(2\),pp\. 1–55\.Cited by:[§III\-A](https://arxiv.org/html/2607.13239#S3.SS1.p1.11)\.
- \[21\]ITS Joint Program Office\(2025\)Next generation transportation management centers and transportation management systems: executive briefing\.Technical reportU\.S\. Department of Transportation\.Note:[https://www\.itskrs\.its\.dot\.gov/briefings/executive\-briefing/next\-generation\-transportation\-management\-centers\-and\-transportation](https://www.itskrs.its.dot.gov/briefings/executive-briefing/next-generation-transportation-management-centers-and-transportation)Updated January 2026Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p1.1),[§I](https://arxiv.org/html/2607.13239#S1.p4.1),[§V\-A](https://arxiv.org/html/2607.13239#S5.SS1.p1.2)\.
- \[22\]S\. Ivanovici\(2025\)Cost of serving LLMs\.Note:Fin\.ai Research BlogAccessed: Jan\. 2026External Links:[Link](https://fin.ai/research/cost-of-serving-llms/)Cited by:[§III\-A](https://arxiv.org/html/2607.13239#S3.SS1.p2.3)\.
- \[23\]G\. Jocher, A\. Chaurasia, and J\. Qiu\(2023\)Ultralytics YOLOv8\.Note:[https://github\.com/ultralytics/ultralytics](https://github.com/ultralytics/ultralytics)Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p2.1),[§V\-A](https://arxiv.org/html/2607.13239#S5.SS1.p2.1)\.
- \[24\]R\. M\. Karp\(2009\)Reducibility among combinatorial problems\.In50 Years of Integer Programming 1958\-2008: from the Early Years to the State\-of\-the\-Art,pp\. 219–241\.Cited by:[§IV\-A](https://arxiv.org/html/2607.13239#S4.SS1.1.p1.24)\.
- \[25\]W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. Gonzalez, H\. Zhang, and I\. Stoica\(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the 29th symposium on operating systems principles,pp\. 611–626\.Cited by:[§III\-A](https://arxiv.org/html/2607.13239#S3.SS1.p2.3)\.
- \[26\]S\. Lai, Z\. Xu, W\. Zhang, H\. Liu, and H\. Xiong\(2025\)LLMLight: large language models as traffic signal control agents\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 1,pp\. 2335–2346\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p2.1),[§II](https://arxiv.org/html/2607.13239#S2.p1.1)\.
- \[27\]NVIDIA\(2020\)NVIDIA A100 tensor core GPU\.Note:[https://www\.nvidia\.com/en\-us/data\-center/a100/](https://www.nvidia.com/en-us/data-center/a100/)80 GB PCIe; market price approximately $15,000 \(2025\)Cited by:[TABLE II](https://arxiv.org/html/2607.13239#S3.T2.5.11.11.1)\.
- \[28\]NVIDIA\(2022\)GeForce RTX 4090\.Note:[https://www\.nvidia\.com/en\-us/geforce/graphics\-cards/40\-series/rtx\-4090/](https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/)MSRP $1,599 at launch \(Oct\. 2022\)Cited by:[TABLE II](https://arxiv.org/html/2607.13239#S3.T2.5.10.10.2),[§V\-A](https://arxiv.org/html/2607.13239#S5.SS1.p3.3)\.
- \[29\]I\. Ong, A\. Almahairi, V\. Wu, W\. Chiang, T\. Wu, J\. E\. Gonzalez, M\. W\. Kadous, and I\. Stoica\(2025\)RouteLLM: learning to route llms from preference data\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p4.1),[§II](https://arxiv.org/html/2607.13239#S2.p2.1)\.
- \[30\]OpenAI\(2025\)API pricing\.Note:[https://openai\.com/api/pricing](https://openai.com/api/pricing)Accessed: Jan\. 2026Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p3.1),[TABLE II](https://arxiv.org/html/2607.13239#S3.T2.5.2.2.2),[TABLE II](https://arxiv.org/html/2607.13239#S3.T2.5.4.4.1),[§V\-A](https://arxiv.org/html/2607.13239#S5.SS1.p3.3),[TABLE IV](https://arxiv.org/html/2607.13239#S5.T4.7.7.3.3.3.2)\.
- \[31\]G\. Pan, V\. Chodnekar, A\. Roy, and H\. Wang\(2025\)A cost\-benefit analysis of on\-premise large language model deployment: breaking even with commercial llm services\.arXiv preprint arXiv:2509\.18101\.Cited by:[§III\-A](https://arxiv.org/html/2607.13239#S3.SS1.p2.3)\.
- \[32\]D\. W\. Pentico\(2007\)Assignment problems: a golden anniversary survey\.European Journal of Operational Research176\(2\),pp\. 774–793\.Cited by:[§II](https://arxiv.org/html/2607.13239#S2.p4.1)\.
- \[33\]F\. Romero, Q\. Li, N\. J\. Yadwadkar, and C\. Kozyrakis\(2021\)\{\\\{infaas\}\\\}: Automated model\-less inference serving\.In2021 USENIX Annual Technical Conference \(USENIX ATC 21\),pp\. 397–411\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p4.1),[§II](https://arxiv.org/html/2607.13239#S2.p4.1)\.
- \[34\]RunPod\(2025\)GPU cloud pricing\.Note:[https://www\.runpod\.io/pricing](https://www.runpod.io/pricing)Accessed: Jan\. 2026Cited by:[TABLE II](https://arxiv.org/html/2607.13239#S3.T2.5.8.8.2),[TABLE II](https://arxiv.org/html/2607.13239#S3.T2.5.9.9.1)\.
- \[35\]M\. Šakota, M\. Peyrard, and R\. West\(2024\)Fly\-swat or cannon? cost\-effective language model choice via meta\-modeling\.InProceedings of the 17th ACM International Conference on Web Search and Data Mining,pp\. 606–615\.Cited by:[§II](https://arxiv.org/html/2607.13239#S2.p2.1)\.
- \[36\]T\. Sharma\(2025\)The complete AI strategy guide: cloud APIs vs self\-hosted models\.Note:MediumAccessed: Jan\. 2026External Links:[Link](https://medium.com/@tuhinsharma121/the-complete-ai-strategy-guide-cloud-apis-vs-self-hosted-models-a68b9bb69778)Cited by:[§III\-A](https://arxiv.org/html/2607.13239#S3.SS1.p2.3)\.
- \[37\]Together AI\(2025\)Inference pricing\.Note:[https://www\.together\.ai/pricing](https://www.together.ai/pricing)Accessed: Jan\. 2026Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p3.1)\.
- \[38\]S\. Wandelt, C\. Zheng, S\. Wang, Y\. Liu, and X\. Sun\(2024\)Large language models for intelligent transportation: a review of the state of the art and challenges\.Applied Sciences14\(17\),pp\. 7455\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p2.1),[§II](https://arxiv.org/html/2607.13239#S2.p1.1)\.
- \[39\]Y\. Yao, X\. Wang, M\. Xu, Z\. Pu, Y\. Wang, E\. Atkins, and D\. J\. Crandall\(2022\)DoTA: unsupervised detection of traffic anomaly in driving videos\.IEEE transactions on pattern analysis and machine intelligence45\(1\),pp\. 444–459\.Cited by:[§II](https://arxiv.org/html/2607.13239#S2.p1.1),[§V\-A](https://arxiv.org/html/2607.13239#S5.SS1.p2.1)\.
- \[40\]L\. Zanella, W\. Menapace, M\. Mancini, Y\. Wang, and E\. Ricci\(2024\)Harnessing large language models for training\-free video anomaly detection\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 18527–18536\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p2.1),[§II](https://arxiv.org/html/2607.13239#S2.p1.1),[§III\-A](https://arxiv.org/html/2607.13239#S3.SS1.p1.11),[§V\-A](https://arxiv.org/html/2607.13239#S5.SS1.p2.1),[TABLE IV](https://arxiv.org/html/2607.13239#S5.T4.7.7.3.3.3.2)\.
- \[41\]Z\. J\. Zhang, J\. Shi, and S\. Tang\(2025\)Cloud or on\-premise? a strategic view of large language model deployment\.A Strategic View of Large Language Model Deployment \(June 16, 2025\)\.Cited by:[§II](https://arxiv.org/html/2607.13239#S2.p3.1)\.
- \[42\]Z\. Zhang, Y\. Sun, Z\. Wang, Y\. Nie, X\. Ma, R\. Li, P\. Sun, and X\. Ban\(2026\)Large language models for mobility analysis in transportation systems: a survey on forecasting tasks\.Transportation Research Record2680\(2\),pp\. 756–774\.Cited by:[§I](https://arxiv.org/html/2607.13239#S1.p2.1),[§II](https://arxiv.org/html/2607.13239#S2.p1.1)\.
- \[43\]L\. Zheng, W\. Chiang, Y\. Sheng,et al\.\(2023\)Judging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\.InNeurIPS,Cited by:[§V\-A](https://arxiv.org/html/2607.13239#S5.SS1.p2.1),[TABLE IV](https://arxiv.org/html/2607.13239#S5.T4.7.7.3.3.3.2)\.
- \[44\]L\. Zheng, L\. Yin, Z\. Xie, C\. L\. Sun, J\. Huang, C\. H\. Yu, S\. Cao, C\. Kozyrakis, I\. Stoica, J\. E\. Gonzalez,et al\.\(2024\)Sglang: efficient execution of structured language model programs\.Advances in neural information processing systems37,pp\. 62557–62583\.Cited by:[§III\-A](https://arxiv.org/html/2607.13239#S3.SS1.p2.3)\.