EXHOLD: Experience-Aware Real-Time Hold Control for Large-Scale Ride-Hailing Matching at DiDi

# EXHOLD: Experience-Aware Real-Time Hold Control for Large-Scale Ride-Hailing Matching at DiDi
Source: [https://arxiv.org/html/2607.09090](https://arxiv.org/html/2607.09090)
###### Abstract\.

In large\-scale ride\-hailing matching systems,*hold control*is a high\-leverage mechanism for improving end\-to\-end passenger–driver experience: by selectively delaying certain driver–order pairs, the system can wait for better opportunities, reduce cancellations and excessive waiting, and mitigate wasted driver effort, ultimately increasing trip success\. In practice, many industrial hold strategies rely on heuristic thresholding over multiple predictive models, which can be brittle under non\-stationary traffic conditions and hard to optimize for multiple, experience\-oriented objectives\.

We proposeEXHOLD, a deployable two\-stage framework that decouples*experience\-aware pair assessment*from*hold\-time execution*\. In Stage I, we learn a decision model that assigns each driver–order pair to discrete and interpretable*experience tiers*, optimizing a unified objective that aggregates multiple satisfaction\-related signals across the matching funnel\. In Stage II, we solve for a monotone hold\-time schedule via constrained optimization over empirical quantiles, explicitly enforcing service guardrails that bound unnecessary holding of promising matches while maximizing overall experience improvement\.

We evaluateEXHOLDthrough online randomized A/B experiments in DiDi’s production ride\-hailing matching system in Brazil\. The results demonstrate consistent gains in both marketplace efficiency and passenger–driver experience:EXHOLDincreases trip completion and driver income, while significantly reducing passenger cancellations before and after acceptance and improving key funnel efficiency signals such as faster call\-to\-acceptance\. We further conduct targeted ablations and behavioral drill\-down analyses, showing that both stages ofEXHOLDare essential to the observed gains, and that the policy makes calibrated decisions under spatiotemporal heterogeneity\.EXHOLDhas been ramped up and is currently serving the Brazil market in production\.

Ride\-Hailing; Decision Making; Representation Learning; Contextual Bandits; Constrained Optimization

††Accepted at the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining \(KDD 2026\)\. This is an arXiv author version\.## 1\.Introduction

Large\-scale ride\-hailing platforms solve a continuous stream of matching decisions under strict latency, reliability, and service\-quality constraints\(Xu et al\.,[2018](https://arxiv.org/html/2607.09090#bib.bib29)\)\. For each incoming order, the system evaluates many candidate driver–order pairs \(DOpair\) and must decide not only*who*to match, but also*when*to match\(Wang et al\.,[2021](https://arxiv.org/html/2607.09090#bib.bib26); Zhang et al\.,[2017](https://arxiv.org/html/2607.09090#bib.bib34)\)\. A critical lever in this process is*hold control*: selectively holding certainDOpair candidates for some system\-decided time so the system can wait for better opportunities, reduce cancellations and excessive waiting, and mitigate wasted driver effort—ultimately improving end\-to\-end passenger–driver experience and trip success\(Tu,[2024](https://arxiv.org/html/2607.09090#bib.bib24); Afèche et al\.,[2023](https://arxiv.org/html/2607.09090#bib.bib2)\)\. Hold control is powerful but delicate: an overly aggressive policy may suppress good matches and harm service levels of the platform, while an overly conservative policy yields little impact\(Xu et al\.,[2018](https://arxiv.org/html/2607.09090#bib.bib29); Afèche et al\.,[2023](https://arxiv.org/html/2607.09090#bib.bib2)\)\.

##### Why hold control is challenging in production\.

Hold decisions are inherently multi\-objective, sequential, and tightly coupled with the matching pipeline\(Supian et al\.,[2024](https://arxiv.org/html/2607.09090#bib.bib19)\)\. First, experience degradation manifests through*heterogeneous funnel outcomes*: Passenger Cancellation Before driver Acceptance \(PCBA\), driver non\-response, Passenger Cancellation After driver Acceptance \(PCAA\), as well as Driver Cancellation After Acceptance \(DCAA\), have distinct user harm and efficiency implications\(Xu et al\.,[2018](https://arxiv.org/html/2607.09090#bib.bib29); Wang et al\.,[2021](https://arxiv.org/html/2607.09090#bib.bib26)\)\. Second, hold is a delayed\-action mechanism: holding a candidate changes future states such as passenger waiting, local supply–demand regimes, and the set of feasible matches the system sees later\(Afèche et al\.,[2023](https://arxiv.org/html/2607.09090#bib.bib2); Barbour and Luiz,[2019](https://arxiv.org/html/2607.09090#bib.bib3)\)\. Besides, production systems require explicit*service guardrails*: the system must bound unnecessary holding of promising matches and remain stable under non\-stationary traffic conditions\(Yan et al\.,[2020](https://arxiv.org/html/2607.09090#bib.bib30); Sühr et al\.,[2019](https://arxiv.org/html/2607.09090#bib.bib18)\)\.

##### Limitations of heuristic model\-threshold strategies\.

Many industrial hold modules are implemented as heuristic thresholding over multiple predictive models, for example mapping each driver\-order pair to a fixed or dynamic threshold and then comparing a predictor against that threshold to decide hold or release\(Chen et al\.,[2021](https://arxiv.org/html/2607.09090#bib.bib4); Wen et al\.,[2024](https://arxiv.org/html/2607.09090#bib.bib27); Yatnalkar,[2019](https://arxiv.org/html/2607.09090#bib.bib32)\)\. While effective as a baseline, this paradigm faces three recurring limitations in practice: \(i\)*Error compounding*arises when multiple models are chained and small calibration shifts amplify through the decision rule\(Somalwar et al\.,[2025](https://arxiv.org/html/2607.09090#bib.bib17)\); \(ii\)*Weak multi\-objective coordination*makes it difficult to simultaneously optimize multiple experience\-related signals across the funnel \(e\.g\.,PCBA,PCAA,DCAAand passenger over\-waiting\) with hand\-tuned threshold rules\(Naumov and Keith,[2023](https://arxiv.org/html/2607.09090#bib.bib15)\); \(iii\)*Limited temporal reasoning*means temporal effects are typically approximated by static calibrations, which can be brittle under traffic shifts\. As a result, these strategies can be hard to tune for experience improvements under spatiotemporal heterogeneity and may not provide explicit, auditable control over “how much holding is acceptable” under service constraints\(Lee et al\.,[2022](https://arxiv.org/html/2607.09090#bib.bib11); Tirachini,[2020](https://arxiv.org/html/2607.09090#bib.bib23); Yang et al\.,[2020](https://arxiv.org/html/2607.09090#bib.bib31)\)\.

##### Our approach: decouple experience\-aware assessment from hold\-time execution\.

We proposeEXperience\-awareHOLD\(EXHOLD\), a deployable two\-stage framework for hold control in industrial ride\-hailing matching systems \(Figure[2](https://arxiv.org/html/2607.09090#S3.F2)\)\. The key idea ofEXHOLDis to separate*what to hold*from*how long to hold*in a way that aligns with production constraints and experience\-oriented objectives:

- •Stage I \(Experience\-aware pair assessment\)\.We learn a decision model that assigns eachDOpair to a discrete, interpretable*experience tier*\. The tiering optimizes a unified objective that aggregates multiple passenger\-driver satisfaction\-related signals across the matching funnel \(order completion, cancellations at different stages, and waiting\-related indicators, shown in Figure[1](https://arxiv.org/html/2607.09090#S2.F1)\)\. To support robustness under non\-stationary traffic, Stage I introduces Transformer\-based temporal representations\(Vaswani et al\.,[2017](https://arxiv.org/html/2607.09090#bib.bib25)\)and LinUCB\(Chu et al\.,[2011](https://arxiv.org/html/2607.09090#bib.bib5)\)head that provides stable online decisions\.
- •Stage II \(Guardrail\-constrained hold\-time optimization\)\.Given the experience tiers from Stage I, we compute a monotone hold\-time schedule via constrained optimization over empirical quantiles\(Kotary et al\.,[2021](https://arxiv.org/html/2607.09090#bib.bib10); Liang et al\.,[2022](https://arxiv.org/html/2607.09090#bib.bib12)\)\. This stage explicitly enforces service guardrails that bound unnecessary holding of promising matches and provides an controllable tool for conservativeness, enabling safe deployment at production platforms\.

The decomposition design ofEXHOLDis not merely an engineering convenience, but also a principled response to the difficulty of optimizing hold control strategies end\-to\-end under heterogeneous feedback, strict operational constraints, and the need for explicit guardrails in production\(Feng et al\.,[2021](https://arxiv.org/html/2607.09090#bib.bib6); Liu et al\.,[2025](https://arxiv.org/html/2607.09090#bib.bib13)\)\.

##### Real\-world deployment and key findings\.

We deployEXHOLDin DiDi’s production ride\-hailing matching system in Brail and evaluate it through large\-scale A/B experiments\. The results demonstrate thatEXHOLDdelivers consistent improvements on trip success and driver welfare while reducing experience\-degrading cancellations and improving matching efficiency, indicating better end\-to\-end passenger–driver matching experience in real operations\. Beyond aggregate outcomes, we provide \(i\) funnel\-level analyses that attribute the gains to improved conversion after exposure and fewer cancellations, and \(ii\) ablations and behavioral drill\-downs that validate the role of multi\-signal experience modeling and guardrail\-constrained hold\-time optimization\. After the A/B validation,EXHOLDhas been ramped up and is now deployed at market scale, serving real traffic across the Brazil marketplace\.

##### Contributions\.

We summarize our main contributions as follows:

- •We formulate hold control in ride\-hailing matching as a decision problem centered on passenger–driver experience, with explicit service guardrails to limit unnecessary holding\.
- •We proposeEXHOLD, a practical two\-stage framework that decouples experience\-aware tier assignment from hold\-time execution, combining representation learning, contextual bandits, and constrained optimization over empirical quantiles\.
- •We develop an experience\-oriented objective and training pipeline that aggregates heterogeneous signals across the ride\-hailing funnel\. Online A/B experiments show improved trip success and driver welfare, reduced experience\-degrading cancellations, and faster funnel efficiency; online ablations further confirm that both Stage I and Stage II are essential for consistent gains\.

## 2\.Problem Formulation

![Refer to caption](https://arxiv.org/html/2607.09090v1/x1.png)Figure 1\.Schematic of heterogeneous experience feedback in ride\-hailing passenger\-driver matching\. At timesteptt, the system first identifies all available candidate drivers\. A hold\-decision module \(highlighted inblue\) then decides which drivers should be held for some time, while the remaining candidates are passed to the dispatch engine for broadcasting\. During the driver’s decision\-making and the subsequent pickup, various heterogeneous experience feedback may occur \(highlighted inyellow\)\.We formalize hold control in large\-scale ride\-hailing matching as a constrained decision problem with*experience\-oriented objectives*and*service guardrails*\. Our formulation highlights industrial aspects that materially affect both optimization and evaluation: \(i\) theDOpair decision unit and its position in the matching pipeline, \(ii\) heterogeneous funnel outcomes that correspond to distinct passenger/driver experience costs, \(iii\) delayed effects through time and passenger–driver interactions, and \(iv\) explicit guardrails that bound unnecessary holding of promising matches\.

### 2\.1\.System Context and Decision Unit

A production ride\-hailing platform continuously evaluates candidate driver–order pairs for each incoming order\. We define the atomic*decision unit*as a driver–order pair \(DOpair\), representing a specific order evaluated with a specific nearby driver at a particular time\. Hold control is applied to these candidates before they are executed by the downstream broadcast/assignment module, and thus can change both the set of candidates exposed to drivers and the composition of accepted and completed trips\.

Specifically, at decision steptt, the system observes a state vectorst∈𝒮s\_\{t\}\\in\\mathcal\{S\}for aDOpair that aggregates:

- •Order context:current passenger waiting time and its evolution, OD attributes, fare/distance proxies, and other order features;
- •Driver context:driver availability/status, recent activity patterns, historical response statistics, and other driver features;
- •Spatio\-temporal feature:time\-of\-day, local supply–demand conditions \(e\.g\., nearby available drivers and orders\), historical supply and demand, and other spatiotemporal features;
- •System feature:estimated pickup distance and time, estimated trip time, and other system\-augmented matching features\.

These signals jointly characterize both*immediate feasibility*\(e\.g\., pickup distance/time\) and*experience sensitivity*\(e\.g\., accumulated waiting and supply\-demand stress\)\.

### 2\.2\.Experience Tiers and Hold\-Time Execution

A hold decision determines whether aDOpair should be released immediately to the downstream pipeline or temporarily delayed\. Directly optimizing a continuous hold duration online is difficult in production due to delayed credit assignment, unsafe exploration, and tight latency budgets\. We therefore model the online decision as selecting a discrete*experience tier*:

\(1\)at∈𝒜=\{0,1,…,K\},a\_\{t\}\\in\\mathcal\{A\}=\\\{0,1,\\dots,K\\\},where largerata\_\{t\}indicates a less favorable pairing from an experience perspective and thus warrants stronger holding\. Each tier is subsequently mapped to a hold timexatx\_\{a\_\{t\}\}computed offline by a constrained optimization procedure \(Stage II, Section[3\.3](https://arxiv.org/html/2607.09090#S3.SS3)\)\. Online execution applies the delayed\-action policy:

\(2\)hold duration=xat,x0=0≤x1≤⋯≤xK\.\\text\{hold duration\}=x\_\{a\_\{t\}\},\\qquad x\_\{0\}=0\\leq x\_\{1\}\\leq\\cdots\\leq x\_\{K\}\.This design keeps online inference fast and bounded while making the timing component explicitly controllable via guardrails\.

### 2\.3\.Heterogeneous Experience Costs for Hold

After aDOpair is released \(immediately or after holding\), the system evolves according to passenger and driver behaviors and downstream matching\. Each decision leads to one of several mutually exclusive outcomes: CBA, driver non\-response, PCAA ir DCAA; these time\-delayed outcomes are shown in Figure[1](https://arxiv.org/html/2607.09090#S2.F1)\. These outcomes correspond to heterogeneous experience costs:PCBAis often tied to pre\-accept friction and excessive waiting,PCAAdirectly wastes driver effort and leads to passenger bad experience, andDCAAcan amplify inefficiency and subsequent cancellations\. Therefore, hold control must optimize*multi\-signal experience objectives*rather than a single proxy outcome\.

### 2\.4\.Hold Objective: Experience Utility Maximization Under Service Guardrails

Ideally, we seek a policyπ\\pithat mapsDOpair states to experience tiers to maximize expected experience utility while satisfying explicit service guardrails\. Letyt∈𝒴y\_\{t\}\\in\\mathcal\{Y\}denote the realized outcome category and letroracle​\(st,at,yt\)r\_\{\\text\{oracle\}\}\(s\_\{t\},a\_\{t\},y\_\{t\}\)be the oracle reward that aggregates multiple experience\-related outcomes, wherests\_\{t\}is the real\-timeDOpair state \(Section[2\.1](https://arxiv.org/html/2607.09090#S2.SS1)\),ata\_\{t\}is the real\-time hold action generated by the policy \(whether to hold and how long to hold\)\. Then the constrained hold objective can be expressed as:

\(3\)maxπ\\displaystyle\\max\_\{\\pi\}\\quad𝔼π​\[∑tγt​roracle​\(st,at,yt\)\],\\displaystyle\\mathbb\{E\}\_\{\\pi\}\\\!\\left\[\\sum\_\{t\}\\gamma^\{t\}\\,r\_\{\\text\{oracle\}\}\(s\_\{t\},a\_\{t\},y\_\{t\}\)\\right\],\(4\)s\.t\.ℙπ​\(unnecessary\-hold∣yt=TC\)≤α,\\displaystyle\\mathbb\{P\}\_\{\\pi\}\\\!\\left\(\\text\{unnecessary\-hold\}\\mid y\_\{t\}=\\textsc\{TC\}\\right\)\\leq\\alpha,whereγ∈\(0,1\]\\gamma\\in\(0,1\]is the discount factor andα\\alphais the tolerated rate of unnecessarily holding cases that would have been a complete trip\. Constraint \([4](https://arxiv.org/html/2607.09090#S2.E4)\) captures an essential production guardrail: improving experience by reducing cancellations and excessive waiting should not come at the cost of suppressing too many promising matches, which would degrade service levels and harm passenger\-driver experience\. In practice, however, the oracleroracler\_\{\\text\{oracle\}\}is typically unavailable at the decision time, as it depends on future outcomes\. This necessitates the design of an effective proxy reward to realize experience\-aware holding policies \(Section[3\.2\.4](https://arxiv.org/html/2607.09090#S3.SS2.SSS4)\)\.

## 3\.Method

![Refer to caption](https://arxiv.org/html/2607.09090v1/x2.png)Figure 2\.Overview ofEXHOLD\. Stage I produces discrete*experience tiers*for each driver–order pair from rich context; Stage II computes a monotone hold\-time table under explicit service guardrails via constrained optimization\.We presentEXperience\-awareHOLD\(EXHOLD\), a deployable two\-stage framework for experience\-aware real\-time hold control in large\-scale ride\-hailing matching\. The core design principle ofEXHOLDis to decouple*experience\-aware pair assessment*\(what to hold, Section[3\.2](https://arxiv.org/html/2607.09090#S3.SS2)\) from*hold\-time execution*\(how long to hold, Section[3\.3](https://arxiv.org/html/2607.09090#S3.SS3)\) under explicit service guardrails\.

### 3\.1\.Method Overview

At each decision steptt, the system evaluates a candidate driver–order pair \(DOpair\) with statests\_\{t\}\(Section[2\.1](https://arxiv.org/html/2607.09090#S2.SS1)\)\. Stage I assigns eachDOpair to a discrete and interpretable experience tierat∈𝒜=\{0,1,…,K\}a\_\{t\}\\in\\mathcal\{A\}=\\\{0,1,\\dots,K\\\}with Transformer and contextual bandits, where largerata\_\{t\}indicates lower expected experience \(e\.g\., higher passenger/driver cancellation propensity or worse waiting outcomes\)\. Stage II maps each tieriito a monotone hold\-time table\{xi\}i=0K\\\{x\_\{i\}\\\}\_\{i=0\}^\{K\}computed with constrained optimization\.

### 3\.2\.Stage I: Experience\-Aware Assessment

Stage I learns a policyπθ,ϕ\\pi\_\{\\theta,\\phi\}that assigns eachDOpair to an experience tier using \(i\) a Transformer encoder that summarizes short\-horizon temporal context and \(ii\) a stable contextual bandit head that enables robust online decisions with controlled exploration\. This hybrid design balances representation power, stability under distribution shift, and production constraints\.

#### 3\.2\.1\.Sequential Context Construction

Hold decisions depend on time\-evolving order state \(e\.g\., accumulated waiting\) and local regime changes \(e\.g\., supply–demand shifts\)\. For eachDOpair at timett, we construct a short history windowhDO,th\_\{\\text\{\\tiny\{DO\}\},t\}composed of:

- •Order trace:recent evolution of order\-level state over the pastLLsteps \(e\.g\., waiting progression and other state changes\);
- •Local regime trace:recent regime summaries \(e\.g\., spatiotemporal buckets capturing supply–demand conditions\);
- •Static attributes:time\-invariant or slowly varying attributes of the order, driver, and the DO pair\.

#### 3\.2\.2\.Transformer Encoder for Temporal Representation

We encodehth\_\{t\}using a Transformer encoder to obtain an embedding:

\(5\)zDO,t=fθ​\(hDO,t\)∈ℝp\.z\_\{\\text\{\\tiny\{DO\}\},t\}=f\_\{\\theta\}\(h\_\{\\text\{\\tiny\{DO\}\},t\}\)\\in\\mathbb\{R\}^\{p\}\.The embedding is trained to summarize both instantaneous pair feasibility and short\-horizon temporal patterns that correlate with experience degradation \(e\.g\., cancellations and waiting outcomes\)\.

Unlike purely supervised thresholding, Stage I learns representations that are directly aligned with downstream hold control by optimizing an experience\-oriented learning signal that aggregates multiple funnel outcomes\. Concretely, we attach a lightweight contextual bandit head on top ofzDO,tz\_\{\\text\{\\tiny\{DO\}\},t\}and train it with Huber loss in our implementation to predict a scalar utility reward; the exact construction of this reward is detailed in Section[3\.2\.4](https://arxiv.org/html/2607.09090#S3.SS2.SSS4), and the training details of the encoder are explained in Appendix[B](https://arxiv.org/html/2607.09090#A2)\.

#### 3\.2\.3\.Contextual Bandit Head for Experience Tier Decision

Given theDOpair embeddingzDO,tz\_\{\\text\{\\tiny\{DO\}\},t\}from the Transformer encoder at decision steptt, we deploy a LinUCB contextual bandit head to select an experience tier with controlled exploration\. For each actiona∈𝒜a\\in\\mathcal\{A\}, the contextual bandit maintains a ridge\-regression estimator and its design matrix computed from interaction data:

\(6\)Aa=λ0​I\+∑τ∈𝒯azDO,τ​zDO,τ⊤,ba=∑τ∈𝒯azDO,τ​rτ,A\_\{a\}=\\lambda\_\{0\}I\+\\sum\_\{\\tau\\in\\mathcal\{T\}\_\{a\}\}z\_\{\\text\{\\tiny\{DO\}\},\\tau\}\\,z\_\{\\text\{\\tiny\{DO\}\},\\tau\}^\{\\top\},\\quad b\_\{a\}=\\sum\_\{\\tau\\in\\mathcal\{T\}\_\{a\}\}z\_\{\\text\{\\tiny\{DO\}\},\\tau\}\\,r\_\{\\tau\},where𝒯a=\{τ<t:aτ=a\}\\mathcal\{T\}\_\{a\}=\\\{\\tau<t:a\_\{\\tau\}=a\\\}is the set of historical steps where tieraawas selected,λ0\>0\\lambda\_\{0\}\>0is the regularization coefficient, andrτr\_\{\\tau\}is the experience\-oriented reward defined in Section[3\.2\.4](https://arxiv.org/html/2607.09090#S3.SS2.SSS4), whererτr\_\{\\tau\}is the realized experience reward logged for stepτ\\tau\. At runtime, the action is selected by an upper\-confidence bound:

\(7\)at=arg⁡maxa∈𝒜⁡\(zDO,t⊤​Aa−1​ba\+λ​zDO,t⊤​Aa−1​zDO,t\),a\_\{t\}=\\arg\\max\_\{a\\in\\mathcal\{A\}\}\\left\(z\_\{\\text\{\\tiny\{DO\}\},t\}^\{\\top\}A\_\{a\}^\{\-1\}b\_\{a\}\+\\lambda\\sqrt\{z\_\{\\text\{\\tiny\{DO\}\},t\}^\{\\top\}A\_\{a\}^\{\-1\}z\_\{\\text\{\\tiny\{DO\}\},t\}\}\\right\),where the first term is the estimated expected utility and the second term is an uncertainty bonus;λ\\lambdacontrols the exploration bonus\.

This head provides two properties that are critical for deployment\. From the algorithm perspective, it implements an adaptive explore–exploit trade\-off: actions with high estimated utility are preferred while actions with high epistemic uncertainty receive a bonus that shrinks asAaA\_\{a\}accumulates data\. From our engineering perspective, it is lightweight and predictable in latency \(small action set and low\-dimensional matrix updates\), and it produces first\-class observability signals—per\-tier counts\|𝒯a\|\|\\mathcal\{T\}\_\{a\}\|, uncertainty magnitudeszt⊤​Aa−1​zt\\sqrt\{z\_\{t\}^\{\\top\}A\_\{a\}^\{\-1\}z\_\{t\}\}, and tier\-histogram drift—which enable rapid diagnosis and conservative rollback in deployment \(Section[4\.5](https://arxiv.org/html/2607.09090#S4.SS5)\)\.

#### 3\.2\.4\.Experience\-Oriented Reward Design

A central challenge in hold control is that our platform cares about overall passenger\-driver experience across the funnel rather than a single metric\. For each decision on aDOpair at steptt, the realized outcome may involve multiple stages \(DOpair broadcast→\\rightarrowdriver acceptance→\\rightarroworder completion\) and heterogeneous failure modes with different user impact\. Moreover, hold influences waiting\-related experience through delayed effects\. To train Stage I with a unified learning signal, we define an*experience\-oriented reward*rtr\_\{t\}that aggregates outcome utility from heterogeneous impacts and waiting\-related penalties while remaining stable for training and deployment\.

##### Outcome utility with heterogeneous funnel impacts\.

Letyt∈𝒴y\_\{t\}\\in\\mathcal\{Y\}denote the realized outcome category associated with theDOpair decision at steptt\(Section[2\.3](https://arxiv.org/html/2607.09090#S2.SS3)and Section[2\.4](https://arxiv.org/html/2607.09090#S2.SS4)\)\. We define a base utility that treats successful completion as unit gain and assigns asymmetric penalties to experience\-degrading outcomes \(Figure[1](https://arxiv.org/html/2607.09090#S2.F1)\):

\(8\)rbase​\(yt\)=\{\+1,yt=TC,−wb,yt=PCBA,−wp,yt=PCAA,−wd,yt=DCAA,r0,driver non\-acceptance,r\_\{\\text\{base\}\}\(y\_\{t\}\)=\\begin\{cases\}\+1,&y\_\{t\}=\\textsc\{TC\},\\\\ \-w\_\{b\},&y\_\{t\}=\\textsc\{PCBA\},\\\\ \-w\_\{p\},&y\_\{t\}=\\textsc\{PCAA\},\\\\ \-w\_\{d\},&y\_\{t\}=\\textsc\{DCAA\},\\\\ r\_\{0\},&\\text\{driver non\-acceptance\},\\end\{cases\}where we setwp≈wb\>wd≥0w\_\{p\}\\approx w\_\{b\}\>w\_\{d\}\\geq 0, and this hierarchical penalty structure is motivated by the fact that passenger\-side negative experiences are usually more difficult to model from our historical data\. Therefore, we impose higher penalty weights on passenger\-related disruptions to ensure the matching policy remains sensitive to these critical but hard\-to\-learn outcomes\. In practice, these weights are calibrated from offline statistics \(e\.g\., conditional rates and funnel shares\) and business impact considerations\.

##### Tier\-dependent cost\-effectiveness shaping\.

While Eq\. \([8](https://arxiv.org/html/2607.09090#S3.E8)\) provides a consistent utility scale, it does not by itself encourage*selective*holding\. In production, a useful hold policy should improve experience per unit of holding, i\.e\., it should reduce cancellations and excessive waiting while minimizing unnecessary holding of promising matches\. Therefore, we apply a tier\- and outcome\-dependent shaping term \(cost\-effectiveness shaping\):

\(9\)rce​\(st,at,yt\)=η​\(at,yt\)⋅rbase​\(yt\),r\_\{\\text\{ce\}\}\(s\_\{t\},a\_\{t\},y\_\{t\}\)=\\eta\(a\_\{t\},y\_\{t\}\)\\cdot r\_\{\\text\{base\}\}\(y\_\{t\}\),whereη​\(at,yt\)\\eta\(a\_\{t\},y\_\{t\}\)is designed to \(i\) reward higher tiers when they successfully filter experience\-degrading outcomes and \(ii\) penalize higher tiers more strongly when they unnecessarily delay completion\. This shaping makes the learning signal explicitly aligned with “experience ROI”: it discourages blunt strategies that hold too many candidates and instead favors policies that reserve longer holds for genuinely low\-experience cases\.

##### Temporal shaping\.

Hold control is tightly coupled with waiting\-related experience\. LetETPt\\text\{ETP\}\_\{t\}andPWTt\\text\{PWT\}\_\{t\}denote waiting\-related signals available at decision timett\(Estimated Time of Pickup and Passenger Waiting Time\), we construct a mild reward shaping term:

\(10\)rtime​\(st,at\)=ρ1​g​\(ETPt\)\+ρ2​g​\(PWTt\),r\_\{\\text\{time\}\}\(s\_\{t\},a\_\{t\}\)=\\rho\_\{1\}g\(\\text\{ETP\}\_\{t\}\)\+\\rho\_\{2\}g\(\\text\{PWT\}\_\{t\}\),whereg​\(⋅\)g\(\\cdot\)is a clipped monotone transform and\(ρ1,ρ2\)\(\\rho\_\{1\},\\rho\_\{2\}\)are small coefficients\. This term encodes a conservative inductive bias: when waiting is already high, overly aggressive hold is more likely to harm experience, and thus the policy should act more cautiously\. We keep this term deliberately small to avoid destabilizing training or overriding the outcome\-based utility\.

##### Final reward\.

The final experience\-aware reward is

\(11\)rt=rce​\(st,at,yt\)\+rtime​\(st,at\)\.r\_\{t\}=r\_\{\\text\{ce\}\}\(s\_\{t\},a\_\{t\},y\_\{t\}\)\+r\_\{\\text\{time\}\}\(s\_\{t\},a\_\{t\}\)\.We note that the design ofrtr\_\{t\}is not to serve as a perfect surrogate for any single target, but to provide a stable learning signal that consistently captures multi\-metric experience trade\-offs with temporal changes in a way that supports effective and stable deployment\.

In practice, jointly updating the encoder and the contextual bandit head can be unstable: encoder updates change the representation space used by the bandit, and exploration during training can amplify simulator imperfections\. To obtain a deployment\-friendly tiering policy with smooth tier evolution and predictable behavior, we adopt an alternating training scheme with explicit freezes\. Parameters for the reward shaping is detailed in Appendix[A](https://arxiv.org/html/2607.09090#A1), and the algorithmic details are provided in Appendix[B](https://arxiv.org/html/2607.09090#A2)\.

### 3\.3\.Stage II: Guardrail\-Constrained Hold\-Time Optimization

Stage II ofEXHOLDconverts experience tiers into hold durations by solving a constrained optimization problem based on empirical quantiles\. This stage provides explicit control over unnecessary holding of promising matches and yields monotone, auditable hold schedules that can be versioned and rolled back independently\.

#### 3\.3\.1\.Empirical Quantile Estimation

Given a fixed Stage I policy, we routeDOpair samples to tiers and estimate outcome\-conditional time distributions\. LetTTdenote the system\-defined hold clock, for action tieriiand outcome categoryj∈𝒴j\\in\\mathcal\{Y\}, we estimate the corresponding cumulative density function \(CDF\):

\(12\)CDFi,j\(x\)=ℙ\(T≤x∣a=i,y=j\),pi,j=Ni,j∑i′Ni′,j,\\text\{CDF\}\_\{i,j\}\(x\)=\\mathbb\{P\}\(T\\leq x\\mid a=i,y=j\),\\quad p\_\{i,j\}=\\frac\{N\_\{i,j\}\}\{\\sum\_\{i^\{\\prime\}\}N\_\{i^\{\\prime\},j\}\},whereNi,jN\_\{i,j\}is the number of samples with\(a=i,y=j\)\(a=i,y=j\)\. Quantile summaries are robust and inexpensive to refresh from recent logs\. In production, the estimated quantiles can be easily tracked over time and across cities to detect drift early, and refreshing these summaries enables lightweight, auditable re\-calibration of Stage II schedules without retraining the Stage I model, supporting safe iteration under non\-stationary traffic\.

#### 3\.3\.2\.Constrained Optimization with Unnecessary\-Hold Guardrail

We choose hold times\{xi\}\\\{x\_\{i\}\\\}to maximize expected utility while bounding the rate of unnecessarily holding completion cases\. We enforce: \(i\) a guardrail on completion \(unnecessary hold\) and \(ii\) a monotone schedule for interpretability:

\(13\)max\{xi\}\\displaystyle\\max\_\{\\\{x\_\{i\}\\\}\}\\quad∑i=0K∑j∈𝒴pi,j​vj​CDFi,j​\(xi\)\\displaystyle\\sum\_\{i=0\}^\{K\}\\sum\_\{j\\in\\mathcal\{Y\}\}p\_\{i,j\}\\,v\_\{j\}\\,\\text\{CDF\}\_\{i,j\}\(x\_\{i\}\)\(14\)s\.t\.∑i=1Kpi,TC​CDFi,TC​\(xi\)≤α,\\displaystyle\\sum\_\{i=1\}^\{K\}p\_\{i,\\textsc\{TC\}\}\\,\\text\{CDF\}\_\{i,\\textsc\{TC\}\}\(x\_\{i\}\)\\leq\\alpha,\(15\)0=x0≤x1≤⋯≤xK≤xmax,\\displaystyle 0=x\_\{0\}\\leq x\_\{1\}\\leq\\cdots\\leq x\_\{K\}\\leq x\_\{\\max\},wherevjv\_\{j\}is consistent with the experience utility in Eq\. \([8](https://arxiv.org/html/2607.09090#S3.E8)\), andα\\alphais a service guardrail that bounds unnecessary holding of promising matches\. SinceKKis small in practice, the optimization can be solved efficiently via dynamic programming under the monotonicity constraint, and it produces an explicit lookup table\{xi\}\\\{x\_\{i\}\\\}that is human\-reviewable, versioned, and immediately rollback\-able\.

### 3\.4\.Deployment and Iteration ofEXHOLD

The deployedEXHOLDat DiDi is the composition of two decoupled components \(shown in Figure[2](https://arxiv.org/html/2607.09090#S3.F2)\): \(i\) Stage I tiering policyπθ,ϕ\\pi\_\{\\theta,\\phi\}that maps eachDOpair statests\_\{t\}to an experience tierat∈𝒜a\_\{t\}\\in\\mathcal\{A\}, and \(ii\) Stage II hold\-time table\{xi\}i=0K\\\{x\_\{i\}\\\}\_\{i=0\}^\{K\}obtained by solving the guardrail\-constrained optimization in Eq\. \([13](https://arxiv.org/html/2607.09090#S3.E13)\)–\([15](https://arxiv.org/html/2607.09090#S3.E15)\)\. At serving time, the system computeszt=fθ​\(ht\)z\_\{t\}=f\_\{\\theta\}\(h\_\{t\}\), selects the tier via Eq\. \([7](https://arxiv.org/html/2607.09090#S3.E7)\), and executes hold durationxatx\_\{a\_\{t\}\}\. This design keeps online inference constant\-time and makes the execution rule fully explicit and auditable\.

One important practical benefit of the two\-stage decomposition inEXHOLDis safe and modular iteration\. Stage I can be updated to improve experience stratification \(e\.g\., better temporal representations or improved bandit calibration\), while Stage II can be refreshed using recent logs to re\-estimate empirical quantiles and re\-solve the constrained optimization, adjusting conservativeness through the guardrail parameterα\\alphawithout changing Stage I model weights\. This separation reduces deployment risk, supports rapid rollback, and provides a clean interface between learning and operations\.

## 4\.Experiments

We present a online A/B evaluation ofEXHOLDon DiDi’s large\-scale ride\-hailing platform\. Our experiments are designed to answer three questions:\(RQ1\)whetherEXHOLDimproves end\-to\-end trip success and passenger–driver experience under real traffic,\(RQ2\)which components are essential for the gains, and\(RQ3\)how the learned policy behaves in online deployment\. The application use and payoff ofEXHOLDis detailed in Section[4\.5](https://arxiv.org/html/2607.09090#S4.SS5)\.

### 4\.1\.Experimental Setup

We evaluateEXHOLDvia an online A/B experiment in DiDi’s production ride\-hailing matching system\. The control group uses our previous production strategy based on multiple prediction models plus hand\-designed threshold rules, which decide whether to hold a candidateDOpair and how long to hold\. The treatment group replaces this module withEXHOLD: Stage I assigns eachDOpair to a discrete experience tier using rich context \(Section[3\.2](https://arxiv.org/html/2607.09090#S3.SS2)\), and Stage II maps tiers to a monotone hold\-time table computed offline via constrained optimization with explicit service guardrails that limit unnecessary hold of promising matches \(Section[3\.3](https://arxiv.org/html/2607.09090#S3.SS3)\)\.

During our experiment, all eligible requests within a 30\-minute window are assigned to either control group or treatment group, and the assignment alternates over time\. We run the experiment from 2025\-12\-21 to 2026\-01\-10 \(21 days\) across five cities in Brazil, covering large\-scale production traffic \(over100100k calls per day across the experiment footprint\), and aggregate the results over the full experiment window and cities\.

We focus on experience\-oriented outcomes that hold control is designed to influence\. Our primary metric is the trip completion \(TC\) rate and driver income \(DI\) as the end\-to\-end measure of trip success\. We also report some important rates at our key funnel stages including PCBA, driver acceptance \(DA\), PCAA and DCAA, which capture distinct forms of experience degradation and wasted effort\. Finally, because hold directly affects timing, we include waiting\-related metrics such as call\-acceptance\-time \(CAT\) that records the duration from passenger call to driver acceptance\.

### 4\.2\.RQ1: Overall Online Evaluation

Table 1\.Online A/B results ofEXHOLDvs\. our production baseline\. We report treatment–control deltas \(T−\-C\)\. Positive deltas are desirable for metrics marked with↑\\uparrow; negative deltas are desirable for metrics marked with↓\\downarrow\.∗indicates statistical significance atp<0\.05p<0\.05\.MetricDelta \(T−\-C\)P\-valueSig\.Core MeasuresTrip completion \(TC\) ratio\(↑\\uparrow\)\+0\.49%0\.00273∗Driver income \(DI\)\(↑\\uparrow\)\+0\.50%0\.00014∗Passenger–Driver Experience MeasuresDriver acceptance \(DA\) ratio\(↑\\uparrow\)\+0\.48%0\.02597∗Passenger cancel before acceptance \(PCBA\)\(↓\\downarrow\)\-1\.95%0\.00402∗Passenger cancel after acceptance \(PCAA\)\(↓\\downarrow\)\-1\.99%0\.00459∗Driver cancel after acceptance \(DCAA\)\(↓\\downarrow\)\-0\.92%0\.21310Ride\-Hailing Funnel MeasuresCall\-acceptance time \(CAT\)\(↓\\downarrow\)\-1\.80%0\.04648∗Broadcasted / Called orders\(↑\\uparrow\)\+0\.42%0\.05775Accepted / Broadcasted orders\(↑\\uparrow\)\+1\.35%0\.01447∗Accepted / Called orders\(↑\\uparrow\)\+1\.77%0\.01143∗Table[1](https://arxiv.org/html/2607.09090#S4.T1)reports the online A/B results ofEXHOLDagainst our production baseline\. Overall,EXHOLDdelivers consistent improvements on both efficiency and experience\. In particular, we observe statistically significant gains in trip completion and driver income\. The uplift in trip completion suggests that the matches produced under experience\-aware hold control lead to a more satisfactory end\-to\-end trip outcome for both passengers and drivers\. Meanwhile, the improvement in driver income indicates that drivers are more willing to accept the recommended orders after the hold policy filters out low\-experience pairings, which directly benefits driver welfare and overall marketplace health\.

##### Experience improvements validate multi\-signal reward modeling\.

A key design choice ofEXHOLDis to learn experience tiers with a unified objective that aggregates multiple experience\-related signals across the matching funnel \(Section[3\.2\.4](https://arxiv.org/html/2607.09090#S3.SS2.SSS4)\)\. Online results support this design: we observe significant reductions in passenger cancellations both before acceptance and after acceptance\. These improvements are particularly important because passenger cancellations are a primary source of experience degradation and wasted matching effort\. Notably, the magnitude and significance ofPCBA/PCAAimprovements are slightly stronger thanDCAA, which is consistent with our reward weighting that places relatively more emphasis on passenger\-side experience\. This suggests a clear path for future online iteration: reward fine\-tuning can further strengthen driver\-side cancellation control without sacrificing overall experience gains\.

##### More precise holding leads to faster and better matching\.

Beyond outcome metrics,EXHOLDimproves key process indicators in the ride\-hailing funnel\. We observe a significant reduction in CAT, indicating that hold is executed more precisely: by delaying a small subset of low\-experienceDOpair candidates, the system surfaces better opportunities sooner, leading to faster driver response and a more efficient alignment of passenger–driver intent\. Consistent with this interpretation, we also see positive shifts in intermediate conversion metrics: Accepted/Broadcasted and Accepted/Called increases, both statistically significant\. Broadcasted/Called shows a mild increase with borderline significance, suggesting thatEXHOLDdoes not rely on aggressively suppressing exposure; instead, improvements primarily come from better conversion after exposure and fewer experience\-degrading cancellations\.

Table 2\.Segment\-level online results ofEXHOLD\. We report treatment–control deltas \(T−\-C\) for representative metrics across cities and time segments\.
##### Robustness across operational segments\.

Aggregate online results may hide heterogeneity across cities and traffic conditions\. To verify that the gains are not driven by a single city or a narrow traffic slice, we further evaluate representative metrics across five cities and peak/off\-peak periods; table[2](https://arxiv.org/html/2607.09090#S4.T2)reports the segment\-level results\. Across all five cities,EXHOLDconsistently improves TC and DI while reducingPCAAand CAT\. The magnitude varies across segments, but the direction remains stable, suggesting that the aggregate gains are not dominated by one city or a specific operational condition\.

The time\-segment results show a similar pattern\. Both peak and off\-peak hours achieve positive gains on TC and DI, together with reductions inPCAAand CAT\. The larger improvements during peak hours are consistent with the intuition that hold decisions become more consequential when marketplace competition and matching pressure are higher\. At the same time, the positive off\-peak results indicate thatEXHOLDdoes not rely solely on peak\-hour traffic conditions\. This robustness analysis focuses on online metric consistency, while Section[4\.4](https://arxiv.org/html/2607.09090#S4.SS4)further examines the behavioral patterns of hold decisions across pair attributes, temporal conditions, and spatial market regimes\.

Table 3\.Ablation study ofEXHOLDin online A/B experiments\. We report treatment–control deltas \(T−\-C\) for the two core measures\. Positive deltas are desirable \(↑\\uparrow\)\.

### 4\.3\.RQ2: Ablation Study

Table[3](https://arxiv.org/html/2607.09090#S4.T3)reports online ablations ofEXHOLDto isolate the contribution of key design choices in Stage I \(experience\-aware reward shaping\) and Stage II \(hold\-time execution\)\. All variants are evaluated in the same production A/B setting and we report deltas on the two core measures, TC and DI\.

Overall, the fullEXHOLDachieves the best joint improvement on TC and DI, while removing either shaping component in Stage I degrades performance substantially\. In particular, cost\-effectiveness shaping is critical for achieving positive trip completion gains: without it, TC turns negative even though DI becomes nearly neutral, suggesting that the policy loses its ability to trade off experience gains against unnecessary holding in a cost\-effective manner\. Removing all shaping terms leads to large regressions on both TC and DI, highlighting that the proposed multi\-signal reward modeling is essential for the performance ofEXHOLD\.

Stage II ablations further show that hold\-time execution must be guardrail\-constrained and tier\-specific\. Replacing Stage II with heuristic holds that only act on the worst tier using a fixed duration \(10s or 30s\) consistently harms both TC and DI\. This indicates that naive “hold only the worst cases” is insufficient in production: effective hold control requires calibrated, monotone hold times across experience tiers to avoid over\-holding and to maintain balanced improvements for both passengers and drivers\.

Table 4\.Sensitivity analysis of Stage I reward design\.##### Reward sensitivity\.

We further test alternative cancellation\-penalty structures in Stage I, including symmetric passenger/driver penalties and driver\-side heavier penalties\. The results, reported in Table[4](https://arxiv.org/html/2607.09090#S4.T4), show thatEXHOLDis not brittle to moderate reward perturbations: all variants remain directionally positive on TC and DI\. However, the penalty structure changes the trade\-off across funnel metrics\. Driver\-side heavier penalties slightly improveDCAAbut weaken passenger\-side cancellation reduction and yield smaller TC/DI gains, while theEXHOLDsetting achieves the best joint improvement on TC and DI\.

### 4\.4\.RQ3: Behavioral Analysis

![Refer to caption](https://arxiv.org/html/2607.09090v1/x3.png)Figure 3\.Pickup\-time/distance profiles ofDOpairs held byEXHOLD; horizontal dashed lines indicate the median pickup time/distance of non\-heldDOpairs\.![Refer to caption](https://arxiv.org/html/2607.09090v1/fig/time_bucket.png)\(a\)Time\-of\-day hold ratio\.
![Refer to caption](https://arxiv.org/html/2607.09090v1/fig/space_fence.png)\(b\)Hold ratio across supply–demand regimes\.

Figure 4\.Behavioral analysis ofEXHOLDacross temporal and spatial regimes\.EXHOLDholds more frequently in late\-night / early\-morning periods and in low\-density or low\-demand\-pressure regimes, suggesting that the learned policy adapts to market context rather than applying a uniform hold rate\.Section[4\.2](https://arxiv.org/html/2607.09090#S4.SS2)shows that the online gains are directionally consistent across cities and time segments\. We next examine the behavior of the learned hold policy itself\. This analysis complements the segment\-level online metrics by asking whetherEXHOLDaffects candidates in an interpretable and domain\-consistent manner, rather than obtaining gains through indiscriminate exposure suppression or a single hard\-coded heuristic\.

##### Pair\-level pickup profiles\.

We first examine the pickup time and pickup distance distributions ofDOpairs held byEXHOLD\. Figure[3](https://arxiv.org/html/2607.09090#S4.F3)plots the percentile curves of pickup time and pickup distance for held candidates\. For reference, the horizontal dashed lines indicate the median pickup time and pickup distance of non\-heldDOpairs\.

The held candidates tend to have longer pickup time and/or longer pickup distance, as their percentile curves lie above the non\-held median at many percentiles\. This pattern is consistent with domain intuition: long pickup time and distance are often associated with worse passenger–driver experience and higher cancellation propensity\. Delaying such candidates can therefore create opportunities for more mutually satisfactory matches\.

Importantly, the behavior is not reducible to a single pickup\-time or pickup\-distance threshold\. Some heldDOpairs still have relatively short pickup time or distance, while not all long\-pickup candidates are necessarily held\. This is consistent with the design ofEXHOLD: Stage I assigns an experience tier by aggregating multiple funnel and waiting\-related signals, and Stage II executes tier\-specific hold durations obtained from constrained optimization\. Therefore, the learned policy performs calibrated experience\-aware filtering rather than applying a one\-size\-fits\-all rule\.

##### Temporal behavior\.

We next examine whether the hold policy behaves sensibly across diurnal cycles\. Figure[4](https://arxiv.org/html/2607.09090#S4.F4)\(a\) reports the ratio of held candidates over half\-hour time buckets\. The hold ratio is higher during late\-night and early\-morning periods, while remaining lower and relatively stable during daytime\. This pattern is consistent with operational intuition that off\-peak periods often contain more uncertain or lower\-connectivity matching opportunities\. The result suggests thatEXHOLDadapts its hold intensity to temporal market conditions, instead of applying a uniform hold rate throughout the day\.

##### Spatial regime behavior\.

Finally, we analyze how hold decisions vary across local supply–demand regimes\. We stratify candidates by spatiotemporal fence level, which reflects demand pressure, and by order density, which reflects local connectivity and available matching opportunities\. As shown in Figure[4](https://arxiv.org/html/2607.09090#S4.F4)\(b\),EXHOLDholds more frequently in low\-demand\-pressure and low\-density regimes \(L0\-L4\), where riskyDOpairs are generally more common\. In medium\-to\-high demand regimes \(L5\-L10\), the hold ratio becomes more moderate and varies with local density, indicating that the policy does not simply map regime labels to fixed hold actions\. Within comparable demand\-pressure groups, the hold ratio generally decreases as density increases \(e\.g\., L5\-L7, L8\-L10\), suggesting that the policy becomes less conservative with higher local connectivity\.

Together, these behavioral analyses show thatEXHOLDmakes hold decisions that are interpretable at multiple levels\. At the pair level, it tends to delay candidates with higher pickup burden\. At the temporal and spatial levels, it increases hold intensity in regimes where matching uncertainty is higher, while avoiding uniformly aggressive holding in better\-connected markets\. These patterns support our interpretation that the online gains are driven by selective risk reduction and calibrated hold execution\.

### 4\.5\.Application Use and Payoff

EXHOLDhas been deployed in DiDi’s production ride\-hailing matching system and is currently running to serve large\-scale real traffic in Brazil, becoming an active component of DiDi’s marketplace policy stack\. In production operation,EXHOLDdelivers sustained gains on trip completion and passenger–driver experience, demonstrating that experience\-aware real\-time hold control can create measurable end\-to\-end benefits\.

##### From offline design to production deployment\.

Offline evaluation played a central role in turningEXHOLDinto a shippable policy\. For Stage I, we iterated the reward design and used a*cost\-effectiveness score*as the primary model\-selection signal to guide reward shaping and weight tuning[A](https://arxiv.org/html/2607.09090#A1)\. This score improved from0\.890\.89to1\.631\.63, and further to2\.622\.62after introducing cost\-effectiveness and temporal shaping, indicating that the learned policy increasingly reduces experience\-degrading outcomes per unit of unnecessary holding\. For Stage II, we explored multiple constraint configurations and found that setting the guardrail parameter toα=0\.01\\alpha=0\.01yields the best balance in our offline simulation and also translates most reliably to online A/B performance, aligning the hold\-time schedule with practical service constraints while preserving experience gains\.

##### Monitoring, rollback, and serving efficiency\.

To ensure stable production performance during experimentation and deployment, we built end\-to\-end monitoring around several critical checkpoints\. We continuously track the distribution of Stage I outputs \(the fraction ofDOpairs assigned to each experience tier\) and alert on abrupt shifts that may indicate feature issues\. We also monitor Stage II optimization outputs \(e\.g\., the monotone hold\-time table and constraint satisfaction\) and key online service indicators, with automatic alarm and rollback mechanisms to protect against regressions\. From an efficiency perspective, engineering profiling shows thatEXHOLDhas a computational footprint comparable to the existing predictive\-model\-based hold module: Stage I adds a bounded\-cost embedding inference and discrete selection, while Stage II is an offline table lookup at runtime\. Stress tests under peak traffic further confirm that the method meets the latency and throughput requirements of real\-time, large\-scale inference in production\.

## 5\.Related Work

##### Ride\-hailing dispatch and matching systems\.

Large\-scale ride\-hailing platforms require real\-time, high\-throughput decision making over a dynamic bipartite market under tight latency and reliability constraints\. Prior work has studied learning\-augmented dispatch and matching mechanisms at industrial scale, including approaches that combine learning with downstream planning or optimization to improve global efficiency and user experience\(Xu et al\.,[2018](https://arxiv.org/html/2607.09090#bib.bib29); Liu et al\.,[2022](https://arxiv.org/html/2607.09090#bib.bib14)\)\. A complementary line of work focuses on modeling pairwise matching quality, e\.g\., predicting the success probability of passenger–driver pairs under multi\-view features and deployment constraints\(Wang et al\.,[2021](https://arxiv.org/html/2607.09090#bib.bib26); Teusch et al\.,[2023](https://arxiv.org/html/2607.09090#bib.bib22); Sühr et al\.,[2019](https://arxiv.org/html/2607.09090#bib.bib18)\)\. More broadly, research in operations and marketplaces has analyzed the coupling between matching, waiting time, and platform objectives such as experience and efficiency\(Yan et al\.,[2020](https://arxiv.org/html/2607.09090#bib.bib30); Taylor,[2024](https://arxiv.org/html/2607.09090#bib.bib21); Ke et al\.,[2021](https://arxiv.org/html/2607.09090#bib.bib9)\), which motivates decision modules that explicitly manage waiting\-related outcomes and cancellations\.

##### Experience quality and funnel\-aware objectives\.

Cancellations in ride\-hailing are heterogeneous and occur at different stages of the funnel, reflecting distinct sources of experience degradation and operational waste\(Gangadharaiah,[2025](https://arxiv.org/html/2607.09090#bib.bib7); Xu,[2023](https://arxiv.org/html/2607.09090#bib.bib28); Feng et al\.,[2021](https://arxiv.org/html/2607.09090#bib.bib6)\)\. While many industrial systems rely on predictive models \(e\.g\., completion/cancellation estimators\) to drive heuristic thresholding policies, such designs can struggle with multi\-objective coordination and cascading errors when multiple models are chained\(Supian et al\.,[2024](https://arxiv.org/html/2607.09090#bib.bib19); Guo et al\.,[\[n\. d\.\]](https://arxiv.org/html/2607.09090#bib.bib8); Supian et al\.,[2025](https://arxiv.org/html/2607.09090#bib.bib20)\)\. Recent platform studies highlight the importance of accounting for two\-sided effects and system\-level externalities in repeated matching markets, including fairness and long\-term outcomes beyond immediate conversion\(Sühr et al\.,[2019](https://arxiv.org/html/2607.09090#bib.bib18)\)\. Contextual bandits and constrained optimization have also been explored for dispatch and other marketplace controls\(Qin et al\.,[2021](https://arxiv.org/html/2607.09090#bib.bib16); Yuan et al\.,[2022](https://arxiv.org/html/2607.09090#bib.bib33); Liu et al\.,[2022](https://arxiv.org/html/2607.09090#bib.bib14),[2025](https://arxiv.org/html/2607.09090#bib.bib13)\), but production deployment often faces challenges including delayed effects, distribution shift, and safety constraints\.

## 6\.Conclusion

This paper presentedEXHOLD, a deployable two\-stage framework for experience\-aware real\-time hold control in large\-scale ride\-hailing matching at DiDi global markets\. The key idea ofEXHOLDis to decouple experience\-aware pair assessment from hold\-time execution under explicit service guardrails: Stage I learns discrete and interpretable experience tiers from sequential context using a robust decision model, and Stage II converts tiers into a monotone hold\-time schedule via constrained optimization over empirical quantiles\. Through online A/B experiments and production deployment, we show thatEXHOLDimproves trip success and driver welfare while reducing experience\-degrading cancellations and improving funnel efficiency, providing real\-world evidence that properly calibrated hold control can benefit both sides of the marketplace at DiDi\.

## References

- \(1\)
- Afèche et al\.\(2023\)Philipp Afèche, Zhe Liu, and Costis Maglaras\. 2023\.Ride\-hailing networks with strategic drivers: The impact of platform control capabilities on performance\.*Manufacturing & Service Operations Management*25, 5 \(2023\), 1890–1908\.
- Barbour and Luiz \(2019\)Olivia Barbour and John Luiz\. 2019\.Embracing solutions\-driven innovation to address institutional voids: The case of uber and the middle of the pyramid\.*California Management Review*62, 1 \(2019\), 31–52\.
- Chen et al\.\(2021\)Long Chen, Piyushimita Thakuriah, and Konstantinos Ampountolas\. 2021\.Short\-term prediction of demand for ride\-hailing services: A deep learning approach\.*Journal of Big Data Analytics in Transportation*3, 2 \(2021\), 175–195\.
- Chu et al\.\(2011\)Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire\. 2011\.Contextual bandits with linear payoff functions\. In*Proceedings of the fourteenth international conference on artificial intelligence and statistics*\. JMLR Workshop and Conference Proceedings, 208–214\.
- Feng et al\.\(2021\)Guiyun Feng, Guangwen Kong, and Zizhuo Wang\. 2021\.We are on the way: Analysis of on\-demand ride\-hailing systems\.*Manufacturing & Service Operations Management*23, 5 \(2021\), 1237–1256\.
- Gangadharaiah \(2025\)Rakesh Gangadharaiah\. 2025\.Understanding and Modeling Pooled Rideshare Acceptance: Influential Factors, Preferred User Experiences, and Implications\.\(2025\)\.
- Guo et al\.\(\[n\. d\.\]\)Yuhan Guo, Wenhua Li, Linfan Xiao, Hamid Allaoui, et al\.\[n\. d\.\]\.Multi\-Objective Dispatching Strategy of Autonomous Service Vehicles in Ride\-Hailing Based on an Interpretable Prediction Model\.*Hamid, Multi\-Objective Dispatching Strategy of Autonomous Service Vehicles in Ride\-Hailing Based on an Interpretable Prediction Model*\(\[n\. d\.\]\)\.
- Ke et al\.\(2021\)Jintao Ke, Zheng Zhu, Hai Yang, and Qiaochu He\. 2021\.Equilibrium analyses and operational designs of a coupled market with substitutive and complementary ride\-sourcing services to public transits\.*Transportation Research Part E: Logistics and Transportation Review*148 \(2021\), 102236\.
- Kotary et al\.\(2021\)James Kotary, Ferdinando Fioretto, Pascal Van Hentenryck, and Bryan Wilder\. 2021\.End\-to\-end constrained optimization learning: A survey\.*arXiv preprint arXiv:2103\.16378*\(2021\)\.
- Lee et al\.\(2022\)Kyunghee Lee, Qianran Jin, Animesh Animesh, and Jui Ramaprasad\. 2022\.Impact of ride\-hailing services on transportation mode choices: evidence from traffic and transit ridership\.*MIS Quarterly*46, 4 \(2022\), 1875–1900\.
- Liang et al\.\(2022\)Jing Liang, Xuanxuan Ban, Kunjie Yu, Boyang Qu, Kangjia Qiao, Caitong Yue, Ke Chen, and Kay Chen Tan\. 2022\.A survey on evolutionary constrained multiobjective optimization\.*IEEE Transactions on Evolutionary Computation*27, 2 \(2022\), 201–221\.
- Liu et al\.\(2025\)Xu Liu, Yiqiang Lu, Jian Liu, Tianyi Zhang, Weiqiang Wang, Qian Liu, and Shuai Li\. 2025\.Adaptive Merchant\-Centric Risk Control via Unbiased Decision\-Making and Dynamic Optimization in E\-Commerce\. In*Proceedings of the AAAI Conference on Artificial Intelligence*, Vol\. 39\. 28783–28791\.
- Liu et al\.\(2022\)Yang Liu, Fanyou Wu, Cheng Lyu, Shen Li, Jieping Ye, and Xiaobo Qu\. 2022\.Deep dispatching: A deep reinforcement learning approach for vehicle dispatching on online ride\-hailing platform\.*Transportation Research Part E: Logistics and Transportation Review*161 \(2022\), 102694\.
- Naumov and Keith \(2023\)Sergey Naumov and David Keith\. 2023\.Optimizing the economic and environmental benefits of ride\-hailing and pooling\.*Production and Operations Management*32, 3 \(2023\), 904–929\.
- Qin et al\.\(2021\)Guoyang Qin, Qi Luo, Yafeng Yin, Jian Sun, and Jieping Ye\. 2021\.Optimizing matching time intervals for ride\-hailing services using reinforcement learning\.*Transportation Research Part C: Emerging Technologies*129 \(2021\), 103239\.
- Somalwar et al\.\(2025\)Anne Somalwar, Bruce D Lee, George J Pappas, and Nikolai Matni\. 2025\.Learning with Imperfect Models: When Multi\-step Prediction Mitigates Compounding Error\.*arXiv preprint arXiv:2504\.01766*\(2025\)\.
- Sühr et al\.\(2019\)Tom Sühr, Asia J Biega, Meike Zehlike, Krishna P Gummadi, and Abhijnan Chakraborty\. 2019\.Two\-sided fairness for repeated matchings in two\-sided markets: A case study of a ride\-hailing platform\. In*Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining*\. 3082–3092\.
- Supian et al\.\(2024\)Sudradjat Supian, Subiyanto, Tubagus Robbi Megantara, and Abdul Talib Bon\. 2024\.Ride\-Hailing Matching with Uncertain Travel Time: A Novel Interval\-Valued Fuzzy Multi\-Objective Linear Programming Approach\.*Mathematics*12, 9 \(2024\), 1355\.
- Supian et al\.\(2025\)Sudradjat Supian, Subiyanto Subiyanto, Sisilia Sylviani, Tubagus Robbi Megantara, Abdul Talib Bon, and Vasile Preda\. 2025\.Mathematical Modeling of Ride\-Hailing Matching Considering Uncertain User and Driver Preferences: Interval\-Valued Fuzzy Approach\.*Mathematics*13, 3 \(2025\), 371\.
- Taylor \(2024\)Terry A Taylor\. 2024\.Shared\-ride efficiency of ride\-hailing platforms\.*Manufacturing & Service Operations Management*26, 5 \(2024\), 1945–1961\.
- Teusch et al\.\(2023\)Julian Teusch, Jan Niklas Gremmel, Christian Koetsier, Fatema Tuj Johora, Monika Sester, David M Woisetschläger, and Jörg P Müller\. 2023\.A systematic literature review on machine learning in shared mobility\.*IEEE Open Journal of Intelligent Transportation Systems*4 \(2023\), 870–899\.
- Tirachini \(2020\)Alejandro Tirachini\. 2020\.Ride\-hailing, travel behaviour and sustainable mobility: an international review\.*Transportation*47, 4 \(2020\), 2011–2047\.
- Tu \(2024\)Yuanjie Tu\. 2024\.*Towards sustainability: decoding ridehailing drivers’ and passengers’ behaviors*\.Ph\. D\. Dissertation\. University of Washington\.
- Vaswani et al\.\(2017\)Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin\. 2017\.Attention is all you need\.*Advances in neural information processing systems*30 \(2017\)\.
- Wang et al\.\(2021\)Yuandong Wang, Hongzhi Yin, Lian Wu, Tong Chen, and Chunyang Liu\. 2021\.Secure your ride: Real\-time matching success rate prediction for passenger\-driver pairs\.*IEEE Transactions on Knowledge and Data Engineering*35, 3 \(2021\), 3059–3071\.
- Wen et al\.\(2024\)Dacheng Wen, Yupeng Li, and Francis CM Lau\. 2024\.A survey of machine learning\-based ride\-hailing planning\.*IEEE Transactions on Intelligent Transportation Systems*25, 6 \(2024\), 4734–4753\.
- Xu \(2023\)Kai Xu\. 2023\.*Modeling and Management of Ridesourcing Services with Order Cancellation and Platform Collaboration*\.Ph\. D\. Dissertation\. UNSW Sydney\.
- Xu et al\.\(2018\)Zhe Xu, Zhixin Li, Qingwen Guan, Dingshui Zhang, Qiang Li, Junxiao Nan, Chunyang Liu, Wei Bian, and Jieping Ye\. 2018\.Large\-scale order dispatch in on\-demand ride\-hailing platforms: A learning and planning approach\. In*Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining*\. 905–913\.
- Yan et al\.\(2020\)Chiwei Yan, Helin Zhu, Nikita Korolko, and Dawn Woodard\. 2020\.Dynamic pricing and matching in ride\-hailing platforms\.*Naval Research Logistics \(NRL\)*67, 8 \(2020\), 705–724\.
- Yang et al\.\(2020\)Bo Yang, Shen Ren, Erika Fille Legara, Zengxiang Li, Edward YX Ong, Louis Lin, and Christopher Monterola\. 2020\.Phase transition in taxi dynamics and impact of ridesharing\.*Transportation Science*54, 1 \(2020\), 250–273\.
- Yatnalkar \(2019\)Govind Pramod Yatnalkar\. 2019\.A Machine Learning Recommender Model for Ride Sharing Based on Rider Characteristics and User Threshold Time\.\(2019\)\.
- Yuan et al\.\(2022\)Enpeng Yuan, Wenbo Chen, and Pascal Van Hentenryck\. 2022\.Reinforcement learning from optimization proxy for ride\-hailing vehicle relocation\.*Journal of Artificial Intelligence Research*75 \(2022\), 985–1002\.
- Zhang et al\.\(2017\)Lingyu Zhang, Tao Hu, Yue Min, Guobin Wu, Junying Zhang, Pengcheng Feng, Pinghua Gong, and Jieping Ye\. 2017\.A taxi order dispatch model based on combinatorial optimization\. In*Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining*\. 2151–2159\.

## Appendix AAdditional Details

### A\.1\.Reward Table and Shaping Details

This appendix provides a concrete instantiation of the experience\-oriented reward used in Stage I \(Section[3\.2\.4](https://arxiv.org/html/2607.09090#S3.SS2.SSS4)\)\. The reward is composed of \(i\) a base outcome utility, \(ii\) tier\-dependent cost\-effectiveness shaping, and \(iii\) mild temporal shaping\. Throughout, the actiona∈\{0,1,2,3\}a\\in\\\{0,1,2,3\\\}denotes the predicted*experience tier*, where largeraacorresponds to lower expected experience and thus a more conservative holding intent\.

#### A\.1\.1\.Base reward table

We define a categorical outcomeyyfor eachDOpair that reflects the realized end state in the funnel: completion \(TC\), passenger cancel before acceptance \(PCBA\), passenger cancel after acceptance \(PCAA\), driver cancel after acceptance \(DCAA\), and*other*\(e\.g\., non\-response or undefined terminals\)\. The base utilityrbase​\(y\)r\_\{\\text\{base\}\}\(y\)sets completion as the unit benefit and assigns asymmetric penalties to experience\-degrading outcomes\. A reasonable instantiation \(used in our offline iteration\) is shown in Table[5](https://arxiv.org/html/2607.09090#A1.T5)\.

Table 5\.Base reward tablerbase​\(y\)r\_\{\\text\{base\}\}\(y\)\.The table reflects two principles: \(i\) passenger cancellations after acceptance are typically the most harmful \(wasted pickup effort plus poor passenger experience\), and \(ii\) non\-response / other outcomes are mildly negative to discourage inefficient pairings\.

#### A\.1\.2\.Tier\-dependent cost\-effectiveness shaping

The base reward alone does not encode*how aggressively*the policy should hold; in production, we want the policy to reduce experience\-degrading outcomes while limiting unnecessary holding of promising matches\. We therefore apply a multiplicative shaping term:

\(16\)rce​\(s,a,y\)=η​\(a,y\)⋅rbase​\(y\),r\_\{\\text\{ce\}\}\(s,a,y\)\\;=\\;\\eta\(a,y\)\\cdot r\_\{\\text\{base\}\}\(y\),whereη​\(a,y\)\\eta\(a,y\)adjusts the utility to reflect effectiveness across tiers\.

##### Design intuition\.

For higher tiers, it is desirable to \(i\) assign larger positive credit when a heldDOpair would likely lead to cancellations and \(ii\) assign larger penalties if the policy unnecessarily delays aDOpair that would complete\. For lower tiers, the policy is encouraged to be permissive and avoid holding\.

##### Deployed instantiation\.

The parameterization for base reward shaping is:

\(17\)η​\(a,y\)=\{1−κmis⋅a,y=TC,1\+κcba⋅a,y=PCBA,1\+κpcaa⋅a,y=PCAA,1\+κdcaa⋅a,y=DCAA,1,otherwise,\\eta\(a,y\)=\\begin\{cases\}1\-\\kappa\_\{\\text\{mis\}\}\\cdot a,&y=\\textsc\{TC\},\\\\ 1\+\\kappa\_\{\\text\{cba\}\}\\cdot a,&y=\\textsc\{PCBA\},\\\\ 1\+\\kappa\_\{\\text\{pcaa\}\}\\cdot a,&y=\\textsc\{PCAA\},\\\\ 1\+\\kappa\_\{\\text\{dcaa\}\}\\cdot a,&y=\\textsc\{DCAA\},\\\\ 1,&\\text\{otherwise\},\\end\{cases\}with non\-negative coefficientsκmis,κcba,κpcaa,κdcaa\\kappa\_\{\\text\{mis\}\},\\kappa\_\{\\text\{cba\}\},\\kappa\_\{\\text\{pcaa\}\},\\kappa\_\{\\text\{dcaa\}\}\. In our tuning, we are usingκpcaa≥κdcaa≥κcba\\kappa\_\{\\text\{pcaa\}\}\\geq\\kappa\_\{\\text\{dcaa\}\}\\geq\\kappa\_\{\\text\{cba\}\}andκmis\\kappa\_\{\\text\{mis\}\}to cap unnecessary holding\. Table[6](https://arxiv.org/html/2607.09090#A1.T6)shows the design ofη​\(a,y\)\\eta\(a,y\)values under a reasonable setting \(κmis=0\.15\\kappa\_\{\\text\{mis\}\}\{=\}0\.15,κcba=0\.15\\kappa\_\{\\text\{cba\}\}\{=\}0\.15,κpcaa=0\.20\\kappa\_\{\\text\{pcaa\}\}\{=\}0\.20,κdcaa=0\.12\\kappa\_\{\\text\{dcaa\}\}\{=\}0\.12\), witha∈\{0,1,2,3\}a\\in\\\{0,1,2,3\\\}\.

Table 6\.Shaping multipliersη​\(a,y\)\\eta\(a,y\)by tieraaand outcomeyy\.
##### Notes\.

This shaping can also be implemented additively \(instead of multiplicatively\) or with a bounded nonlinear form\. We choose the simple linear form in Eq\. \([17](https://arxiv.org/html/2607.09090#A1.E17)\) because it is easy to audit and tune, and we found it stable in offline iteration and online ablations\.

#### A\.1\.3\.Temporal shaping

Hold decisions are closely tied to waiting\-related experience\. We incorporate a mild bounded temporal term:

\(18\)rtime​\(s,a\)=ρ1⋅g​\(ETA\)\+ρ2⋅g​\(PWT\),r\_\{\\text\{time\}\}\(s,a\)\\;=\\;\\rho\_\{1\}\\cdot g\(\\textsc\{ETA\}\)\\;\+\\;\\rho\_\{2\}\\cdot g\(\\textsc\{PWT\}\),whereETAis estimated pickup time,PWTis passenger waiting time \(or time since call\), andg​\(⋅\)g\(\\cdot\)is a clipped monotone transform\. A concrete choice is:

\(19\)g​\(u\)=clip​\(u−u0u1−u0,0,1\),g\(u\)=\\text\{clip\}\\Big\(\\frac\{u\-u\_\{0\}\}\{u\_\{1\}\-u\_\{0\}\},\\,0,\\,1\\Big\),with\(u0,u1\)\(u\_\{0\},u\_\{1\}\)chosen from operational ranges \(e\.g\.,u0=2​minu\_\{0\}\{=\}2\\text\{ min\},u1=12​minu\_\{1\}\{=\}12\\text\{ min\}for time features, and analogous values for distance if used\)\. In practice, we keep temporal shaping small to avoid overwhelming outcome utility; typical ranges areρ1,ρ2∈\[0\.02,0\.10\]\\rho\_\{1\},\\rho\_\{2\}\\in\[0\.02,0\.10\]\.

##### Final reward\.

The final scalar reward is:

\(20\)r​\(s,a,y\)=rce​\(s,a,y\)\+rtime​\(s,a\)\.r\(s,a,y\)\\;=\\;r\_\{\\text\{ce\}\}\(s,a,y\)\\;\+\\;r\_\{\\text\{time\}\}\(s,a\)\.This construction supports experience\-aware learning while remaining robust and tunable for deployment\.

### A\.2\.Optimization Derivations

This appendix details Stage II, which converts experience tiers into hold durations via constrained optimization over empirical quantiles \(Section[3\.3\.2](https://arxiv.org/html/2607.09090#S3.SS3.SSS2)\)\. We provide \(i\) an equivalent formulation, \(ii\) solvability notes, and \(iii\) implementation details used in production\.

#### A\.2\.1\.Formulation recap and interpretation

For each tieri∈\{0,…,K\}i\\in\\\{0,\\dots,K\\\}and outcome categoryj∈𝒴j\\in\\mathcal\{Y\}, we estimate an empirical CDF:

\(21\)CDFi,j\(x\)=ℙ\(T≤x∣a=i,y=j\),\\text\{CDF\}\_\{i,j\}\(x\)=\\mathbb\{P\}\(T\\leq x\\mid a=i,y=j\),whereTTis the system\-defined execution clock \(e\.g\., a waiting\-related time variable\), and we also compute routing fractionspi,jp\_\{i,j\}\. Given utility weightsvjv\_\{j\}\(consistent with the reward table\), we solve:

\(22\)max\{xi\}\\displaystyle\\max\_\{\\\{x\_\{i\}\\\}\}\\quad∑i=0K∑j∈𝒴pi,j​vj​CDFi,j​\(xi\)\\displaystyle\\sum\_\{i=0\}^\{K\}\\sum\_\{j\\in\\mathcal\{Y\}\}p\_\{i,j\}\\,v\_\{j\}\\,\\text\{CDF\}\_\{i,j\}\(x\_\{i\}\)\(23\)s\.t\.∑i=1Kpi,TC​CDFi,TC​\(xi\)≤α,\\displaystyle\\sum\_\{i=1\}^\{K\}p\_\{i,\\textsc\{TC\}\}\\,\\text\{CDF\}\_\{i,\\textsc\{TC\}\}\(x\_\{i\}\)\\leq\\alpha,\(24\)0=x0≤x1≤⋯≤xK≤xmax\.\\displaystyle 0=x\_\{0\}\\leq x\_\{1\}\\leq\\cdots\\leq x\_\{K\}\\leq x\_\{\\max\}\.
For a fixed tierii,CDFi,j​\(xi\)\\text\{CDF\}\_\{i,j\}\(x\_\{i\}\)is the fraction of outcome\-jjevents that would be “covered” by holding for durationxix\_\{i\}under the execution clockTT\. The objective aggregates expected utility across outcomes; the constraint bounds the fraction of completion cases that are unnecessarily held, controlled byα\\alpha\.

#### A\.2\.2\.Solvability and structure

Table 7\.Stage II optimization parameters and ranges\.AlthoughCDFi,j​\(⋅\)\\text\{CDF\}\_\{i,j\}\(\\cdot\)is non\-decreasing, the optimization is not necessarily convex because empirical CDFs are step functions\. However, the problem is low\-dimensional \(KKis small in our production\) and admits efficient exact or near\-exact solutions\. Because eachCDFi,j​\(x\)\\text\{CDF\}\_\{i,j\}\(x\)changes only at observed sample values ofTT, the optimizer only needs to consider a discrete candidate set𝒢i\\mathcal\{G\}\_\{i\}for each tierii\. Our practical choice is to use quantile grid points:

\(25\)𝒢i=\{Qi​\(q\)∣q∈\{0,0\.01,0\.02,…,0\.99\}\},\\mathcal\{G\}\_\{i\}=\\\{Q\_\{i\}\(q\)\\mid q\\in\\\{0,0\.01,0\.02,\\dots,0\.99\\\}\\\},whereQi​\(q\)Q\_\{i\}\(q\)is the empirical quantile ofTTamong tier\-iisamples\. This converts the problem into a finite constrained search\.

##### DP under monotonicity\.

Under the monotone constraintx0≤x1≤⋯≤xKx\_\{0\}\\leq x\_\{1\}\\leq\\cdots\\leq x\_\{K\}, we can solve the discretized problem by dynamic programming\. Let𝒢i\\mathcal\{G\}\_\{i\}be sorted in ascending order and define

\(26\)Ui​\(x\)=∑j∈𝒴pi,j​vj​CDFi,j​\(x\),Ci​\(x\)=pi,TC​CDFi,TC​\(x\)\.U\_\{i\}\(x\)=\\sum\_\{j\\in\\mathcal\{Y\}\}p\_\{i,j\}\\,v\_\{j\}\\,\\text\{CDF\}\_\{i,j\}\(x\),\\qquad C\_\{i\}\(x\)=p\_\{i,\\textsc\{TC\}\}\\,\\text\{CDF\}\_\{i,\\textsc\{TC\}\}\(x\)\.Then the problem becomes:

\(27\)maxxi∈𝒢i​∑i=0KUi​\(xi\)s\.t\.∑i=1KCi​\(xi\)≤α,x0≤x1≤⋯≤xK\.\\max\_\{x\_\{i\}\\in\\mathcal\{G\}\_\{i\}\}\\sum\_\{i=0\}^\{K\}U\_\{i\}\(x\_\{i\}\)\\quad\\text\{s\.t\.\}\\quad\\sum\_\{i=1\}^\{K\}C\_\{i\}\(x\_\{i\}\)\\leq\\alpha,\\ \\ x\_\{0\}\\leq x\_\{1\}\\leq\\cdots\\leq x\_\{K\}\.With a discretized constraint budget, the DP complexity is modest\. In practice,K=3K\{=\}3and\|𝒢i\|\|\\mathcal\{G\}\_\{i\}\|is on the order of 50–200, making the solver easily fast enough for frequent recalibration\. Table[7](https://arxiv.org/html/2607.09090#A1.T7)summarizes the parameters used in Stage II optimization\.

## Appendix BAlternating Training for Stable Tiering under Simulator Imperfections

Here we provide additional details on the alternating training procedure used in Stage I ofEXHOLD\. The goal is to obtain a tiering policy that is \(i\) stable under representation drift, \(ii\) robust to non\-stationary traffic regimes, and \(iii\) safe to deploy in DiDi’s production matching system where rapid rollback and predictable behavior are required\.

### B\.1\.Why Alternating Updates are Necessary

Stage I combines an encoderfθf\_\{\\theta\}with a contextual bandit head parameterized byϕ\\phi\. Although one could jointly update\(θ,ϕ\)\(\\theta,\\phi\), we found such training unstable in deployment\-oriented settings\. The main reason is that the bandit head relies on embeddingszt=fθ​\(ht\)z\_\{t\}=f\_\{\\theta\}\(h\_\{t\}\)to maintain calibrated statistics\. When the encoder changes continuously, the feature space drifts, which can invalidate the accumulated bandit statistics and lead to abrupt tier\-distribution shifts\.

A second issue is simulator\-induced bias\. Exploration is necessary for improving the tiering policy beyond logged behavior, but aggressive exploration in a simulator may exploit artifacts that do not transfer to production\. If the encoder and bandit are updated simultaneously under such biased trajectories, the entire policy can drift toward unstable or non\-transferable behavior\.

Therefore, we use alternating updates to decouple representation learning from bandit policy improvement, which yields smoother tier evolution and more predictable rollout behavior\.

### B\.2\.Training Data Sources and Notation

We assume access to an offline log dataset𝒟0\\mathcal\{D\}\_\{0\}collected under a strong production policy and an interaction environmentℰ\\mathcal\{E\}for iterative improvement\. In our implementation,ℰ\\mathcal\{E\}is instantiated as a simulator, though the same procedure can also use controlled traffic slices under stricter deployment constraints\. Each interaction produces a tuple\(ht,at,rt,yt\)\(h\_\{t\},a\_\{t\},r\_\{t\},y\_\{t\}\), wherehth\_\{t\}is the sequential context,ata\_\{t\}is the selected tier,rtr\_\{t\}is the experience reward defined in Section[3\.2\.4](https://arxiv.org/html/2607.09090#S3.SS2.SSS4), andyty\_\{t\}is the realized outcome category\.

The encoder maps the context to an embeddingzt=fθ​\(ht\)z\_\{t\}=f\_\{\\theta\}\(h\_\{t\}\), and the bandit head selects tiers according to the UCB rule in Eq\. \([7](https://arxiv.org/html/2607.09090#S3.E7)\)\. For encoder training, we define a utility labelR~t\\tilde\{R\}\_\{t\}as a short\-horizon discounted return:

\(28\)R~t=∑τ=0H−1γτ​rt\+τ,\\tilde\{R\}\_\{t\}=\\sum\_\{\\tau=0\}^\{H\-1\}\\gamma^\{\\tau\}r\_\{t\+\\tau\},whereHHis the horizon length andγ∈\(0,1\]\\gamma\\in\(0,1\]is the discount factor\.

### B\.3\.Alternating Training Procedure

We adopt alternating optimization with explicit freezes to decouple representation learning from exploration\. Each outer iteration consists of two phases:

##### Phase A: Bandit adaptation with frozen encoder\.

We freezeθ\\thetaand update the bandit headϕ\\phiby interacting with the simulated environmentℰ\\mathcal\{E\}\. Because the representation is fixed, the bandit statistics remain well\-defined and the tier distribution evolves smoothly\. This phase is responsible for controlled exploration and fast adaptation of tier selection\.

Algorithm 1Alternating Training for Stable Tiering1:Offline logs

𝒟0\\mathcal\{D\}\_\{0\}, simulated environment

ℰ\\mathcal\{E\}, outer iterations

KK, bandit steps

TbanditT\_\{\\text\{bandit\}\}, encoder steps

TencT\_\{\\text\{enc\}\}
2:Initialize encoder parameters

θ\\theta\(supervised pretraining on

𝒟0\\mathcal\{D\}\_\{0\}\)

3:Initialize bandit parameters

ϕ\\phi
4:for

k=1k=1to

KKdo

5:Phase A \(Bandit\):freeze

θ\\theta
6:for

t=1t=1to

TbanditT\_\{\\text\{bandit\}\}do

7:Observe

hth\_\{t\}, compute

zt=fθ​\(ht\)z\_\{t\}=f\_\{\\theta\}\(h\_\{t\}\), select

ata\_\{t\}via Eq\. \([7](https://arxiv.org/html/2607.09090#S3.E7)\)

8:Execute tier

ata\_\{t\}in

ℰ\\mathcal\{E\}
9:Observe

\(rt,yt\)\(r\_\{t\},y\_\{t\}\), append

\(ht,at,rt,yt\)\(h\_\{t\},a\_\{t\},r\_\{t\},y\_\{t\}\)to

𝒟k\\mathcal\{D\}\_\{k\}
10:Update

ϕ\\phiusing

\(zt,at,rt\)\(z\_\{t\},a\_\{t\},r\_\{t\}\)
11:endfor

12:Phase B \(Encoder\):freeze

ϕ\\phi
13:Construct labels

R~t\\tilde\{R\}\_\{t\}on

𝒟0∪𝒟1:k\\mathcal\{D\}\_\{0\}\\cup\\mathcal\{D\}\_\{1:k\}\(Eq\. \([28](https://arxiv.org/html/2607.09090#A2.E28)\)\)

14:for

u=1u=1to

TencT\_\{\\text\{enc\}\}do

15:Sample batches from

𝒟0∪𝒟1:k\\mathcal\{D\}\_\{0\}\\cup\\mathcal\{D\}\_\{1:k\}and update

θ\\thetausing Eq\. \([29](https://arxiv.org/html/2607.09090#A2.E29)\)

16:endfor

17:endfor

18:returnTiering policy

πθ,ϕ\\pi\_\{\\theta,\\phi\}

##### Phase B: Representation improvement with frozen bandit\.

We freeze the bandit head and update the encoder on aggregated trajectories \(offline logs plus interaction data\)\. Because the decision rule is fixed during this phase, the encoder learns to better predict experience\-aligned utility labels without introducing additional exploration\-induced non\-stationarity\.

Concretely, given the aggregated dataset𝒟≤k=𝒟0∪𝒟1∪⋯∪𝒟k\\mathcal\{D\}\_\{\\leq k\}=\\mathcal\{D\}\_\{0\}\\cup\\mathcal\{D\}\_\{1\}\\cup\\cdots\\cup\\mathcal\{D\}\_\{k\}, we optimize the encoder objective:

\(29\)minθ⁡𝔼\(ht,R~t\)∼𝒟≤k​\[ℓ​\(gθ​\(ht\),R~t\)\],\\min\_\{\\theta\}\\ \\mathbb\{E\}\_\{\(h\_\{t\},\\tilde\{R\}\_\{t\}\)\\sim\\mathcal\{D\}\_\{\\leq k\}\}\\big\[\\ell\(g\_\{\\theta\}\(h\_\{t\}\),\\tilde\{R\}\_\{t\}\)\\big\],whereℓ\\ellis a robust regression loss \(e\.g\., Huber\) andgθg\_\{\\theta\}is a prediction head on top of the Transformer encoder\. The overall training procedure is shown in Algorithm[1](https://arxiv.org/html/2607.09090#alg1)\.