SalesLoop: Reinforcement Learning from Performance Feedback for Sales Lead Ranking

# Reinforcement Learning from Performance Feedback for Sales Lead Ranking
Source: [https://arxiv.org/html/2607.20655](https://arxiv.org/html/2607.20655)
###### Abstract

Lead ranking in Customer Relationship Management \(CRM\) systems faces a persistent challenge: models achieving high offline accuracy often underperform in production\. We identify three fundamental gaps responsible for this disconnect: offline\-online metric mismatch, pointwise\-listwise objective misalignment, and temporal distribution drift\. To address these gaps, we propose SalesLoop, a reinforcement learning framework that establishes a closed feedback loop between model predictions and real\-world business outcomes\. Our approach introduces \(1\) a performance\-aware reward that encodes conversion outcomes weighted by ranking position and conversion velocity, and \(2\) Discriminative GRPO, a listwise optimization objective that adapts Group Relative Policy Optimization to discriminative ranking models\.

SalesLoop improves NDCG@K by \+7\.9% and P@K by \+15\.8% over the strongest static baseline\. A 160\-day production A/B test at a New Energy Vehicle manufacturer, spanning 16\.5M leads and 280 sales specialists across two provincial markets, validates statistically significant cumulative lift of \+4\.7% \(p=0\.047p=0\.047\) and \+8\.7% \(p=0\.002p=0\.002\)\. In production, the ranking backbone achieves Top\-10% recall of 44\.1% and surfaces high\-intent leads at2\.3×2\.3\\timesthe conversion rate of specialist baselines\.

## 1Introduction

Lead ranking—prioritizing sales leads by conversion likelihood—is a cornerstone of modern Customer Relationship Management \(CRM\)\[[33](https://arxiv.org/html/2607.20655#bib.bib1),[21](https://arxiv.org/html/2607.20655#bib.bib2)\]\. In high\-value industries such as automotive sales, real estate, and enterprise B2B, the quality of lead prioritization directly determines sales efficiency and revenue\. Industrial CRM platforms routinely process millions of leads daily, but limited sales capacity means that follow\-up effort must be prioritized: only the top\-ranked leads \(e\.g\., the daily Top\-10,000\) are surfaced to specialists for timely follow\-up\. Because the highest\-priority effort is concentrated on these top\-ranked leads, how well the model ranks them directly shapes how many conversions the sales team realizes—and thus revenue\.

Despite significant advances in deep learning\[[8](https://arxiv.org/html/2607.20655#bib.bib3),[31](https://arxiv.org/html/2607.20655#bib.bib4)\]and the recent emergence of Large Language Models \(LLMs\) for CRM tasks\[[10](https://arxiv.org/html/2607.20655#bib.bib16)\], a persistent gap remains between model development and real\-world deployment\. In our experience deploying lead ranking systems at a major New Energy Vehicle manufacturer, we observe that models achieving high offline metrics frequently underperform in production\. We identify three fundamental gaps that existing approaches fail to address:

##### Gap 1: Offline\-Online Metric Mismatch\.

Offline model evaluation optimizes for prediction accuracy \(e\.g\., AUC on binary conversion labels\), but the online business outcome depends on a fundamentally different set of factors: theeffectiveness of sales follow\-up\. In practice, conversion is not a passive event that a model merely predicts—it is an outcome jointly determined by lead qualityandthe intensity of human effort \(follow\-up frequency, call duration, engagement depth\)\. This creates a critical evaluation mismatch: an offline metric can improve while the corresponding online outcome stagnates or even deteriorates\. An analogous phenomenon has been documented in multi\-objective recommendation systems, where offline AUC improvements for both watch time and user interaction coexist, yet online deployment yields increased watch timealongsidedecreased interaction\[[35](https://arxiv.org/html/2607.20655#bib.bib32)\]\. The subtlety is that historical conversion labels are themselves confounded by human effort: a lead converts partly because it was promising and partly because a specialist invested time in it\. A model trained on these labels therefore learns a mixture ofintentandeffort, and can raise offline AUC by favoring leads that are “easier to follow up” \(e\.g\., responsive but low\-intent\) over leads that need more effort but carry higher latent intent\. The result: offline AUC increases, but online lock\-in rate does not, because the model’s ranking no longer aligns with the effective deployment of limited sales capacity\.

##### Gap 2: Pointwise Prediction≠\\neqListwise Ranking Quality\.

Standard training objectives optimize pointwise accuracy: minimizing prediction error for individual leads\. However, business value depends entirely onlistwiseranking quality: whether therightleads appear within the operational Top\-KKwindow\. These two objectives are not equivalent\. A model may achieve excellent pointwise accuracy across the full lead population, yet still rank true conversions at positions 11K–20K—just outside the Top\-10K operational window where sales capacity is allocated\. In this scenario, the model’s pointwise performance is strong, but its business impact is zero, because the leads that actually convert are not surfaced to sales teams\. Conversely, a model with modest pointwise accuracy but strong Top\-KKconcentration of conversions delivers substantially higher business value\. Pointwise\-optimal models are not necessarily listwise\-optimal, and optimizing for the former does not guarantee improvement in the latter\.

##### Gap 3: Temporal Distribution Drift\.

Customer behaviors, market conditions, and sales strategies continuously evolve over time\[[7](https://arxiv.org/html/2607.20655#bib.bib14)\]\. Promotional campaigns, seasonal variations, and competitive dynamics cause the lead population distribution to shift, yet static models trained on historical data remain fixed\. In automotive sales, the Chinese New Year promotional period alone can shift lead volume by 40–60%, with qualitatively different conversion patterns \(e\.g\., higher volume of casual browsers, lower immediate intent\)\. A model that performed well in one quarter may become progressively misaligned a few months later because the underlying distribution of incoming leads has changed\. Traditional training pipelines lack mechanisms for continuous adaptation to these temporal shifts, creating a growing gap between what the model was trained on and what it encounters in production\.

A common thread runs through these three gaps: models are trained once on historical data but must operate in a continuously evolving deployment environment\. Addressing them calls for a system that adapts to feedback from production rather than relying solely on historical signals\. Inspired by recent advances in Reinforcement Learning from Performance Feedback \(RLPF\) for ad optimization\[[13](https://arxiv.org/html/2607.20655#bib.bib19)\]and Group Relative Policy Optimization \(GRPO\) for language models\[[28](https://arxiv.org/html/2607.20655#bib.bib20)\], we proposeSalesLoop, a reinforcement learning framework that closes the feedback loop between model deployment and business outcomes\.

The reward signal comes directly from what deployment reveals: which leads actually converted, how quickly, and at what rank the model had placed them\.SalesLoopturns this deployment feedback into a continuous learning signal:

Deploy→\\rightarrowObserve→\\rightarrowReward→\\rightarrowUpdate→\\rightarrowDeploy

Our approach introduces two key innovations:

\(1\) Aperformance\-aware rewardthat encodes conversion outcomes weighted by ranking position \(following the logarithmic attention decay model\[[6](https://arxiv.org/html/2607.20655#bib.bib33)\]\) and conversion velocity \(faster conversions indicate stronger intent\)\. Unlike binary conversion labels, this reward differentiates among converted leads and directly encodes ranking quality from deployment\.

\(2\) Alistwise optimization objectivethat adapts GRPO to discriminative ranking models—a setting we termDiscriminative GRPO\. By treating each training batch as a group and computing relative advantages across leads, we optimize the ranking distribution rather than individual predictions, directly addressing Gap 2\.

We evaluateSalesLoopthrough a multi\-stage protocol at a New Energy Vehicle manufacturer, anchored by a live 160\-day production deployment spanning 16\.5M leads and 280 sales specialists:

- •Offline benchmarking:SalesLoopachieves significant improvements over traditional ML \(XGBoost, DeepFM\) and LLM\-based baselines \(SFT, DPO\), with the largest gains on ranking\-sensitive metrics \(NDCG@K: \+7\.9%, P@K: \+15\.8% over the strongest static baseline\)\.
- •Production A/B test: A 160\-day deployment across two provincial markets \(280 specialists\) validates \+4\.7% and \+8\.7% cumulative lift in lock\-in conversions \(p<0\.05p<0\.05\), with the advantage growing as the feedback loop accumulates deployment data\.
- •Deployment validation: In production over 103 days, the ranking backbone attains Top\-10% recall of 44\.1% \(4\.4×\\timesrandom\) and surfaces incremental high\-intent leads at 2\.3×\\timesthe conversion rate of specialist baselines, quantifying absolute ranking quality and business value independent of the A/B comparison\.

Our contributions are threefold:

1. 1\.We characterize three gaps that cause offline\-strong lead ranking models to underperform in production—offline\-online metric mismatch, pointwise\-listwise objective misalignment, and temporal distribution drift—and propose closing them with a performance\-feedback loop rather than a static training pipeline\.
2. 2\.We instantiate this loop with two components: a performance\-aware reward that combines conversion outcomes with ranking position and conversion velocity, andDiscriminative GRPO, a listwise objective that adapts group\-relative advantages to discriminative ranking models\.
3. 3\.We validateSalesLoopin a 160\-day production A/B test over 16\.5M leads and 280 specialists across two markets—a long\-horizon, real\-world evaluation that is rare in online ranking—demonstrating statistically significant lift \(\+4\.7% to \+8\.7%,p<0\.05p<0\.05\)\.

## 2Related Work

##### Lead Scoring and CRM Analytics\.

Traditional lead scoring relies on rule\-based scorecards\[[33](https://arxiv.org/html/2607.20655#bib.bib1)\]or shallow machine learning models such as logistic regression and gradient boosting\[[4](https://arxiv.org/html/2607.20655#bib.bib9)\]\. Recent work explores deep learning for CRM and click\-through prediction, including Wide & Deep\[[5](https://arxiv.org/html/2607.20655#bib.bib6)\], DeepFM\[[8](https://arxiv.org/html/2607.20655#bib.bib3)\], xDeepFM\[[14](https://arxiv.org/html/2607.20655#bib.bib7)\], AutoInt\[[29](https://arxiv.org/html/2607.20655#bib.bib8)\], and DCN and its successor\[[31](https://arxiv.org/html/2607.20655#bib.bib4),[32](https://arxiv.org/html/2607.20655#bib.bib5)\]for modeling feature interactions\. CRMArena\[[10](https://arxiv.org/html/2607.20655#bib.bib16)\]benchmarks LLMs on CRM tasks including lead qualification\. SalesRLAgent\[[17](https://arxiv.org/html/2607.20655#bib.bib17)\]applies RL to real\-time conversion prediction in sales conversations\. Sun et al\.\[[30](https://arxiv.org/html/2607.20655#bib.bib35)\]propose asLLR, integrating CTR and QA losses within a decoder\-only LLM for automotive lead ranking, with gains validated in A/B testing\. HPRO\[[37](https://arxiv.org/html/2607.20655#bib.bib18)\]scores leads with an LLM under a hierarchical preference\-ranking objective, but is trained once on historical data and deployed as a static model\.SalesLoopinstead closes the loop between deployment and training, continuously adapting the ranking model from observed conversion outcomes\.

##### Learning to Rank\.

Learning\-to\-rank methods are categorized into pointwise\[[15](https://arxiv.org/html/2607.20655#bib.bib12)\], pairwise\[[1](https://arxiv.org/html/2607.20655#bib.bib13)\], and listwise\[[2](https://arxiv.org/html/2607.20655#bib.bib10),[34](https://arxiv.org/html/2607.20655#bib.bib11)\]approaches\. Pointwise methods treat ranking as regression or classification on individual items\. Pairwise methods optimize relative orderings between item pairs\. Listwise methods directly optimize ranking quality over the entire list\. While listwise approaches better align with ranking objectives, most operate in offline settings with static relevance labels\. Online learning\-to\-rank\[[18](https://arxiv.org/html/2607.20655#bib.bib25),[12](https://arxiv.org/html/2607.20655#bib.bib26)\]adapts rankings from user feedback, but assumes immediate click signals rather than the delayed, sparse conversion outcomes with long feedback cycles \(30\+ days\) that characterize lead ranking\.

##### Reinforcement Learning for Ranking and Recommendation\.

RL has been applied to ranking and recommendation tasks\[[11](https://arxiv.org/html/2607.20655#bib.bib21),[3](https://arxiv.org/html/2607.20655#bib.bib22)\]\. RLHF\[[19](https://arxiv.org/html/2607.20655#bib.bib23)\]aligns LLMs using human feedback as a reward signal, with PPO\[[26](https://arxiv.org/html/2607.20655#bib.bib36)\]as the standard policy\-optimization algorithm\. Xue et al\.\[[36](https://arxiv.org/html/2607.20655#bib.bib37)\]apply RL to adaptive user retention in recommender systems, showing that online RL can outperform static scoring in production; their reward, however, is driven by near\-immediate user\-retention signals, whereas lead ranking must learn from conversions that materialize weeks later\. Most relevant to our work, Jiang et al\.\[[13](https://arxiv.org/html/2607.20655#bib.bib19)\]introduce Reinforcement Learning from Performance Feedback \(RLPF\) for ad text generation at Meta, using a CTR prediction model as the reward signal and reporting consistent CTR gains in a large\-scale production A/B test\. This line of work establishes that real\-world performance metrics can serve as effective reward signals, bypassing the need for human annotation\. Their setting, however, isgenerative—optimizing token\-level generation probabilities against an immediate CTR proxy\. GRPO\[[28](https://arxiv.org/html/2607.20655#bib.bib20)\]likewise operates on generative models, introducing group\-relative advantages that remove the need for a learned value network\.SalesLoopadapts these insights to adiscriminativeranking model underdelayed, sparserewards, proposing a listwise variant that computes group\-relative advantages over lead scores rather than token\-level generation probabilities\.

##### Online and Continual Learning\.

Online learning\[[27](https://arxiv.org/html/2607.20655#bib.bib27)\]updates models incrementally as data arrives\. Continual learning\[[20](https://arxiv.org/html/2607.20655#bib.bib28)\]addresses catastrophic forgetting when learning from non\-stationary distributions\. Concept drift detection\[[7](https://arxiv.org/html/2607.20655#bib.bib14)\]identifies distribution shifts requiring model updates\. Recent work on unbiased learning to rank\[[25](https://arxiv.org/html/2607.20655#bib.bib15),[38](https://arxiv.org/html/2607.20655#bib.bib38)\]highlights the importance of disentangling relevance from position and selection bias in deployed ranking systems\. While these methods provide theoretical foundations, they rarely address the specific challenges of lead ranking: extremely delayed feedback \(30\+ days\), sparse positive labels \(<<2% conversion rate\), and the need to optimize listwise business metrics rather than pointwise accuracy\.

Taken together, prior work leaves a distinct gap: static offline LTR does not adapt after deployment; online LTR assumes immediate feedback; and performance\-feedback RL has so far targeted generative models with near\-term proxies\. None addresses discriminative, listwise ranking that must learn from delayed and sparse business outcomes—the regimeSalesLoopis designed for\.

## 3Methodology

![Refer to caption](https://arxiv.org/html/2607.20655v1/figures/salesloop3.png)Figure 1:Overview of theSalesLoopframework\. In each monthly iteration, the deployed ranking model scores all leads and selects Top\-KKcandidates for sales follow\-up\. After the 30\-day conversion window, observed outcomes are converted into performance\-aware rewards, which are used to update the model through a Discriminative GRPO objective\. The updated model is then deployed in the next iteration, forming a closed online feedback loop\.### 3\.1Problem Formulation

Let𝒳\\mathcal\{X\}denote the feature space and𝒴=\{0,1\}\\mathcal\{Y\}=\\\{0,1\\\}the binary conversion outcomes\. At iterationtt, we observe a deployment dataset𝒟t=\{\(𝐱i,yi,ri,τi\)\}i=1Nt\\mathcal\{D\}\_\{t\}=\\\{\(\\mathbf\{x\}\_\{i\},y\_\{i\},r\_\{i\},\\tau\_\{i\}\)\\\}\_\{i=1\}^\{N\_\{t\}\}, where:

- •𝐱i∈𝒳\\mathbf\{x\}\_\{i\}\\in\\mathcal\{X\}: lead features \(tabular attributes \+ dialogue transcripts\),
- •yi∈𝒴y\_\{i\}\\in\\mathcal\{Y\}: binary conversion indicator \(1 if locked\-in withinTTdays\),
- •ri∈\{1,…,Nt\}r\_\{i\}\\in\\\{1,\\ldots,N\_\{t\}\\\}: rank assigned by the deployed modelfθt−1f\_\{\\theta\_\{t\-1\}\},
- •τi∈\[0,T\]\\tau\_\{i\}\\in\[0,T\]: conversion latency in days from ranking to lock\-in, defined only foryi=1y\_\{i\}=1\.

Objective\.Learn a scoring functionfθ:𝒳→ℝf\_\{\\theta\}:\\mathcal\{X\}\\to\\mathbb\{R\}that maximizes business\-relevant ranking metrics within the operational capacityKK:

maxθ⁡𝔼\(𝐱,y\)∼pt​\[Precision@​K\+λ⋅Recall@​K\],\\max\_\{\\theta\}\\;\\mathbb\{E\}\_\{\(\\mathbf\{x\},y\)\\sim p\_\{t\}\}\\left\[\\text\{Precision@\}K\+\\lambda\\cdot\\text\{Recall@\}K\\right\],\(1\)whereKKis the operational capacity \(e\.g\.,K=10,000K=10\{,\}000leads for daily follow\-up\) andλ\\lambdaexpresses the precision\-recall trade\-off inherent in the business goal\. Eq\.[1](https://arxiv.org/html/2607.20655#S3.E1)states this goal conceptually; since Precision@KKand Recall@KKare non\-differentiable, we do not optimize it directly \(and thus do not tuneλ\\lambdaexplicitly\)\. Instead, our position\-weighted listwise objective \(§[3\.4](https://arxiv.org/html/2607.20655#S3.SS4)\) serves as a differentiable surrogate that concentrates converted leads near the top of the ranking, which is precisely what Top\-KKmetrics reward\. The key challenge is thatptp\_\{t\}—the deployment distribution at timett—differs from historical training distributions and evolves over time due to market dynamics and seasonal patterns\.

### 3\.2SalesLoop Framework

SalesLoopoperates as a closed\-loop system with three stages per iteration \(Figure[1](https://arxiv.org/html/2607.20655#S3.F1)\):

Stage 1: Deploy\.Modelfθtf\_\{\\theta\_\{t\}\}scores all incoming leads and produces a ranking\. The Top\-KKleads are selected for sales follow\-up based on operational capacity constraints\.

Stage 2: Collect\.After the conversion windowTT\(30 days\), we observe real\-world outcomes: which leads converted \(yiy\_\{i\}\) and how quickly \(τi\\tau\_\{i\}\)\. These outcomes are transformed into performance\-aware rewardsRiR\_\{i\}\(§[3\.3](https://arxiv.org/html/2607.20655#S3.SS3)\)\.

Stage 3: Update\.The model is optimized via a listwise loss function that aligns predicted scores with observed rewards \(§[3\.4](https://arxiv.org/html/2607.20655#S3.SS4)\)\. Training continues fromfθtf\_\{\\theta\_\{t\}\}\(warm start\) with a reduced learning rate to preserve previously learned representations while adapting to new patterns\.

This closed\-loop cycle repeats monthly, enabling the model to continuously adapt to the evolving deployment distribution\. A critical design choice is the warm\-start strategy: rather than training from scratch, each iteration initializes from the previous model’s parameters, providing a strong prior that accelerates convergence while the reduced learning rate and regularization keep the model close to previously learned patterns\. This design enablesSalesLoopto accumulate knowledge across iterations, creating a compounding effect where each iteration benefits from all prior deployment experience\.

### 3\.3Performance\-Aware Reward Design

The reward design is central toSalesLoop’s ability to optimize ranking quality rather than pointwise accuracy\. We construct rewards that encode three business\-relevant signals: the conversion outcome \(yiy\_\{i\}\), the ranking position at which the lead was surfaced \(rir\_\{i\}\), and the speed of conversion \(τi\\tau\_\{i\}\)\.

###### Definition 1\(Performance\-Aware Reward\)\.

For leadiiwith conversion outcomeyi∈\{0,1\}y\_\{i\}\\in\\\{0,1\\\}, assigned rankrir\_\{i\}, and conversion latencyτi\\tau\_\{i\}:

Ri=yi⋅g​\(ri\)⋅v​\(τi\),R\_\{i\}=y\_\{i\}\\cdot g\(r\_\{i\}\)\\cdot v\(\\tau\_\{i\}\),\(2\)where the position gaing​\(ri\)g\(r\_\{i\}\)and velocity bonusv​\(τi\)v\(\\tau\_\{i\}\)are defined as:

g​\(ri\)=1log2⁡\(ri\+1\),v​\(τi\)=1−τiT\.g\(r\_\{i\}\)=\\frac\{1\}\{\\log\_\{2\}\(r\_\{i\}\+1\)\},\\qquad v\(\\tau\_\{i\}\)=1\-\\frac\{\\tau\_\{i\}\}\{T\}\.\(3\)The leading factoryiy\_\{i\}ensuresRi=0R\_\{i\}=0for non\-converted leads, sov​\(τi\)v\(\\tau\_\{i\}\)only needs to be defined for converted leads, whereτi∈\[0,T\]\\tau\_\{i\}\\in\[0,T\]\.

Position gain\.We adopt logarithmic decay following the position\-based click model\[[6](https://arxiv.org/html/2607.20655#bib.bib33)\], which assumes user \(sales\) attention decays logarithmically with rank\. This choice is motivated by our operational setting: a conversion at rank 1 is substantially more valuable than at rank 10,000, as the former represents a correctly prioritized lead while the latter indicates a missed opportunity\. A conversion at rank 1 yieldsg​\(1\)=1\.0g\(1\)=1\.0; at rank 1,000,g​\(1000\)≈0\.1g\(1000\)\\approx 0\.1\.

Velocity bonus\.Faster conversions indicate stronger purchase intent\. We use linear decay over the conversion windowTT: leads that convert on day 1 receive the maximum velocity bonus \(v=1v=1\), while those converting on day 30 receive zero \(v=0v=0\)\. This simple, interpretable signal differentiates among converted leads—something binary labels cannot do\.

Reward properties\.The rewardRiR\_\{i\}is zero for non\-converted leads and strictly positive for converted leads, with magnitude determined by ranking position and conversion speed\. This design ensures that: \(1\) the model receives no credit for non\-conversions, \(2\) correctly ranking high\-value leads \(early in the list\) yields higher rewards, and \(3\) faster conversions are preferentially rewarded, encouraging the model to identify leads with immediate purchase intent\.

### 3\.4Discriminative GRPO for Ranking

We adapt Group Relative Policy Optimization \(GRPO\)\[[28](https://arxiv.org/html/2607.20655#bib.bib20)\]—originally designed for generative language models—todiscriminative ranking models\. This adaptation, which we termDiscriminative GRPO, preserves the core benefit of GRPO \(variance reduction through group\-normalized advantages\) while enabling its application to scoring functions\.

##### From generative to discriminative\.

In the original GRPO framework, the model generates multiple candidate responses for each prompt, and advantages are computed across candidates within each prompt group\. In our discriminative setting, the model outputs a scalar scoresi=fθ​\(𝐱i\)s\_\{i\}=f\_\{\\theta\}\(\\mathbf\{x\}\_\{i\}\)for each lead\. We treat each training batch as a “group” and compute relative advantages across leads within the batch\. This parallels the generative case: instead of comparing different text outputs for the same input, we compare different lead scores within the same batch\.

##### Scope of the adaptation\.

We emphasize which parts of GRPO we adopt and which we do not\. We inherit GRPO’s central idea—replacing a learned value baseline withgroup\-relative, within\-batch advantage normalization—which is what stabilizes learning under our sparse rewards\. We deliberately donotcarry over the components tied to autoregressive generation: there is no importance\-sampling ratioπθ/πθold\\pi\_\{\\theta\}/\\pi\_\{\\theta\_\{\\text\{old\}\}\}and no PPO\-style probability\-ratio clipping\. These are unnecessary here because a discriminative ranker produces a single scalar per lead rather than a sampled token sequence, so there is no per\-token, off\-policy credit\-assignment problem to correct\. Instead, we regulate policy drift across iterations through the explicit BCE regularization of Eq\.[7](https://arxiv.org/html/2607.20655#S3.E7)and the conservative warm\-start learning rate described in the implementation details\. We therefore view Discriminative GRPO as a group\-relative, reward\-weighted listwise objective in the spirit of GRPO, rather than a literal transplant of the PPO\-based algorithm\.

##### Listwise optimization\.

For a training batchℬ=\{\(𝐱i,Ri\)\}i=1B\\mathcal\{B\}=\\\{\(\\mathbf\{x\}\_\{i\},R\_\{i\}\)\\\}\_\{i=1\}^\{B\}:

Step 1: Compute model scores\.

si=fθ​\(𝐱i\),i=1,…,B\.s\_\{i\}=f\_\{\\theta\}\(\\mathbf\{x\}\_\{i\}\),\\quad i=1,\\ldots,B\.\(4\)
Step 2: Compute group\-relative advantages\.Standardize rewards within batch to obtain relative advantages:

Ai=Ri−R¯σR\+ϵ,R¯=1B​∑j=1BRj,σR=1B​∑j=1B\(Rj−R¯\)2,A\_\{i\}=\\frac\{R\_\{i\}\-\\bar\{R\}\}\{\\sigma\_\{R\}\+\\epsilon\},\\quad\\bar\{R\}=\\frac\{1\}\{B\}\\sum\_\{j=1\}^\{B\}R\_\{j\},\\quad\\sigma\_\{R\}=\\sqrt\{\\frac\{1\}\{B\}\\sum\_\{j=1\}^\{B\}\(R\_\{j\}\-\\bar\{R\}\)^\{2\}\},\(5\)whereϵ=10−8\\epsilon=10^\{\-8\}for numerical stability\. This standardization removes the batch\-level mean and scales by variance, analogous to GRPO’s group normalization\. In sparse\-reward settings \(∼\\sim1\.5% conversion rate in our deployment\), this reduces variance by centering around the batch average rather than an absolute baseline, which is critical for stable gradient estimation when the majority of labels are zero\.

Step 3: Align distributions via KL divergence\.We optimize the score distribution to match the advantage distribution:

ℒlist=DKL​\(softmax​\(𝐀/γ\)∥softmax​\(𝐬\)\),\\mathcal\{L\}\_\{\\text\{list\}\}=D\_\{\\text\{KL\}\}\\Big\(\\text\{softmax\}\(\\mathbf\{A\}/\\gamma\)\\;\\Big\\\|\\;\\text\{softmax\}\(\\mathbf\{s\}\)\\Big\),\(6\)whereγ\>0\\gamma\>0is a temperature controlling the sharpness of the target distribution\. The two operations are complementary rather than redundant: the standardization in Eq\.[5](https://arxiv.org/html/2607.20655#S3.E5)fixes thescaleof advantages across batches, whileγ\\gammacontrols how sharply probability mass concentrates on the highest\-reward leads\. Lowerγ\\gammacreates sharper targets that emphasize high\-reward leads; higherγ\\gammaproduces softer targets\. We setγ=0\.1\\gamma=0\.1based on sensitivity analysis \(§[4\.6](https://arxiv.org/html/2607.20655#S4.SS6)\)\.

This formulation is listwise because the softmax couples all leads in the batch: changing one lead’s score affects its relative probability mass, directly optimizing the ranking distribution rather than individual predictions\.

##### Relation to ListNet\.

The listwise loss in Eq\.[6](https://arxiv.org/html/2607.20655#S3.E6)is structurally related to ListNet\[[2](https://arxiv.org/html/2607.20655#bib.bib10)\], which also matches score and target distributions via a cross\-entropy/KL objective\. The key difference is the source of the target: ListNet uses static relevance labels, whereas our target distribution is built fromdynamicperformance\-aware rewards that encode real deployment feedback \(position and velocity\), combined with the group\-relative normalization inherited from GRPO\.

##### Training objective\.

We combine the listwise loss with pointwise regularization to prevent calibration drift:

ℒtotal=ℒlist\+α⋅ℒBCE​\(𝐬,𝐲\),\\mathcal\{L\}\_\{\\text\{total\}\}=\\mathcal\{L\}\_\{\\text\{list\}\}\+\\alpha\\cdot\\mathcal\{L\}\_\{\\text\{BCE\}\}\(\\mathbf\{s\},\\mathbf\{y\}\),\(7\)whereℒBCE\\mathcal\{L\}\_\{\\text\{BCE\}\}is binary cross\-entropy against conversion labels andα\\alphacontrols regularization strength\. The pointwise term anchors individual predictions to conversion labels, while the listwise term optimizes the relative ordering\. We setα=0\.5\\alpha=0\.5based on ablation studies\.

### 3\.5Implementation Details

##### Model architecture\.

We use Qwen2\.5\-1\.5B\[[23](https://arxiv.org/html/2607.20655#bib.bib29)\]as the language backbone for processing dialogue transcripts, augmented with a tabular encoder for structured features\. Both representations are fused via cross\-attention and projected to a scalar score through a linear head\. This architecture follows our prior work on HPRO\[[37](https://arxiv.org/html/2607.20655#bib.bib18)\]\.

##### Efficient fine\-tuning\.

We apply LoRA\[[9](https://arxiv.org/html/2607.20655#bib.bib30)\]with rankr=16r=16and scaling factorα=32\\alpha=32, targeting query and value projection layers\. The tabular encoder and fusion layers are fully trained\. Online iterations use a reduced learning rate of10−510^\{\-5\}\(10×\\timeslower than initial SFT\) with batch size 64 for 3 epochs\. This conservative update strategy serves two purposes: \(1\) it preserves the base model’s calibration while adapting to distributional shifts, and \(2\) it provides an implicit KL constraint between consecutive iterations, complementing the explicit regularization in Eq\.[7](https://arxiv.org/html/2607.20655#S3.E7)\.

##### Hyperparameters\.

Temperatureγ=0\.1\\gamma=0\.1, regularizationα=0\.5\\alpha=0\.5, conversion windowT=30T=30days\. Iteration frequency: monthly, aligned with the 30\-day conversion window\. Initial modelfθ0f\_\{\\theta\_\{0\}\}is trained via supervised fine\-tuning \(SFT\) on 9\.2M historical samples \(July–September 2025\) with binary cross\-entropy loss\. Optimization uses AdamW\[[16](https://arxiv.org/html/2607.20655#bib.bib31)\]with linear warmup \(5% of steps\) and cosine decay\.

## 4Experiments

We evaluateSalesLoopthrough a comprehensive experimental protocol spanning offline benchmarking, long\-term online deployment across two provincial markets, and detailed mechanism analysis\.

### 4\.1Experimental Setup

##### Datasets\.

We collect lead data from a leading New Energy Vehicle \(NEV\) manufacturer’s CRM system\. Our evaluation uses two datasets spanning different operational periods:

- •Historical Dataset\(9\.2M samples,∼\\sim1\.5% positive\): Collected from July–September 2025\. Used for initial model training and fair comparison with offline baselines\.
- •Online Deployment Dataset\(16\.5M samples,∼\\sim1\.5% positive\): Collected from December 2025–June 2026, representing a 160\-day production A/B test with delayed conversion feedback across two provincial markets\.

In addition, we use a 103\-day production log \(September 20 – December 31, 2025\) from the same CRM system to characterize the ranking quality of the deployed base model prior to the A/B test \(§[4\.5](https://arxiv.org/html/2607.20655#S4.SS5)\)\.

Each lead comprises 47 tabular features \(demographics, source channel, interaction history, behavioral signals\) and sales dialogue transcripts \(average 2,682 tokens\)\. The operational capacityK=10,000K=10\{,\}000represents the top\-ranked leads surfaced to sales specialists daily for prioritized follow\-up\. Conversion labels are determined by 30\-day lock\-in status\. Table[1](https://arxiv.org/html/2607.20655#S4.T1)summarizes statistics\.

Table 1:Dataset statistics for lead ranking evaluation\.StatisticValueTotal leads \(Historical\)9\.2MTotal leads \(Online, 160 days\)16\.5MTabular features47Avg\. dialogue tokens2,682Conversion windowTT30 daysOperational capacityKK10,000
##### Baselines\.

We compare against representative methods spanning traditional ML, deep learning, and LLM\-based approaches:

- •XGBoost\[[4](https://arxiv.org/html/2607.20655#bib.bib9)\]: Gradient boosting on tabular features only, representing the traditional industrial baseline\.
- •DeepFM\[[8](https://arxiv.org/html/2607.20655#bib.bib3)\]: Deep factorization machine capturing feature interactions, representing deep CTR models\.
- •LLM\+SFT: Qwen2\.5\-1\.5B fine\-tuned on historical conversion labels with binary cross\-entropy loss\. This serves as our base modelfθ0f\_\{\\theta\_\{0\}\}and thecontrol groupin online A/B testing\.
- •LLM\+DPO\[[24](https://arxiv.org/html/2607.20655#bib.bib24)\]: Direct Preference Optimization using converted vs\. non\-converted lead pairs, representing static preference\-based methods\.

##### Metrics\.

We report standard ranking metrics atK=10,000K=10\{,\}000:

- •Precision@K \(P@K\): Fraction of Top\-KKleads that convert \(lock\-in rate within Top\-KK\)\.
- •Recall@K \(R@K\): Fraction of all conversions captured in Top\-KK\.
- •NDCG@K: Normalized discounted cumulative gain\.
- •AUC: Area under ROC curve for pointwise prediction accuracy\.

##### Implementation\.

All LLM\-based methods share the same architecture: Qwen2\.5\-1\.5B with LoRA \(r=16r=16,α=32\\alpha=32\) for efficient fine\-tuning, targeting query and value projections\. The tabular encoder and cross\-attention fusion layers are fully trained\. Training uses AdamW with learning rate10−510^\{\-5\}, batch size 64, for 3 epochs per iteration on 8×\\timesA100 GPUs\.SalesLoopperforms monthly online iterations on the deployment dataset, using the previous month’s conversion outcomes as reward signals\.

##### Online A/B Test Configuration\.

The production A/B test was conducted simultaneously in two anonymized provincial markets with the following configuration:

- •Duration: 160 days \(December 26, 2025 – June 4, 2026\)\.
- •Province A: 60 treatment specialists \(SalesLoop\) vs\. 60 control specialists \(LLM\+SFT\), randomly assigned\.
- •Province B: 80 treatment specialists \(SalesLoop\) vs\. 80 control specialists \(LLM\+SFT\), randomly assigned\.
- •Primary metric: Cumulative lock\-in conversions and per\-specialist productivity\.

The control group uses the static LLM\+SFT model \(fθ0f\_\{\\theta\_\{0\}\}\) throughout the entire test period without any parameter updates\. The treatment group usesSalesLoop, which is initialized from the samefθ0f\_\{\\theta\_\{0\}\}but undergoes monthly online iterations using deployment feedback\.

### 4\.2Offline Benchmarking

We first evaluate all methods under a controlled offline setting using the Historical Dataset, ensuring fair comparison with consistent data splits and temporal ordering\. Table[2](https://arxiv.org/html/2607.20655#S4.T2)presents results\.

Table 2:Offline results on Historical Dataset \(K=10,000K=10\{,\}000\)\. For a controlled, single\-round comparison with the static baselines,SalesLoopis evaluated after 1 iteration using held\-out feedback; the multi\-iteration behavior is analyzed separately in the ablation \(Table[5](https://arxiv.org/html/2607.20655#S4.T5)\) and online study\. Best inbold, secondunderlined\.##### LLM backbones substantially outperform traditional models\.

LLM\+SFT achieves \+40\.9% P@K over XGBoost \(6\.37% vs\. 4\.52%\), demonstrating the value of dialogue understanding\. This aligns with our hypothesis: sales conversations contain rich intent signals that tabular features alone cannot capture\. However, the AUC gap between LLM\+SFT and DeepFM \(0\.781 vs\. 0\.739\) is notably smaller than the P@K gap \(\+26\.4%\), suggesting that AUC alone underestimates the business value of LLM\-based ranking—precisely because of the pointwise\-listwise gap we identified \(Gap 2\)\.

##### DPO does not uniformly improve over SFT\.

LLM\+DPO achieves slightly higher P@K and NDCG@K than LLM\+SFT \(\+2\.5% and \+1\.2% respectively\), but its AUC islower\(0\.776 vs\. 0\.781\) and MRR also drops\. This suggests that DPO’s pairwise preference objective can hurt pointwise calibration while improving top\-KKdiscrimination—a known trade\-off between ranking and classification objectives\[[24](https://arxiv.org/html/2607.20655#bib.bib24)\]\.

##### SalesLoopachieves the best performance across all metrics\.

Compared to the better of LLM\+SFT and LLM\+DPO on each metric,SalesLoopimproves P@K by \+15\.8% and NDCG@K by \+7\.9%, while also recovering AUC to 0\.789\. The consistent gains across both ranking\-sensitive and pointwise metrics validate that our listwise optimization with performance\-aware rewards effectively addresses Gap 2 without sacrificing calibration\.

### 4\.3Online Deployment Results

We now present the core empirical contribution of this work: a 160\-day production A/B test across two provincial markets\. To our knowledge, this is among the longest continuously\-running online RL evaluations reported for industrial lead ranking\.

#### 4\.3\.1Overall Results

Table[3](https://arxiv.org/html/2607.20655#S4.T3)summarizes the end\-of\-period results\.

Table 3:Online A/B test results after 160 days of deployment\.SalesLoopachieves statistically significant improvements in both markets: \+4\.7% cumulative lift in Province A \(p=0\.047p=0\.047, weekly pairedtt\-test\) and \+8\.7% in Province B \(p=0\.002p=0\.002\)\. The effect in Province B is strongly significant, while Province A, though significant at the 0\.05 level, is more marginal—consistent with its longer warm\-up period before the feedback loop takes effect \(§[4\.3\.2](https://arxiv.org/html/2607.20655#S4.SS3.SSS2)\)\. In per\-specialist terms, each sales specialist in the treatment group converts 1\.6–2\.7 additional leads over the 160\-day period compared to their control counterparts\. Because each conversion corresponds to a high\-value durable good, these per\-specialist gains translate into substantial revenue impact at team scale\.

#### 4\.3\.2Temporal Dynamics: The Feedback Loop Effect

Beyond the end\-of\-period totals, thetemporal dynamicsof the treatment\-control gap provide direct evidence for the feedback loop mechanism\. Figure[2](https://arxiv.org/html/2607.20655#S4.F2)shows cumulative conversion curves for both provinces\.

![Refer to caption](https://arxiv.org/html/2607.20655v1/x1.png)Figure 2:Cumulative lock\-in conversions over 160 days\. Green shading indicates periods where treatment outperforms control; red shading indicates the reverse\. Vertical dashed lines mark the crossover point whereSalesLoopbegins to persistently outperform the static baseline\.As deployment feedback accumulates,SalesLoopcrosses over the static baseline and steadily widens the gap in both markets\. The cumulative difference grows from 0 to \+96 \(Province A\) and \+212 \(Province B\) by the end of the observation period, and once established the divergence does not reverse within our observation window—suggesting that the feedback loop yields a compounding advantage rather than a one\-time boost\. The onset is fast: Province B turns positive within weeks, and the effect strengthens over successive iterations\. The main exception is an initial warm\-up in Province A, whereSalesLooptrails the control over roughly the first 70 days \(−5\.9%\-5\.9\\%on average\) before the loop takes effect—a one\-time, first\-iteration transient that can be avoided by warming up the initial iteration offline\. This difference in time\-to\-effect likely reflects market characteristics: Province B has a higher daily lead volume relative to team size, providing denser reward signals, which suggests thatSalesLoop’s feedback loop efficiency is modulated by theinformation densityof the deployment environment\.

#### 4\.3\.3Month\-over\-Month Analysis

Figure[3](https://arxiv.org/html/2607.20655#S4.F3)presents the monthly relative lift, providing a granular view of the feedback loop’s temporal evolution\.

![Refer to caption](https://arxiv.org/html/2607.20655v1/x2.png)Figure 3:Monthly conversion lift \(%\) of treatment over control\. Province B turns positive almost immediately and stays there, while Province A transitions from a neutral warm\-up phase to consistently positive lift\.Table 4:Monthly conversion breakdown\. Values are new conversions per month\. June 2025 is partial \(through June 4, the end of the 160\-day test window\), hence the lower counts\.Table[4](https://arxiv.org/html/2607.20655#S4.T4)reveals several important patterns:

- •Warm\-up duration varies by market: Province A’s warm\-up persists through February \(three months of neutral\-to\-slightly\-negative lift\), while Province B reaches positive lift as early as January \(\+12\.5%\) despite a brief dip in February\.
- •Lift magnitude grows over time: Both provinces show their strongest lifts in the later months \(May–June\), confirming the compounding nature of the feedback loop\. As each iteration improves ranking quality, subsequent iterations benefit from higher\-quality deployment data\.
- •Non\-monotonic but trending positive: The month\-to\-month lift is not perfectly monotonic \(e\.g\., Province A shows \+13\.9% in March but only \+0\.4% in April\), reflecting natural market fluctuations and the monthly granularity of model updates\.

#### 4\.3\.4Cumulative Gap Analysis

Figure[4](https://arxiv.org/html/2607.20655#S4.F4)visualizes the treatment\-control gap over time for both provinces simultaneously, providing the clearest evidence of the feedback loop’s sustained effect\.

![Refer to caption](https://arxiv.org/html/2607.20655v1/x3.png)Figure 4:Cumulative conversion difference \(Treatment−\-Control\) over time\. Both curves show a clear regime change from negative/flat to persistently positive and growing\.Both provinces converge to a cleardivergence regimein which the gap grows approximately linearly \(Province A:∼\\sim1\.0 additional conversions/day; Province B:∼\\sim1\.4/day\), indicating stable per\-iteration improvement rather than diminishing returns within the observation window\. Province B enters this regime almost immediately, whereas Province A first passes through a brief warm\-up before the gap crosses zero and begins to grow\.

This roughly linear growth suggests that, within the observed horizon, the feedback loop has not yet saturated—further iterations would likely continue to improve performance, consistent with the argument for continuous online learning in non\-stationary environments\.

### 4\.4Ablation Study

We analyze the contribution of each component in Table[5](https://arxiv.org/html/2607.20655#S4.T5)\. All variants are evaluated offline after 4 iterations, so that each component’s effect is measured once the feedback loop has had several rounds to take effect\.

Table 5:Ablation study on reward and loss components \(offline evaluation after 4 iterations\)\.##### Listwise loss is the most critical component\.

Removingℒlist\\mathcal\{L\}\_\{\\text\{list\}\}causes the largest single\-component drop \(−8\.9%\-8\.9\\%P@K\), confirming that group\-relative optimization is essential\. The “Pointwise Only” variant, which uses online deployment data but trains with BCE loss alone, performs only marginally better than the SFT baseline \(6\.61% vs\. 6\.37%\)\. This demonstrates that access to online data alone is insufficient; the listwise objective with performance\-aware rewards is what drives the improvement\.

##### Position gain provides substantial benefit\.

Removingg​\(r\)g\(r\)reduces P@K by−5\.0%\-5\.0\\%\. Position feedback directly encodes ranking quality from deployment: the model learns that correctly ranking a conversion at position 50 is more valuable than at position 5,000\. This signal is unique to the online setting and unavailable in offline training\.

##### BCE regularization prevents calibration loss\.

Removing the BCE regularization term reduces P@K by−3\.6%\-3\.6\\%and notably drops AUC by−2\.1%\-2\.1\\%\(from 0\.789 to 0\.772\)\. This indicates that the listwise loss alone, while optimizing ranking quality, can cause the model to lose calibration on individual predictions\. The regularization anchors pointwise predictions while the listwise term optimizes relative ordering\.

##### Velocity provides incremental gains\.

Removingv​\(τ\)v\(\\tau\)reduces P@K by−2\.0%\-2\.0\\%\. Faster conversions signal stronger purchase intent; this auxiliary signal refines the model’s quality estimation beyond binary conversion labels\.

### 4\.5Ranking Quality Validation with Deployment Metrics

The A/B test in §[4\.3](https://arxiv.org/html/2607.20655#S4.SS3)quantifies therelativegain of the online feedback loop over a static baseline\. Here we complement it with anabsoluteassessment of ranking quality and business value in production\. Concretely, we analyze the base rankerfθ0f\_\{\\theta\_\{0\}\}—the supervised backbone from whichSalesLoopis initialized, prior to any online iteration—using production data from 103 days of operation \(September 20 – December 31, 2025\)\. This isolates the contribution of the ranking architecture itself: any lift the online loop adds \(§[4\.3](https://arxiv.org/html/2607.20655#S4.SS3)\) is stackedon top ofthe strong absolute performance established here\.

##### Top\-10% Recall Analysis\.

We evaluate how effectively the model’s top\-ranked leads capture actual conversions\. Specifically, we compute the 7\-day and 14\-day recall of the Top\-10% ranked leads \(i\.e\., the fraction of all conversions that fall within the model’s top decile\)\. Table[6](https://arxiv.org/html/2607.20655#S4.T6)reports summary statistics\.

Table 6:Top\-10% recall rates of the base rankerfθ0f\_\{\\theta\_\{0\}\}over 103 days of production operation\. Recall = conversions in Top\-10% / total conversions\.The model consistently captures∼\\sim44% of 7\-day conversions and∼\\sim41% of 14\-day conversions within just the top decile of ranked leads\. This demonstrates strong ranking quality even before the online loop is engaged: a random ranker would achieve 10% recall at the 10% cutoff, while the base ranker achieves 4\.4×\\timeslift over random for 7\-day conversions\.

##### Incremental High\-Intent Identification\.

A key business application ofSalesLoopis identifying high\-intent leads that sales specialists have not yet recognized\. We defineincremental leadsas those ranked high\-intent by the model butnotpreviously flagged by specialists as likely\-to\-convert \(3/7\-day intent markers\)\.

Over the same 103\-day production window:

- •The model identifies∼\\sim4,600 incremental high\-intent leads per day\.
- •These incremental leads achieve a14\-day conversion rate of 4\.66%, compared to 2\.05% for the specialist\-flagged baseline—a2\.3×\\timesimprovement\.
- •Cumulative incremental conversions attributed to the model:7,220 lock\-ins\.

These 7,220 lock\-ins are especially consequential in our setting: each conversion corresponds to a high\-value durable good, so incremental lock\-ins that would otherwise have been overlooked translate directly into substantial revenue\. More broadly, this validates thatSalesLoop’s ranking captures genuine purchase intent signals from dialogue understanding that complement human judgment, rather than merely replicating existing specialist labels\.

##### Temporal Stability\.

Figure[7](https://arxiv.org/html/2607.20655#A1.F7)in Appendix shows that the Top\-10% 7\-day recall remains stable between 40–48% throughout the 103\-day observation, with no degradation trend\. This confirms that the ranking backbone delivers consistent quality under real production traffic—a prerequisite for the online loop to build upon reliably in long\-term deployment\.

### 4\.6Sensitivity Analysis

We examine howSalesLoop’s advantage varies with the operational capacityKK, which determines the size of the daily lead pool distributed to sales specialists\. Table[7](https://arxiv.org/html/2607.20655#S4.T7)shows P@K lift over LLM\+DPO at differentKKvalues\.

Table 7:P@K relative lift ofSalesLoopover LLM\+DPO at differentKK\.SalesLoop’s advantage is largest at smallerKKvalues \(\+21\.4% atK=1,000K=1\{,\}000\) and diminishes asKKincreases\. This is expected: the listwise optimization with position\-weighted rewards primarily improves Top\-KKranking quality, which matters most when the operational window is tight\. At largeKKvalues approaching full coverage, the ranking advantage naturally diminishes as most convertible leads are already included regardless of ordering\. Note thatKKis set by operational capacity \(hereK=10,000K=10\{,\}000leads per day\) rather than tuned; the analysis shows thatSalesLoop’s benefit would be even larger under tighter capacity, and remains positive across the full range\.

## 5Conclusion

We presentedSalesLoop, an online reinforcement learning framework for lead ranking that addresses three fundamental gaps in existing approaches: the offline\-online distribution mismatch, the pointwise\-listwise objective misalignment, and the historical\-current distribution drift\.

Our key contributions include: \(1\) formalizing these three gaps and proposing a closed\-loop paradigm for continuous adaptation; \(2\) designing a performance\-aware reward that incorporates conversion outcomes, ranking position, and conversion velocity; and \(3\) developing Discriminative GRPO, a listwise optimization objective that adapts group\-relative advantages to discriminative ranking models\.

Comprehensive experiments on large\-scale industrial data \(16\.5M leads, 160 days, two provincial markets with 280 specialists\) demonstrate thatSalesLoopachieves consistent improvements over static baselines\. A production A/B test validates \+4\.7% and \+8\.7% cumulative lift in lock\-in conversions \(p=0\.047p=0\.047andp=0\.002p=0\.002respectively\), with the advantage widening as the feedback loop accumulates deployment data—evidence of its compounding nature\. Deployment metrics further validate the absolute ranking quality of the backbone: Top\-10% recall of 44\.1% \(4\.4×\\timesrandom\) and 7,220 incremental high\-value conversions from model\-identified high\-intent leads that specialists had overlooked\.

##### Limitations and Future Work\.

The current system operates at monthly granularity due to the 30\-day conversion window\. Future directions include: \(1\)faster feedback: leveraging intermediate signals \(e\.g\., test drive bookings, showroom visits\) to enable weekly or daily iterations; \(2\)multi\-objective optimization: balancing short\-term conversion with long\-term customer lifetime value; \(3\)cross\-domain transfer: extendingSalesLoopto other long\-cycle sales domains such as real estate and enterprise B2B; and \(4\)exploration\-exploitation: incorporating uncertainty\-aware selection to balance ranking quality with data collection for future learning\. Additionally, investigating alternative reward formulations \(e\.g\., DPO\-style pairwise preferences derived from performance outcomes\) and more sophisticated listwise losses \(e\.g\., NeuralNDCG\[[22](https://arxiv.org/html/2607.20655#bib.bib34)\]\) are promising directions\.

## References

- \[1\]C\. Burges, T\. Shaked, E\. Renshaw, A\. Lazier, M\. Deeds, N\. Hamilton, and G\. Hullender\(2005\)Learning to rank using gradient descent\.InProceedings of ICML,pp\. 89–96\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px2.p1.1)\.
- \[2\]Z\. Cao, T\. Qin, T\. Liu, M\. Tsai, and H\. Li\(2007\)Learning to rank: from pairwise approach to listwise approach\.InProceedings of ICML,pp\. 129–136\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px2.p1.1),[§3\.4](https://arxiv.org/html/2607.20655#S3.SS4.SSS0.Px4.p1.1)\.
- \[3\]M\. Chen, A\. Beutel, P\. Covington,et al\.\(2019\)Top\-k off\-policy correction for a reinforce recommender system\.InProceedings of WSDM,pp\. 456–464\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px3.p1.1)\.
- \[4\]T\. Chen and C\. Guestrin\(2016\)XGBoost: a scalable tree boosting system\.InProceedings of KDD,pp\. 785–794\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1),[1st item](https://arxiv.org/html/2607.20655#S4.I2.i1.p1.1)\.
- \[5\]H\. Chenget al\.\(2016\)Wide & deep learning for recommender systems\.InProceedings of DLRS,pp\. 7–10\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1)\.
- \[6\]A\. Chuklin, I\. Markov, and M\. d\. Rijke\(2015\)Click models for web search\.Morgan & Claypool Publishers\.Cited by:[§1](https://arxiv.org/html/2607.20655#S1.SS0.SSS0.Px3.p5.1),[§3\.3](https://arxiv.org/html/2607.20655#S3.SS3.p2.2)\.
- \[7\]J\. Gama, I\. Žliobaitė, A\. Bifet, M\. Pechenizkiy, and A\. Bouchachia\(2014\)A survey on concept drift adaptation\.ACM Computing Surveys46\(4\),pp\. 1–37\.Cited by:[§1](https://arxiv.org/html/2607.20655#S1.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px4.p1.1)\.
- \[8\]H\. Guo, R\. Tang, Y\. Ye, Z\. Li, and X\. He\(2017\)DeepFM: a factorization\-machine based neural network for ctr prediction\.InProceedings of IJCAI,pp\. 1725–1731\.Cited by:[§1](https://arxiv.org/html/2607.20655#S1.p2.1),[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1),[2nd item](https://arxiv.org/html/2607.20655#S4.I2.i2.p1.1)\.
- \[9\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2022\)LoRA: low\-rank adaptation of large language models\.InProceedings of ICLR,Cited by:[§3\.5](https://arxiv.org/html/2607.20655#S3.SS5.SSS0.Px2.p1.4)\.
- \[10\]K\. Huanget al\.\(2024\)CRMArena: understanding the capacity of llm agents to perform professional crm tasks in realistic environments\.arXiv preprint arXiv:2411\.02305\.Cited by:[§1](https://arxiv.org/html/2607.20655#S1.p2.1),[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1)\.
- \[11\]E\. Ie, V\. Jain, J\. Wang,et al\.\(2019\)SlateQ: a tractable decomposition for reinforcement learning with recommendation sets\.InProceedings of IJCAI,pp\. 2592–2599\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px3.p1.1)\.
- \[12\]R\. Jagerman, H\. Oosterhuis, and M\. de Rijke\(2019\)To model or to intervene: a comparison of counterfactual and online learning to rank from user interactions\.InProceedings of SIGIR,pp\. 15–24\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px2.p1.1)\.
- \[13\]D\. R\. Jiang, A\. Nikulkov, Y\. Chen, Y\. Bai, and Z\. Zhu\(2025\)Improving generative ad text on facebook using reinforcement learning\.arXiv preprint arXiv:2507\.21983\.Cited by:[§1](https://arxiv.org/html/2607.20655#S1.SS0.SSS0.Px3.p2.1),[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px3.p1.1)\.
- \[14\]J\. Lianet al\.\(2018\)XDeepFM: combining explicit and implicit feature interactions for recommender systems\.InProceedings of KDD,pp\. 1754–1763\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1)\.
- \[15\]T\. Liu\(2009\)Learning to rank for information retrieval\.Springer\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px2.p1.1)\.
- \[16\]I\. Loshchilov and F\. Hutter\(2019\)Decoupled weight decay regularization\.InProceedings of ICLR,Cited by:[§3\.5](https://arxiv.org/html/2607.20655#S3.SS5.SSS0.Px3.p1.4)\.
- \[17\]N\. Mukkunnothet al\.\(2025\)SalesRLAgent: a reinforcement learning approach for real\-time sales conversion prediction and optimization\.arXiv preprint arXiv:2503\.23303\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1)\.
- \[18\]H\. Oosterhuis and M\. de Rijke\(2018\)Differentiable unbiased online learning to rank\.InProceedings of CIKM,pp\. 1293–1302\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px2.p1.1)\.
- \[19\]L\. Ouyang, J\. Wu, X\. Jiang,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in Neural Information Processing Systems35,pp\. 27730–27744\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px3.p1.1)\.
- \[20\]G\. I\. Parisi, R\. Kemker, J\. L\. Part, C\. Kanan, and S\. Wermter\(2019\)Continual lifelong learning with neural networks: a review\.Neural Networks113,pp\. 54–71\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px4.p1.1)\.
- \[21\]A\. Payne and P\. Frow\(2005\)A strategic framework for customer relationship management\.Journal of Marketing69\(4\),pp\. 167–176\.Cited by:[§1](https://arxiv.org/html/2607.20655#S1.p1.1)\.
- \[22\]P\. Pobrotyn and R\. Białobrzeski\(2021\)NeuralNDCG: direct optimisation of a ranking metric via differentiable relaxation of sorting\.arXiv preprint arXiv:2102\.07831\.Cited by:[§5](https://arxiv.org/html/2607.20655#S5.SS0.SSS0.Px1.p1.1)\.
- \[23\]Qwen, :, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu\(2025\)Qwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§3\.5](https://arxiv.org/html/2607.20655#S3.SS5.SSS0.Px1.p1.1)\.
- \[24\]R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn\(2024\)Direct preference optimization: your language model is secretly a reward model\.Advances in Neural Information Processing Systems36\.Cited by:[4th item](https://arxiv.org/html/2607.20655#S4.I2.i4.p1.1),[§4\.2](https://arxiv.org/html/2607.20655#S4.SS2.SSS0.Px2.p1.1)\.
- \[25\]T\. Schnabel, A\. Swaminathan, A\. Singh, N\. Chandak, and T\. Joachims\(2016\)Recommendations as treatments: debiasing learning and evaluation\.InProceedings of ICML,pp\. 1670–1679\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px4.p1.1)\.
- \[26\]J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov\(2017\)Proximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px3.p1.1)\.
- \[27\]S\. Shalev\-Shwartz\(2012\)Online learning and online convex optimization\.Foundations and Trends in Machine Learning4\(2\),pp\. 107–194\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px4.p1.1)\.
- \[28\]Z\. Shaoet al\.\(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§1](https://arxiv.org/html/2607.20655#S1.SS0.SSS0.Px3.p2.1),[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px3.p1.1),[§3\.4](https://arxiv.org/html/2607.20655#S3.SS4.p1.1)\.
- \[29\]W\. Songet al\.\(2019\)AutoInt: automatic feature interaction learning via self\-attentive neural networks\.InProceedings of CIKM,pp\. 1161–1170\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1)\.
- \[30\]Y\. Sun, Y\. Liu, J\. Song, C\. Zhang, X\. Zhang, L\. Liu, S\. Chen, and Y\. Cao\(2025\)AsLLR: llm based leads ranking in auto sales\.arXiv preprint arXiv:2510\.21713\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1)\.
- \[31\]R\. Wang, B\. Fu, G\. Fu, and M\. Wang\(2017\)Deep & cross network for ad click predictions\.InProceedings of ADKDD,pp\. 1–7\.Cited by:[§1](https://arxiv.org/html/2607.20655#S1.p2.1),[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1)\.
- \[32\]R\. Wanget al\.\(2021\)DCN v2: improved deep & cross network and practical lessons for web\-scale learning to rank systems\.InProceedings of WWW,pp\. 1785–1797\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1)\.
- \[33\]R\. S\. Winer\(2001\)Customer relationship management: a framework, research directions, and the future\.University of California at Berkeley\.Cited by:[§1](https://arxiv.org/html/2607.20655#S1.p1.1),[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1)\.
- \[34\]F\. Xia, T\. Liu, J\. Wang, W\. Zhang, and H\. Li\(2008\)Listwise approach to learning to rank: theory and algorithm\.InProceedings of ICML,pp\. 1192–1199\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px2.p1.1)\.
- \[35\]X\. Xu, M\. Xie, R\. Li,et al\.\(2025\)An end\-to\-end multi\-objective ensemble ranking framework for video recommendation\.arXiv preprint arXiv:2508\.05093\.Cited by:[§1](https://arxiv.org/html/2607.20655#S1.SS0.SSS0.Px1.p1.1)\.
- \[36\]Z\. Xue, Q\. Cai, T\. Zuo, B\. Yang, L\. Hu, P\. Jiang, K\. Gai, and B\. An\(2025\)AURO: reinforcement learning for adaptive user retention optimization in recommender systems\.InProceedings of WWW,Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px3.p1.1)\.
- \[37\]C\. Zhang, Y\. Liu, Y\. Sun, X\. Zhang, Y\. Cao, J\. Jiao, and J\. Qiao\(2026\)Rethinking sales lead scoring with llm\-based hierarchical preference ranking\.arXiv preprint arXiv:2606\.04387\.Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px1.p1.1),[§3\.5](https://arxiv.org/html/2607.20655#S3.SS5.SSS0.Px1.p1.1)\.
- \[38\]Y\. Zhang, L\. Yan, Z\. Qin, H\. Zhuang, J\. Shen, X\. Wang, M\. Bendersky, and M\. Najork\(2023\)Towards disentangling relevance and bias in unbiased learning to rank\.InProceedings of KDD,Cited by:[§2](https://arxiv.org/html/2607.20655#S2.SS0.SSS0.Px4.p1.1)\.

## Appendix AAdditional Online Deployment Analysis

This appendix provides supplementary analyses of the online deployment data to further characterizeSalesLoop’s feedback loop dynamics\.

### A\.1Daily Conversion Rate Analysis

Figure[5](https://arxiv.org/html/2607.20655#A1.F5)presents the 7\-day moving average of daily new conversions for treatment and control groups in both provinces\. Unlike the cumulative view in Figure[2](https://arxiv.org/html/2607.20655#S4.F2), this visualization reveals theinstantaneousproductivity differences\.

![Refer to caption](https://arxiv.org/html/2607.20655v1/x4.png)Figure 5:Daily new conversions \(7\-day moving average\) for treatment and control groups\. Province A shows treatment overtaking control around March 2026; Province B shows treatment consistently above control from early January\.Key observations:

- •Province A: The treatment and control daily rates are nearly identical during December–February, with occasional crossovers\. Starting March 2026, the treatment curve consistently exceeds the control, with the gap widening particularly during high\-volume periods \(March, May\)\.
- •Province B: The treatment daily rate exceeds the control almost immediately and maintains this advantage throughout, with the gap becoming more pronounced in later months as the feedback loop compounds\.
- •Shared seasonal patterns: Both treatment and control exhibit similar seasonal fluctuations \(e\.g\., dips during Chinese New Year in mid\-February, peaks during promotional periods in March and May\), confirming that the randomization successfully balanced external factors between groups\.

### A\.2Per\-Specialist Productivity

To control for the difference in team sizes between provinces \(60 per group in Province A vs\. 80 in Province B\), Figure[6](https://arxiv.org/html/2607.20655#A1.F6)normalizes cumulative conversions by the number of specialists in each group\.

![Refer to caption](https://arxiv.org/html/2607.20655v1/x5.png)Figure 6:Cumulative conversions per specialist, enabling cross\-province comparison on a normalized scale\.After normalization, the per\-specialist productivity reveals:

- •Both provinces have similar per\-specialist baselines \(∼\\sim30–36 conversions per specialist over 160 days\)\.
- •SalesLoopprovides \+1\.6 conversions per specialist in Province A and \+2\.7 in Province B\.
- •The per\-specialist curves are closely matched across provinces for the same condition \(e\.g\., both control groups converge to∼\\sim30–34/specialist\), suggesting comparable market conditions despite different team sizes\.

These normalized results confirm that the lift observed in aggregate is not an artifact of team size differences and reflects genuine per\-specialist productivity gains from improved lead ranking\.

### A\.3Ranking Quality Temporal Stability

Figure[7](https://arxiv.org/html/2607.20655#A1.F7)shows the daily Top\-10% recall rates over the 103\-day production window \(September 20 – December 31, 2025\)\.

![Refer to caption](https://arxiv.org/html/2607.20655v1/x6.png)Figure 7:Top\-10% recall stability over 103 days\. The base rankerfθ0f\_\{\\theta\_\{0\}\}maintains 7\-day recall between 37–49% \(mean 44\.1%\) and 14\-day recall between 36–45% \(mean 41\.3%\), with no degradation trend\. The random baseline \(10%\) is shown for reference\.Key observations:

- •The 7\-day recall remains in the 40–48% range during stable periods, representing a4\.4×\\timeslift over random\(10% baseline\)\.
- •No downward trend is observed over the 103\-day window, indicating that the ranking backbone is robust to day\-to\-day distribution shifts in production\.
- •Brief dips \(e\.g\., early October, early November\) coincide with major product launches that temporarily shift the lead distribution, after which recall recovers\.

### A\.4Incremental Value Over Human Judgment

Figure[8](https://arxiv.org/html/2607.20655#A1.F8)compares the conversion rate ofSalesLoop\-identified incremental leads versus the specialist\-flagged baseline\.

![Refer to caption](https://arxiv.org/html/2607.20655v1/x7.png)Figure 8:14\-day conversion rate comparison: specialist\-flagged leads \(2\.05%\) vs\.SalesLoopincremental identification \(4\.66%\)\. The model identifies high\-intent leads that specialists miss, at 2\.3×\\timeshigher conversion rate\.This result demonstrates thatSalesLoop’s dialogue\-based ranking captures latent purchase intent signals that even experienced sales specialists cannot identify from their standard workflow\. The model effectively serves as acomplementary intelligence layerrather than a replacement for human judgment\.