Selection Bias Correction in Retail Intelligence

# Selection Bias Correction in Retail Intelligence
Source: [https://arxiv.org/html/2608.26156](https://arxiv.org/html/2608.26156)
Spandan Ghose Chowdhury1![[Uncaptioned image]](https://arxiv.org/html/2608.26156v1/x1.png) 1Walmart Inc\., California, USA spandan\.ghose\.chowdhury@walmart\.com

###### Abstract

Retail intelligence often relies on monitoring popular, high\-velocity products, potentially biasing economic indicators by ignoring the “long tail” of niche items\. This simulation study investigates selection bias in inflation estimation and compares correction methods across diverse data\-generating processes\. Through 400 Monte Carlo replications spanning four scenarios—aligned step functions, smooth gradients, misaligned breaks, and polynomial relationships—we test the robustness of Inverse Probability Weighting \(IPW\) with five specifications against stratification with varying strata counts\. Our findings reveal fundamental limits of weighting methods in retail long\-tail contexts: stratification achieves superior performance in three of four scenarios, maintaining sub\-0\.04pp median error even when boundaries deliberately misalign with population breaks \(116×\\timesadvantage over IPW\)\. However, IPW with spline propensity models wins under smooth polynomial relationships \(median error 0\.007pp vs\. 0\.013pp\), demonstrating context\-dependency\. Critically, even an oracle IPW specification with perfect structural knowledge achieves 6\.06pp error compared to stratification’s 0\.008pp in step\-function scenarios\. This reflectsviolation of the Positivity Assumption—a fundamental causal inference requirement—rather than IPW methodological inferiority\. When selection probabilities differ dramatically \(90% vs\. 1%\), weighting methods operate outside their theoretical design envelope\. These results demonstrate that stratification provides a safer engineering choice in retail long\-tail distributions with severe positivity violations\.

## 1INTRODUCTION

In the data driven retail sector, data insights is critical for strategic decision\-making\. A standard industry practice involves monitoring a curated list of popular products to gauge market trends, measure metrics like category\-level inflation\. While operationally efficient, this method introduces significant selection bias by systematically excluding the vast “long tail” of lower\-velocity items\.

Empirical evidence and economic theories suggest that niche products may experience different dynamics than very popular items\. The “long tail” phenomenon indicates that while individually less significant, collectively these items represent sizable market dynamics\. If niche products experience higher price volatility or inflation than tracked popular items — due to economies of scale, competitive pressure differences, macro\-economic changes, or strategic pricing — selective monitoring could systematically misestimate true market inflation or related business metrics\.

This study uses a comprehensive simulation methodology to rigorously quantify this bias and evaluate correction methods in various data\-generating processes\. Our analysis includes 400 Monte Carlo replications in four scenarios \(aligned step functions, smooth gradients, misaligned breaks, and polynomial relationships\), testing five IPW specifications and three stratification configurations\. We perform complete diagnostic assessments including covariate balance, weight distributions, and power analysis to understand the performance of the method under varied conditions\.

Our findings challenge conventional wisdom while acknowledging edge cases: stratification achieves superior performance in three of four scenarios, maintaining robustness even when boundaries deliberately misalign with population breaks \(116×\\timesadvantage over IPW\)\. However, IPW with flexible spline specifications wins under smooth polynomial relationships \(1\.9×\\timesadvantage\), demonstrating that no single method dominates universally\. These results provide practitioners with evidence\-based, context\-dependent guidance for selecting bias correction methods\.

## 2RELATED WORK

Correction of selection bias through weighting has its roots in survey sampling theory, notably the Horvitz\-Thompson estimator\[[8](https://arxiv.org/html/2608.26156#bib.bib9)\], which uses the inverse of inclusion probabilities to provide unbiased population estimates\. Rosenbaum and Rubin\[[15](https://arxiv.org/html/2608.26156#bib.bib15)\]extended this framework by introducing the propensity score—the conditional probability of selection given observed covariates\.

IPW has since become a staple in causal inference\. Lunceford and Davidian\[[13](https://arxiv.org/html/2608.26156#bib.bib19)\]showed that both stratification and IPW can adjust for confounding, but their relative performance depends on model specification and covariate overlap\. However, IPW is highly sensitive to the “Positivity Assumption”: when nearly violated—as in retail datasets where niche items are tracked at rates orders of magnitude lower than popular items—weights become extreme, yielding high variance and unstable estimates\[[3](https://arxiv.org/html/2608.26156#bib.bib4)\]\. Matsouaka and Zhou\[[14](https://arxiv.org/html/2608.26156#bib.bib20)\]show that overlap, matching, and entropy weights are preferable to IPW trimming under severe positivity violations, targeting only the subpopulation with sufficient propensity score overlap\.

King and Nielsen\[[10](https://arxiv.org/html/2608.26156#bib.bib11)\]argue that propensity score methods can inadvertently increase imbalance and model dependence, advocating instead for coarsened methods like stratification\. Stuart\[[18](https://arxiv.org/html/2608.26156#bib.bib17)\]similarly finds that stratification often achieves comparable bias reduction with greater robustness to model misspecification\.

Despite these debates, empirical comparisons within retail “long tail” distributions remain sparse\. This study bridges that gap by stress\-testing these estimators under severe selection imbalance\.

## 3METHODOLOGY

To test our hypothesis, we performed a comprehensive simulation study with three key components: \(1\) baseline scenario analysis with full diagnostics, \(2\) robustness testing across four diverse data generation processes, and \(3\) Monte Carlo simulation with 100 replications per scenario\. The methodology was designed to validate correction approaches in controlled settings where ground truth is known, while ensuring findings are not artifacts of design choices\.

### 3\.1Synthetic Data Generation and Theoretical Justification

We generated synthetic datasets of varying sizes \(10,000 to 1,000,000 items\) to create a “ground truth” scenario where true inflation is known\. The baseline scenario consisted of 100,000 items with the following characteristics:

1. 1\.Item Ranking:Each itemiireceived a rankRi∈\{1,2,…,100,000\}R\_\{i\}\\in\\\{1,2,\\ldots,100\{,\}000\\\}, where lower ranks indicate higher sales velocity\.
2. 2\.Inflation Assignment:A bimodal inflation distribution was created for the true inflation rateYiY\_\{i\}\(the target variable we aim to measure\): - •Popular items \(Ri≤20,000R\_\{i\}\\leq 20\{,\}000\):Yi=2%Y\_\{i\}=2\\% - •Niche items \(Ri\>20,000R\_\{i\}\>20\{,\}000\):Yi=10%Y\_\{i\}=10\\% The threshold of 20,000 items reflects empirical retail patterns: approximately 20% of items typically generate 80% of Gross Merchandise Value \(GMV\), consistent with the Pareto principle observed in retail operations\[[1](https://arxiv.org/html/2608.26156#bib.bib18)\]\. This 20% represents the high\-velocity items that retailers actively track\. Theoretical Justification:This bimodal structure reflects documented market phenomena: - •Economies of scale:High\-volume products often have lower marginal costs and more competitive pricing - •Price volatility in thin markets:Products with lower sales volumes face less competitive pressure and greater susceptibility to supply chain disruptions - •Strategic pricing:Retailers often use popular items as “loss leaders” while maintaining higher margins on specialty products To add realism, we introduced random variation around these base rates\. The true inflation rate for each item was drawn from a normal distribution:Yi∼𝒩​\(μsegment,0\.5\)Y\_\{i\}\\sim\\mathcal\{N\}\(\\mu\_\{\\text\{segment\}\},0\.5\), whereμsegment\\mu\_\{\\text\{segment\}\}is the segment\-specific mean \(2% for popular items, 10% for niche items\) and 0\.5 represents the standard deviation in percentage points\. This noise captures the natural heterogeneity in item\-level price changes within each segment\.
3. 3\.Selection Mechanism:The probability of tracking itemiiwas: P​\(Ti=1∣Ri\)=\{0\.90if​Ri<20,0000\.01otherwiseP\(T\_\{i\}=1\\mid R\_\{i\}\)=\\begin\{cases\}0\.90&\\text\{if \}R\_\{i\}<20\{,\}000\\\\ 0\.01&\\text\{otherwise\}\\end\{cases\}\(1\) This created a realistic scenario where≈\\approx18,000 popular items and≈\\approx800 niche items were tracked \(total≈\\approx18,800 tracked items\)\.

#### 3\.1\.1Robustness to Data Generation Process

An important consideration in simulation studies is making sure conclusions are not artifacts of design choices\. To address this concern, we test robustness across four data generation processes with varying empirically\-motivated thresholds:

DGP 1 \(Step/Aligned\):This baseline scenario mirrors the empirical retail context where high\-velocity “Head” items \(top 20%\) are preferentially tracked\. Selection probabilityP​\(tracked\|Ri\)=0\.90P\(\\text\{tracked\}\|R\_\{i\}\)=0\.90for items withRi<20,000R\_\{i\}<20\{,\}000\(top 20%\), dropping to 0\.01 for lower\-velocity items—reflecting that retailers intensively monitor popular products while sporadically sampling the long tail\. The true inflation rateYiY\_\{i\}\(not noise, but the actual outcome variable\) followsYi∼𝒩​\(2%,0\.5\)Y\_\{i\}\\sim\\mathcal\{N\}\(2\\%,0\.5\)ifRi≤20,000R\_\{i\}\\leq 20\{,\}000, else𝒩​\(10%,0\.5\)\\mathcal\{N\}\(10\\%,0\.5\), where 0\.5 is the standard deviation capturing within\-segment heterogeneity\. Stratification uses 5 strata at 20,000 intervals, aligned with the selection and inflation breaks\.

DGP 2 \(Smooth Gradient\):Selection follows a logistic curveP​\(tracked\|Ri\)=0\.95/\(1\+exp⁡\(\(Ri−15,000\)/3,000\)\)P\(\\text\{tracked\}\|R\_\{i\}\)=0\.95/\(1\+\\exp\(\(R\_\{i\}\-15\{,\}000\)/3\{,\}000\)\); Inflation increases smoothly with rank:Yi∼𝒩​\(2%\+\(Ri/100,000\)×10%,0\.5\)Y\_\{i\}\\sim\\mathcal\{N\}\(2\\%\+\(R\_\{i\}/100\{,\}000\)\\times 10\\%,0\.5\)\. This tests continuous relationships without structural breaks, reflecting gradual market segmentation rather than discrete categories\.

DGP 3 \(Step/Misaligned\):To test stratification’s worst\-case performance, we deliberately misalign all structural elements: selection probability breaks at rank 15,000, true inflation breaks at rank 25,000, while stratification boundaries are placed at 14k/28k/42k/56k/70k/84k \(7 strata\)\. None of these thresholds coincide, forcing stratification to cutacrossnatural population breaks rather than aligning with them\. This stress\-tests whether stratification’s advantages are merely artifacts of fortuitous boundary alignment\.

DGP 4 \(Polynomial\):Quadratic selection and inflation functions:Yi∼𝒩​\(6%\+4%×\[\(Ri−50,000\)/50,000\]2,0\.5\)Y\_\{i\}\\sim\\mathcal\{N\}\(6\\%\+4\\%\\times\[\(R\_\{i\}\-50\{,\}000\)/50\{,\}000\]^\{2\},0\.5\)\. This U\-shaped inflation pattern tests non\-linear smooth relationships where coarse stratification boundaries cannot approximate continuous curvature as well as flexible splines\.

We compare five IPW specifications \(Linear, Polynomial, Spline with 5 knots, Oracle withI​\(Ri\>20,000\)I\(R\_\{i\}\>20\{,\}000\)dummy\) and three stratification configurations \(5, 7, 9 strata\)\.

### 3\.2Inverse Probability Weighting \(IPW\) Correction

We implemented IPW correction following Rosenbaum and Rubin\[[15](https://arxiv.org/html/2608.26156#bib.bib15)\]to create a pseudo\-population where covariate distribution \(Rank\) is independent of selection\. LetTi=1T\_\{i\}=1if itemiiis tracked,Ti=0T\_\{i\}=0otherwise\. The propensity scoree​\(Xi\)=P​\(Ti=1∣Xi\)e\(X\_\{i\}\)=P\(T\_\{i\}=1\\mid X\_\{i\}\)was estimated via logistic regression:logit​\(e​\(Xi\)\)=β0\+β1⋅Rank​\(i\)\\text\{logit\}\(e\(X\_\{i\}\)\)=\\beta\_\{0\}\+\\beta\_\{1\}\\cdot\\text\{Rank\}\(i\)\. Each tracked item received weightwi=1/e^​\(Xi\)w\_\{i\}=1/\\hat\{e\}\(X\_\{i\}\)\.

To mitigate variance from extreme weights\[[3](https://arxiv.org/html/2608.26156#bib.bib4)\], we stabilized weights by multiplying byP​\(T=1\)P\(T=1\)and clipped at the 95th percentile\[[11](https://arxiv.org/html/2608.26156#bib.bib12)\]\. Weighted inflation was calculated as:∑i:Ti=1wi⋅Yi/∑i:Ti=1wi\\sum\_\{i:T\_\{i\}=1\}w\_\{i\}\\cdot Y\_\{i\}/\\sum\_\{i:T\_\{i\}=1\}w\_\{i\}\.

### 3\.3Alternative Correction Methods

For comparison, we implemented: \(1\) Stratification: 5 rank\-based strata with weighted averages; \(2\) Propensity Score Matching: 1:1 nearest\-neighbor; \(3\) Regression Adjustment: linear regression controlling for rank and tracking status; \(4\) Naive: unweighted average of tracked items\.

### 3\.4Propensity Score Model Diagnostics

We assessed propensity score model balance using standardized mean difference \(SMD\) of Rank:SMD=\(μtracked−μuntracked\)/\(σtracked2\+σuntracked2\)/2\\text\{SMD\}=\(\\mu\_\{\\text\{tracked\}\}\-\\mu\_\{\\text\{untracked\}\}\)/\\sqrt\{\(\\sigma^\{2\}\_\{\\text\{tracked\}\}\+\\sigma^\{2\}\_\{\\text\{untracked\}\}\)/2\}\. Successful balancing requires SMD<<0\.1\[[2](https://arxiv.org/html/2608.26156#bib.bib3)\]\. We also calculated Effective Sample Size \(ESS\) to quantify information loss:ESS=\(∑wi\)2/∑wi2\\text\{ESS\}=\(\\sum w\_\{i\}\)^\{2\}/\\sum w\_\{i\}^\{2\}\.

### 3\.5Monte Carlo Simulation Design

We replicated the baseline scenario 1,000 times with different random seeds, calculating bias, variance, and MSE \(Bias2\+ Variance\) for all methods\. The comprehensive robustness analysis \(Section 3\.2\) uses 100 replications per scenario across four diverse DGPs\.

## 4RESULTS

We present results in three parts: Baseline scenario with complete diagnostics, Monte Carlo simulation results, and robustness testing across four data generation processes\.

### 4\.1Baseline Scenario: Method Comparison

The baseline simulation \(100,000 items, 90%/1% tracking probabilities, 2%/10% inflation rates\) yielded the following estimates:

Table 1:Baseline Scenario Results\.Key Finding:In the baseline scenario, stratification achieves near\-perfect accuracy \(0\.016 pp error\), substantially outperforming other methods\. IPW modestly improves over naive \(6\.057→\\rightarrow5\.849 pp\), reducing absolute error by 3\.4%\. Matching and regression adjustment show intermediate performance\.

### 4\.2Diagnostic Analysis

#### 4\.2\.1Covariate Balance Assessment

Table 2:Covariate Balance Diagnostics\.Despite weighting, SMD remains 22×\\timesabove the recommended threshold, indicating severe residual imbalance and model misspecification\. The reduction from 2\.420 to 2\.197 represents only a 9\.2% improvement, far short of the 95\.5% reduction needed to reach the 0\.10 threshold\.

#### 4\.2\.2Weight Distribution Statistics

Table 3:IPW Weight Distribution Statistics\.Critical Observation:Before trimming, maximum weights reached 163,509—nearly 1 million times the median\. This extreme range \(max/median ratio: 681,289:1\) indicates severe overlap violations\. Weight trimming at the 95th percentile dramatically improved stability, reducing the max/median ratio from 681,289:1 to 1\.75:1 and increasing ESS from 0\.5% to 93\.5% of the tracked sample\.

### 4\.3Monte Carlo Simulation Results \(Baseline Scenario: 1,000 Replications\)

To quantify estimation variability in the baseline scenario, we replicated it 1,000 times\. Each replication usedn=100,000n=100\{,\}000total items with stochastic selection \(probabilities: 90% forRi<20,000R\_\{i\}<20\{,\}000, 1% otherwise\), resulting in variable tracked sample sizes \(mean≈18,800\\approx 18\{,\}800, range: 18,650\-18,950\)\. All metrics are in percentage points \(pp\)\.

Table 4:Monte Carlo Results \(Baseline Scenario: 1,000 Replications\)\. MSE reported in squared percentage points \(pp2\)\. Each replication:n=100,000n=100\{,\}000items, tracked sample size varies stochastically \(mean≈18,800\\approx 18\{,\}800\)\.Key Findings:

1. 1\.IPW reduces MSE by 6\.6%compared to naive \(34\.29 vs 36\.72\)
2. 2\.IPW variance is 42% higherthan naive \(SD: 0\.017 vs 0\.012\)
3. 3\.Stratification achieves near\-zero MSE\(0\.00\), outperforming all methods
4. 4\.IPW outperformed naive in 100% of replications

### 4\.4Robustness Across Data Generation Processes

To address concerns about alignment between simulation design and method assumptions, we conducted 400 additional Monte Carlo replications \(100 per scenario\) across four diverse data\-generating processes \(DGP\), testing five IPW specifications and three stratification configurations\.

Table 5:Method Performance Across Data Generation Processes \(Median Absolute Error in pp, 100 replications each\)\.Key Findings:

1. 1\.Stratification wins 3 of 4 scenarios:Even when boundaries deliberately misalign with population breaks \(DGP 3\), stratification achieves 0\.030pp error vs\. 3\.470pp for IPW\-Linear \(116×\\timesadvantage\)\.
2. 2\.Oracle IPW still fails in step\-function scenarios:With perfect structural knowledge \(including dummy variableI​\(Ri\>20,000\)I\(R\_\{i\}\>20\{,\}000\)\), IPW achieves 6\.061pp error vs\. stratification’s 0\.008pp \(758×\\timesworse\), demonstrating that common support violations cannot be overcome through better modeling\.
3. 3\.IPW\-Spline wins under polynomial relationships:In smooth, non\-linear scenarios without structural breaks, flexible propensity models outperform coarse stratification \(0\.007pp vs\. 0\.013pp, 1\.9×\\timesadvantage\)\.

## 5DISCUSSION

Our comprehensive analysis reveals critical findings about selection bias correction in retail inflation estimation: \(1\) stratification’s superior performance across diverse data generation processes, \(2\) IPW’s modest gains with important diagnostic failures, and \(3\) context\-dependent method selection based on data characteristics\.

### 5\.1Method Performance: Stratification Dominates, IPW Shows Modest Gains

Our findings challenge conventional assumptions about weighting methods\. Stratification achieves near\-zero MSE \(0\.00\) in the baseline scenario compared to IPW’s 34\.29, representing 99\.7% lower error\. This isnotmerely by design: while the baseline scenario uses aligned step functions, stratification maintains superior performance even in DGP 3 where boundaries deliberately misalign with population breaks \(0\.030pp vs\. 3\.470pp for IPW, 116×\\timesadvantage\)\. The near\-zero MSE reflects stratification’s fundamental advantage when dealing with severe selection imbalances—it estimates within\-group averages without requiring counterfactual predictions across non\-overlapping populations, whereas IPW must extrapolate from a 90%/1% selection imbalance that violates the Positivity Assumption\. This superiority extends across 3 of 4 comprehensive DGP scenarios\. However, IPW does reduce estimation error modestly \(6\.6% MSE reduction\) compared to naive estimation, though at the cost of 42% higher variance\.

Why Stratification Often Outperforms:Stratification groups similar items together, ensuring balance without relying on propensity score model specification\. Each stratum receives equal weight proportional to its population size, preventing extreme weight ratios\. The method maintains robustness even when boundaries misalign with population breaks \(DGP 3: 116×\\timesadvantage despite deliberate misalignment\)\.

Impact of Strata Count:Interestingly, fewer strata perform better under misalignment: in DGP 3, 5 strata achieve 0\.030pp error vs\. 0\.038pp for 7 strata and 0\.050pp for 9 strata\. This counter\-intuitive pattern reflects the bias\-variance tradeoff—coarser stratification \(fewer strata\) creates larger, more heterogeneous groups that are robust to boundary placement, while finer stratification \(more strata\) creates more opportunities for boundaries to cut across natural population breaks\. When the analyst lacks knowledge of true structural breaks, conservative stratification \(5\-7 strata\) provides insurance against misspecification\. This simplicity and transparency make it ideal for stakeholder communication\.

IPW’s Limitations:The variance inflation in IPW results from extreme weights—before trimming, some weights reached 163,509×\\times\. This indicates severe propensity score model misspecification and overlap violations\. Post\-weighting SMD remained 22×\\timesabove recommended thresholds \(2\.197 vs\. 0\.10 target\), signaling fundamental balance failures\.

Context\-Dependency:IPW with flexible spline specifications outperforms stratification under smooth polynomial relationships \(0\.007pp vs\. 0\.013pp, 1\.9×\\timesadvantage\)\. This demonstrates that no single method dominates universally\. These findings align with Stuart\[[18](https://arxiv.org/html/2608.26156#bib.bib17)\]and King & Nielsen\[[10](https://arxiv.org/html/2608.26156#bib.bib11)\], who argue that simpler matching/stratification methods often outperform sophisticated weighting in practice\.

### 5\.2Diagnostic Failures as Early Warning Signs

Our diagnostic analysis provided clear early warnings of IPW’s limitations before seeing final estimates\. Post\-weighting SMD remained at 2\.197 \(22×\\timesabove the 0\.10 threshold\), extreme weight ratios reached 681,289:1, and ESS fell to 93\.5% of the tracked sample\. Practitioners should use these diagnostic thresholds as decision rules: if SMD\>\>0\.10, max/median weight ratio\>\>100:1, or ESS<<50%, switch to stratification instead of IPW\.

### 5\.3Positivity Violation: When IPW Operates Outside Its Design Envelope

IPW’s poor performance reflects violation of the Positivity \(Overlap\) Assumption—a fundamental causal inference requirement that all units must have non\-zero probability of each treatment level\[[16](https://arxiv.org/html/2608.26156#bib.bib14)\]\. In our baseline scenario, selection probabilities differ 90\-fold \(90% vs\. 1%\), creating near\-complete group separation\. The extreme weight ratio \(681,289:1\) is not IPW failure—it’s a mathematical consequence of applying weighting to non\-overlapping populations\. Even the oracle specification with perfect structural knowledge achieved 6\.061pp error \(758×\\timesworse than stratification\), demonstrating that no amount of flexible modeling can create valid counterfactual predictions when groups barely overlap\.

Stratification succeeds because it operates differently: it estimates within\-stratum averages without counterfactual predictions for items outside the tracked distribution\. When a stratum has few tracked items, only that stratum’s estimate suffers—other strata remain valid\. This graceful degradation makes stratification a safer engineering choice when positivity violations are severe, as in retail long\-tail distributions\.

### 5\.4Addressing the Tautological Design Critique

A legitimate concern in simulation studies is whether favorable results stem from alignment between the data\-generating process and the favored method’s assumptions\. Our initial design—step functions at Rank=20,000 with stratification boundaries at 20k intervals—could be criticized as tautological: “Of course stratification wins when you design the world to match its assumptions\.”

We address this concern across three dimensions:

Boundary Alignment:DGP 3 deliberately creates a worst\-case scenario for stratification\. Selection probability breaks at 15,000, inflation breaks at 25,000, and stratification boundaries are placed at 14k/28k/42k/56k/70k/84k\. This triple misalignment means stratification cutsacrossnatural population breaks rather than aligning with them\. Despite this handicap, stratification achieves 0\.030pp median error compared to IPW\-Linear’s 3\.470pp \(116×\\timesadvantage\)\. If stratification’s success were merely an artifact of boundary alignment, it should fail catastrophically in DGP 3—yet it remains robust\.

Functional Form Flexibility:We tested five IPW specifications to provide “every possible advantage”:

- •IPW\-Linear: Original specification
- •IPW\-Polynomial: Quadratic terms to capture curvature
- •IPW\-Spline: Natural cubic splines with 5 knots for maximum flexibility
- •IPW\-Oracle: Perfect structural knowledge withI​\(Ri\>20,000\)I\(R\_\{i\}\>20\{,\}000\)dummy variable \(DGPs 1 & 3 only\)

The oracle specification is particularly revealing\. In DGP 1 \(aligned step functions\), we explicitly tell the propensity model where the break occurs—perfect information that stratification doesn’t receive in functional form\. Yet IPW\-Oracle achieves 6\.061pp error vs\. stratification’s 0\.008pp \(758×\\timesworse\)\. This demonstrates that the issue is not model misspecification but rather fundamental limitations from common support violations\. When tracked and untracked populations barely overlap \(90% vs\. 1% selection probabilities\), no amount of flexible modeling can manufacture good counterfactual predictions\.

Generalization Testing:DGPs 2 and 4 test smooth, continuous relationships without structural breaks—scenarios where stratification’s discrete boundaries seem disadvantaged\. In DGP 2 \(smooth gradient\), stratification maintains its advantage \(1\.350pp vs\. 3\.872pp, 2\.9×\\times\)\. However, in DGP 4 \(polynomial\), IPW\-Spline finally wins \(0\.007pp vs\. 0\.013pp, 1\.9×\\times\)\. This demonstrates intellectual honesty: we report scenarios where the alternative method wins, strengthening confidence that stratification’s overall superiority is not a simulation artifact\.

Bottom Line:Stratification’s dominance persists across aligned breaks, misaligned breaks, and smooth gradients\. It fails only under polynomial relationships where its discrete boundaries cannot approximate continuous curvature as well as flexible splines\. The robustness across three of four diverse scenarios demonstrates that our findings reflect genuine methodological advantages rather than tautological design\.

## 6RECOMMENDATIONS

Based on our comprehensive simulation study, we offer evidence\-based recommendations for practitioners:

Primary Recommendation: Use Stratification in Long\-Tail Contexts

- •Wins in 3 of 4 scenarios; robust even when boundaries misalign with population breaks
- •Does not require overlap between tracked/untracked populations
- •Gracefully handles severe selection imbalances \(90% vs\. 1%\)
- •Implementation: Divide items into 5\-10 rank\-based groups, calculate inflation within each, then take weighted average

When to Consider IPW:

- •Positivity Assumption satisfied \(adequate propensity score overlap\)
- •Smooth polynomial relationships without structural breaks
- •Only if diagnostics pass: SMD<<0\.10, max/median weight ratio<<100:1, ESS\>\>50%

When IPW is Inappropriate:

- •Selection probabilities differ by orders of magnitude—violates Positivity Assumption
- •Step functions or structural breaks creating near\-complete group separation
- •Poor overlap \(<<10% common support\)

Reporting Standards:Always report naive, corrected, and \(if available\) true estimates; diagnostic metrics \(SMD, ESS, weight distribution\); and confidence intervals\.

## 7CONCLUSION

This comprehensive simulation study evaluated selection bias correction methods across 400 Monte Carlo replications spanning four diverse data\-generating processes\. Our findings provide evidence\-based, nuanced and retail specific guidance that challenges conventional wisdom while acknowledging edge cases\.

1\. Stratification Dominates in Three of Four Scenarios

Rank\-based stratification achieves superior performance when relationships contain structural breaks or smooth gradients, maintaining robustness even when boundaries deliberately misalign with population breaks \(median error: 0\.030pp vs\. 3\.470pp for IPW, 116×\\timesadvantage\)\. This dominance persists across:

- •Step functions with aligned breaks \(730×\\timesadvantage\)
- •Smooth gradient relationships \(2\.9×\\timesadvantage\)
- •Step functions with misaligned breaks \(116×\\timesadvantage\)

2\. IPW Wins Under Polynomial Relationships

Flexible propensity score models \(splines with 5 knots\) outperform coarse stratification when true relationships are smooth polynomials \(median error: 0\.007pp vs\. 0\.013pp, 1\.9×\\timesadvantage\)\. This showcases that no single method dominates universally—practitioners must consider anticipated data characteristics\.

3\. Positivity Violations Render Weighting Methods Invalid

The severe overlap violations in retail long\-tail distributions \(90% vs\. 1% tracking probabilities\) violate the Positivity Assumption—a fundamental requirement for valid causal inference\. Even an oracle IPW specification with perfect structural knowledge achieves 6\.061pp error compared to stratification’s 0\.008pp \(758×\\timesworse\) under step\-function scenarios\. This demonstrates that IPW’s poor performance reflects operationoutside its theoretical design enveloperather than methodological inferiority\. When groups barely overlap, no amount of flexible modeling can manufacture valid counterfactual predictions\. The extreme weight ratio \(681,289:1\) is not a failure of IPW—it is a mathematical signal that the Positivity Assumption is violated\.

4\. Robustness Testing Strengthens Confidence

By testing worst\-case scenarios \(misaligned boundaries, polynomial relationships\), we showcase that stratification’s advantages reflect genuine methodological properties rather than tautological design\. The one scenario where IPW wins \(polynomials\) validates our intellectual honesty and strengthens confidence in the three scenarios where stratification dominates\.

Practical Recommendations:

- •Default to stratificationwhen selection mechanisms likely involve rank thresholds, step changes, or moderate non\-linearity
- •Consider flexible IPW \(splines\)when relationships are known to be smooth polynomials without structural breaks
- •Avoid IPW entirelywhen selection probabilities differ by orders of magnitude \(e\.g\., 90% vs\. 1%\)—no amount of flexible modeling overcomes poor overlap
- •Use diagnostics as decision rules:SMD\>\>0\.10 or max/median weight ratios\>\>100:1 after weighting signal IPW failure

The Bottom Line:In retail, intelligence with rank\-based selection and severe long\-tail distributions, stratification provides a safer engineering choice not because weighting methods are inferior, but because the data structure violates their foundational assumptions\. When selection probabilities differ by orders of magnitude \(90% vs\. 1%\), practitioners are operating outside the envelope where IPW is theoretically valid—making coarse stratification the principled alternative\.

## ACKNOWLEDGEMENTS

This work was made possible by the support of Walmart Tech leadership Prasad Savadi, Ashish Gupta, Rahul Ghosh, and Tim Jacobs\. Additionally, special thanks to Feifei Pan and Inderdeep Kaur for providing strategic insights\. Gemini was used during the drafting process to enhance readability\.

## REFERENCES

- \[1\]C\. Anderson\(2006\)The long tail: why the future of business is selling less of more\.Hyperion,New York\.Cited by:[item 2](https://arxiv.org/html/2608.26156#S3.I1.i2.p2.1),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[2\]P\. C\. Austin\(2011\)An introduction to propensity score methods for reducing the effects of confounding in observational studies\.Multivariate Behavioral Research46\(3\),pp\. 399–424\.Cited by:[§3\.4](https://arxiv.org/html/2608.26156#S3.SS4.p1.3),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[3\]S\. R\. Cole and M\. A\. Hernán\(2008\)Constructing inverse probability weights for marginal structural models\.American Journal of Epidemiology168\(6\),pp\. 656–664\.Cited by:[§2](https://arxiv.org/html/2608.26156#S2.p2.1),[§3\.2](https://arxiv.org/html/2608.26156#S3.SS2.p2.2),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[4\]J\. C\. Deville and C\. E\. Särndal\(1992\)Calibration estimators in survey sampling\.Journal of the American Statistical Association87\(418\),pp\. 376–382\.Cited by:[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[5\]D\. A\. Freedman and R\. A\. Berk\(2008\)Weighting regressions by propensity scores\.Evaluation Review32\(4\),pp\. 392–409\.Cited by:[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[6\]M\. J\. Funk, D\. Westreich, C\. Wiesen, T\. Stürmer, M\. A\. Brookhart, and M\. Davidian\(2011\)Doubly robust estimation of causal effects\.American Journal of Epidemiology173\(7\),pp\. 761–767\.Cited by:[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[7\]J\. Hainmueller\(2012\)Entropy balancing for causal effects: a multivariate reweighting method to produce balanced samples in observational studies\.Political Analysis20\(1\),pp\. 25–46\.Cited by:[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[8\]D\. G\. Horvitz and D\. J\. Thompson\(1952\)A generalization of sampling without replacement from a finite universe\.Journal of the American Statistical Association47\(260\),pp\. 663–685\.Cited by:[§2](https://arxiv.org/html/2608.26156#S2.p1.1),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[9\]J\. D\. Kang and J\. L\. Schafer\(2007\)Demystifying double robustness: a comparison of alternative strategies for estimating a population mean from incomplete data\.Statistical Science22\(4\),pp\. 523–539\.Cited by:[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[10\]G\. King and R\. Nielsen\(2019\)Why propensity scores should not be used for matching\.Political Analysis27\(4\),pp\. 435–454\.Cited by:[§2](https://arxiv.org/html/2608.26156#S2.p3.1),[§5\.1](https://arxiv.org/html/2608.26156#S5.SS1.p5.1),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[11\]B\. K\. Lee, J\. Lessler, and E\. A\. Stuart\(2011\)Weight trimming and propensity score weighting\.PloS One6\(3\),pp\. e18174\.Cited by:[§3\.2](https://arxiv.org/html/2608.26156#S3.SS2.p2.2),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[12\]F\. Li, K\. L\. Morgan, and A\. M\. Zaslavsky\(2018\)Balancing covariates via propensity score weighting\.Journal of the American Statistical Association113\(521\),pp\. 390–400\.Cited by:[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[13\]J\. K\. Lunceford and M\. Davidian\(2004\)Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study\.Statistics in Medicine23\(19\),pp\. 2937–2960\.Cited by:[§2](https://arxiv.org/html/2608.26156#S2.p2.1),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[14\]R\. A\. Matsouaka and Y\. Zhou\(2024\)Causal inference in the absence of positivity: the role of overlap weights\.Biometrical Journal66\(4\),pp\. e2300156\.External Links:[Document](https://dx.doi.org/10.1002/bimj.202300156)Cited by:[§2](https://arxiv.org/html/2608.26156#S2.p2.1),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[15\]P\. R\. Rosenbaum and D\. B\. Rubin\(1983\)The central role of the propensity score in observational studies for causal effects\.Biometrika70\(1\),pp\. 41–55\.Cited by:[§2](https://arxiv.org/html/2608.26156#S2.p1.1),[§3\.2](https://arxiv.org/html/2608.26156#S3.SS2.p1.6),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[16\]P\. R\. Rosenbaum\(2002\)Observational studies\.2nd edition,Springer,New York\.Cited by:[§5\.3](https://arxiv.org/html/2608.26156#S5.SS3.p1.1),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[17\]M\. S\. Schuler and S\. Rose\(2017\)Targeted maximum likelihood estimation for causal inference in observational studies\.American Journal of Epidemiology185\(1\),pp\. 65–73\.Cited by:[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.
- \[18\]E\. A\. Stuart\(2010\)Matching methods for causal inference: a review and a look forward\.Statistical Science25\(1\),pp\. 1–21\.Cited by:[§2](https://arxiv.org/html/2608.26156#S2.p3.1),[§5\.1](https://arxiv.org/html/2608.26156#S5.SS1.p5.1),[Selection Bias Correction in Retail Intelligence](https://arxiv.org/html/2608.26156#p1.1)\.