Benchmarking Instance-Dependent Label Noise with Controlled Corruptions

# [Experiment, Analysis, and Benchmark] Benchmarking Instance-Dependent Label Noise with Controlled Corruptions
Source: [https://arxiv.org/html/2606.14965](https://arxiv.org/html/2606.14965)
###### Abstract

Synthetic instance\-dependent label noise \(IDN\) benchmarks are widely used to evaluate noisy\-label learning methods, yet existing approaches typically generate noise through imperfect annotators or classifier raters, leaving the source of ambiguity implicit\. We introduceCILN, a benchmark\-generation framework that creates IDN through controlled input corruptions\. A diverse voter pool labels corrupted instances, producing benchmark datasets in which both the source and severity of ambiguity are explicit and controllable\. Using CIFAR\-10, MNIST, and Adult, we construct 90 benchmark settings spanning multiple corruption families and severity levels\. Our experiments show that the resulting benchmarks exhibit genuine instance\-dependent noise, provide diverse confusion structures, and, on CIFAR\-10, can produce label distributions that are closer to human uncertainty than an existing synthetic IDN benchmark\. We further demonstrate that corruption\-mediated IDN can expose failure modes of popular noisy\-label learning methods, including Co\-Teaching and DivideMix, that are not observed under comparable levels of rater\-fallibility noise\. These findings suggest that noise structure, not only noise rate, plays an important role in benchmark difficulty and algorithm behavior\. By making ambiguity generation explicit and controllable,CILNprovides a complementary benchmarking framework for studying noisy\-label learning under diverse sources of instance difficulty\.

## IIntroduction

Modern machine learning systems rely on large labeled datasets, yet labels are often imperfect due to annotation mistakes, ambiguous examples, weak supervision pipelines, or automatically generated annotations\[[19](https://arxiv.org/html/2606.14965#bib.bib5),[21](https://arxiv.org/html/2606.14965#bib.bib6),[7](https://arxiv.org/html/2606.14965#bib.bib7)\]\. Such label noise can substantially degrade downstream performance by reducing predictive accuracy and introducing bias\[[26](https://arxiv.org/html/2606.14965#bib.bib11),[11](https://arxiv.org/html/2606.14965#bib.bib8),[16](https://arxiv.org/html/2606.14965#bib.bib27)\]\. While early work primarily studied*symmetric*and*class\-conditional*noise, these assumptions often fail in practice because labeling errors depend not only on the class but also on the particular example being labeled\. Difficult or ambiguous instances are naturally more likely to receive incorrect labels than easy ones\. This more realistic setting, in which corruption probabilities depend on both the class and the individual instance, is commonly referred to as*instance\-dependent label noise*\(IDN\)\[[29](https://arxiv.org/html/2606.14965#bib.bib14),[3](https://arxiv.org/html/2606.14965#bib.bib13)\]\.

Despite its practical importance, evaluating methods under IDN remains challenging due to the limited availability of realistic and controllable benchmarks\. Existing approaches generally fall into two categories\. Early synthetic methods generate IDN through instance\-specific corruption probabilities or feature\-space transformations\[[29](https://arxiv.org/html/2606.14965#bib.bib14),[3](https://arxiv.org/html/2606.14965#bib.bib13)\], making some examples more likely than others to receive incorrect labels\. More recently, pseudo\-labeling approaches generate IDN using diverse classifiers acting as synthetic annotators, where disagreement among model predictions induces label uncertainty\[[6](https://arxiv.org/html/2606.14965#bib.bib4)\]\. These approaches provide practical mechanisms for generating IDN and can produce noise levels and uncertainty patterns that resemble human annotation behavior\.

While these approaches provide practical mechanisms for generating IDN, the source of uncertainty remains largely implicit\. In feature\-space approaches, label errors arise from latent transformations that are often difficult to interpret\. In pseudo\-labeling approaches, uncertainty emerges through disagreement among synthetic annotators, and noise levels are typically controlled by modifying properties of the annotators themselves, such as their training procedure or predictive quality\. As a result, although these benchmarks can generate realistic instance\-dependent noise, the ambiguity responsible for a particular label error is often difficult to identify or manipulate directly\. Consequently, existing benchmarks provide limited support for systematically studying how specific ambiguity mechanisms—such as blur, occlusion, missing information, or measurement errors—affect classifier disagreement, label uncertainty, and downstream learning behavior\.

In this paper, we introduceCILN\(*Corruption\-Induced Label Noise*\), a framework for generating instance\-dependent label\-noise benchmarks through controlled input corruptions\. A*corruption*is a transformation that degrades or alters the original input while preserving its semantic content, such as blur, noise, geometric distortions, missing values, or attribute perturbations\. Starting from clean instances,CILNapplies controlled corruptions at varying severity levels\[[9](https://arxiv.org/html/2606.14965#bib.bib1),[18](https://arxiv.org/html/2606.14965#bib.bib2),[27](https://arxiv.org/html/2606.14965#bib.bib22)\]and measures their effect on the predictions of a diverse voter pool\. For image datasets, we consider corruption families such as noise, blur, weather, and geometric transformations, while for tabular datasets we consider missing\-data and value corruptions\. The corrupted instances are then evaluated by the voter pool to generate noisy labels and soft label distributions\. As a result, label uncertainty emerges from observable and explicitly controlled ambiguity mechanisms whose type and severity are known\.

Unlike existing IDN benchmarks, which primarily vary the labeling process to generate noisy labels,CILNgenerates label uncertainty by modifying the input itself through controlled corruptions\. This shifts the focus from controlling how noisy labels are produced to controlling why they arise in the first place\. As a result,CILNenables systematic studies of how specific corruption mechanisms affect classifier disagreement, label uncertainty, and downstream learning behavior\. Moreover, because the type and severity of each corruption are known, the resulting label transitions can often be traced to specific changes in the input\. The following example illustrates this property\.

###### Example 1\.

Figure[1](https://arxiv.org/html/2606.14965#S1.F1)shows four representativeCILNexamples—two fromCIFAR10and two fromMNIST–under increasing corruption severity\. Each column follows a single example from mild to severe corruption for a fixed corruption type\. The examples illustrate how controlled corruptions can progressively increase classification difficulty and ultimately change the most plausible label\.

- •\(a\)CIFAR10,contrast, true: dog\.As contrast drops, the fine fur, snout, and ear details that distinguish the animal as a dog gradually disappear, leaving primarily its silhouette\. Because cats and dogs share similar body shapes, the corrupted image increasingly resembles a black cat, leading to a plausible dog→\\rightarrowcat transition\.
- •\(b\)CIFAR10,gaussian, true: bird\.The original image already contains visual cues compatible with multiple classes\. As Gaussian noise increases, the bird becomes harder to distinguish from the noisy background, and the resulting appearance increasingly resembles a frog photographed in vegetation, producing a bird→\\rightarrowfrog transition\.
- •\(c\)MNIST,impulse, true: 5\.Impulse noise progressively fills gaps in the digit\. At moderate severity, the lower curve begins to resemble part of an 8, while at higher severity both open regions become closed\. The resulting digit is visually consistent with an 8, leading to a 5→\\rightarrow8 transition\.
- •\(d\)MNIST,shot, true: 4\.Shot noise thickens the strokes of the digit and gradually fills the open region at the top of the 4\. As the rectangular opening closes and becomes more loop\-like, the digit increasingly resembles a 9, leading to a plausible 4→\\rightarrow9 transition\.

![Refer to caption](https://arxiv.org/html/2606.14965v1/x1.png)

![Refer to caption](https://arxiv.org/html/2606.14965v1/x2.png)

![Refer to caption](https://arxiv.org/html/2606.14965v1/x3.png)

\(a\)contrast
![Refer to caption](https://arxiv.org/html/2606.14965v1/x4.png)

![Refer to caption](https://arxiv.org/html/2606.14965v1/x5.png)

![Refer to caption](https://arxiv.org/html/2606.14965v1/x6.png)

\(b\)gaussian
![Refer to caption](https://arxiv.org/html/2606.14965v1/x7.png)

![Refer to caption](https://arxiv.org/html/2606.14965v1/x8.png)

![Refer to caption](https://arxiv.org/html/2606.14965v1/x9.png)

\(c\)impulse
![Refer to caption](https://arxiv.org/html/2606.14965v1/x10.png)

![Refer to caption](https://arxiv.org/html/2606.14965v1/x11.png)

![Refer to caption](https://arxiv.org/html/2606.14965v1/x12.png)

\(d\)shot

Figure 1:Examples of corruption\-induced label ambiguity under increasing corruption severity \(top to bottom\)\.These examples illustrate that corruption\-induced label transitions arise from identifiable changes in the input rather than arbitrary label flips\. As corruption severity increases, evidence supporting the true class gradually weakens while features associated with alternative classes become more prominent\. Consequently, the resulting label uncertainty can often be traced to a specific corruption mechanism, providing a transparent and controllable source of instance\-dependent label noise\.

The explicit control provided byCILNenables several forms of analysis that are difficult or impossible to perform with existing IDN benchmarks\. First, because the source and severity of each corruption are known, benchmark instances can be generated under specific corruption mechanisms, allowing systematic comparisons across different sources of input difficulty\. Second, an optional*clean\-start*configuration retains only instances that are unanimously classified correctly before corruption, enabling corruption\-induced label uncertainty to be isolated from pre\-existing ambiguity in the original data\. Finally, by varying corruption types and severity levels,CILNproduces benchmark instances with different noise characteristics and levels of instance dependence, enabling controlled studies of how these factors affect downstream learning methods\.

These capabilities also distinguishCILNfrom existing human\-uncertainty and pseudo\-label benchmarks such as CIFAR10\-H\[[22](https://arxiv.org/html/2606.14965#bib.bib9)\]and Gu et al\.\[[6](https://arxiv.org/html/2606.14965#bib.bib4)\]\. Existing benchmarks primarily generate label uncertainty through human disagreement or synthetic annotators, whereasCILNgenerates uncertainty through controlled modifications of the input itself\. As a result,CILNenables studies not only of how much label noise is present, but also of how specific corruption mechanisms contribute to classifier disagreement, label uncertainty, and downstream learning behavior\. The two perspectives are complementary: one focuses on the labeling process, while the other focuses on the characteristics of the data that make labeling difficult\.

We evaluateCILNon CIFAR\-10, MNIST, and Adult, covering both image and tabular domains\. First, we demonstrate that controlled corruptions generate genuinely instance\-dependent label noise and that the degree of instance dependence can be systematically influenced through corruption type and severity\. Second, we show that particular corruption settings induce confusion and uncertainty patterns whose distributions are statistically closer to human annotation uncertainty, such as that observed in CIFAR10\-H\[[22](https://arxiv.org/html/2606.14965#bib.bib9)\], than existing synthetic IDN benchmark\-generation approaches\. Third, we evaluate state\-of\-the\-art noisy\-label learning methods, including Co\-Teaching\[[7](https://arxiv.org/html/2606.14965#bib.bib7)\]and DivideMix\[[14](https://arxiv.org/html/2606.14965#bib.bib18)\], on the generated benchmarks and show that their behavior depends not only on the overall noise rate but also on the underlying corruption mechanism\. Together, these results demonstrate that controlled corruptions provide a practical and complementary approach to constructing realistic, interpretable, and mechanism\-controlled IDN benchmarks for evaluating methods under instance\-dependent label noise\.

Our technical contributions are summarized as follows:

- •We proposeCILN, a corruption\-mediated framework for generating mechanism\-controlled IDN benchmarks\. The framework introduces a clean\-start condition and explicit corruption settings that establish a direct link between the source of ambiguity and the resulting label uncertainty\.
- •We show that the proposed framework generates genuine IDN and that the degree of instance dependence can be systematically controlled by the type and severity of corruption\.
- •We demonstrate that controlled corruptions induce diverse confusion and uncertainty patterns and that, under appropriate corruption settings, the resulting label distributions are statistically closer to human annotation uncertainty than existing synthetic IDN benchmark\-generation approaches\.
- •We evaluate state\-of\-the\-art noisy\-label learning methods on the generated benchmarks and show that their performance depends not only on the overall noise rate but also on the underlying ambiguity mechanism, highlighting benchmark characteristics that are not captured by traditional synthetic noise models\.

## IIPreliminaries

This section introduces the notation and concepts used in the paper\. Section[II\-A](https://arxiv.org/html/2606.14965#S2.SS1)reviews the classification setting and label\-noise models, Section[II\-B](https://arxiv.org/html/2606.14965#S2.SS2)introduces voter\-generated label distributions, and Section[II\-C](https://arxiv.org/html/2606.14965#S2.SS3)defines corruption settings\.

### II\-AClassification Setting and Noise Models

We consider aKK\-class classification problem with input space𝒳\\mathcal\{X\}and label space𝒴=\{1,…,K\}\\mathcal\{Y\}=\\\{1,\\ldots,K\\\}\. LetD=\{\(xi,yi\)\}i=1ND=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}denote a clean dataset, wherexi∈𝒳x\_\{i\}\\in\\mathcal\{X\}is an input sample andyi∈𝒴y\_\{i\}\\in\\mathcal\{Y\}is its ground\-truth class label\. Depending on the dataset,xix\_\{i\}may represent an image or a tabular record\.

Label noise occurs when the observed labely~i\\tilde\{y\}\_\{i\}differs from the clean labelyiy\_\{i\}\. The noisy dataset is written asD~=\{\(xi,y~i\)\}i=1N\\tilde\{D\}=\\\{\(x\_\{i\},\\tilde\{y\}\_\{i\}\)\\\}\_\{i=1\}^\{N\}\. Existing work commonly studies several noise\-generation settings\. In symmetric label noise, labels are randomly flipped with fixed probabilityδ\\delta:

P​\(y~=j∣y=i\)=\{1−δ,j=i,δK−1,j≠i\.P\(\\tilde\{y\}=j\\mid y=i\)=\\begin\{cases\}1\-\\delta,&j=i,\\\\\[5\.69054pt\] \\dfrac\{\\delta\}\{K\-1\},&j\\neq i\.\\end\{cases\}
Class\-conditional noise \(CCN\), assumes that the corruption process depends only on the true class label and not on the input instance\. Formally,

P​\(y~=j∣x,y=i\)=P​\(y~=j∣y=i\)\.P\(\\tilde\{y\}=j\\mid x,y=i\)=P\(\\tilde\{y\}=j\\mid y=i\)\.
Under this assumption, the corruption process can be represented by a transition matrixT∈ℝK×KT\\in\\mathbb\{R\}^\{K\\times K\}:

P​\(y~=j∣y=i\)=Ti​j,P\(\\tilde\{y\}=j\\mid y=i\)=T\_\{ij\},
where each row ofTTspecifies the corruption probabilities for classii\. Both symmetric label noise and CCN ignore the properties of individual examples and assume that all instances from the same class share the same corruption behavior\.

In practical settings, however, the probability of label corruption often depends on the input itself\. Harder examples, ambiguous inputs, or degraded observations are naturally more likely to receive incorrect labels\. IDN relaxes the CCN assumption by allowing the corruption process to depend on both the input instance and the true label\. Consequently, the noisy\-label distribution is modeled asP​\(y~∣x,y\)P\(\\tilde\{y\}\\mid x,y\), allowing examples from the same class to have different probabilities of receiving incorrect labels depending on their individual characteristics and difficulty\.

### II\-BLabel Distributions and Voter Pools

In many classification tasks, uncertainty cannot be adequately represented by a single class label\. Instead, uncertainty can be represented as a label distribution over theKKclasses\. For an inputxx, we write such a distribution as

p​\(y∣x\)=\(p1,…,pK\),pk≥0​∀k,∑k=1Kpk=1,p\(y\\mid x\)=\(p\_\{1\},\\ldots,p\_\{K\}\),\\qquad p\_\{k\}\\geq 0\\;\\;\\forall k,\\qquad\\sum\_\{k=1\}^\{K\}p\_\{k\}=1,
wherepkp\_\{k\}denotes the probability assigned to classkk\.

Label distributions arise naturally when multiple annotators or classifiers provide labels for the same instance\. LetV=f1,…,fMV=\{f\_\{1\},\\ldots,f\_\{M\}\}denote a pool ofMMvoters\. Given an inputxx, each voter produces either a hard label or a predictive distribution over classes\. Collectively, these predictions characterize the uncertainty associated withxx\.

The predictions of multiple voters can be aggregated into a single label distribution using an aggregation operator\. When voters provide hard labels, common strategies include majority voting and empirical vote frequencies\. When voters provide predictive distributions, aggregation can be performed directly in distribution space\. The resulting aggregate distribution summarizes the collective uncertainty of the voter pool and provides a richer representation than a single hard label\.

InCILN, each voter produces a predictive distributionp\(m\)​\(y∣x\)p^\{\(m\)\}\(y\\mid x\)\. For each instance, the benchmark releases an aggregated label distributionp¯i​\(y∣x\)\\bar\{p\}\_\{i\}\(y\\mid x\)obtained by arithmetic averaging:

p¯i​\(y∣x\)=1M​∑m=1Mpi\(m\)​\(y∣x\)\.\\displaystyle\\bar\{p\}\_\{i\}\(y\\mid x\)=\\frac\{1\}\{M\}\\sum\_\{m=1\}^\{M\}p\_\{i\}^\{\(m\)\}\(y\\mid x\)\.\(1\)
This aggregation admits a natural probabilistic interpretation:p¯i\\bar\{p\}\_\{i\}corresponds to the expected label distribution obtained by selecting a voter uniformly at random from the pool\. In addition top¯i\\bar\{p\}\_\{i\},CILNstores the individual voter distributions\{pi\(m\)\}m=1M\\\{p\_\{i\}^\{\(m\)\}\\\}\_\{m=1\}^\{M\}, which enable alternative aggregation strategies and analyses of voter disagreement and benchmark noise characteristics\.

### II\-CCorruption Families

The corruption types in our benchmark are drawn from established robustness benchmarks and data\-quality literature\[[9](https://arxiv.org/html/2606.14965#bib.bib1),[18](https://arxiv.org/html/2606.14965#bib.bib2),[27](https://arxiv.org/html/2606.14965#bib.bib22)\], allowing us to study how different forms of data degradation induce ambiguity and disagreement among classifiers\. We consider both image and tabular corruptions to demonstrate that the framework is not restricted to a single modality\.

A*corruption setting*is a pair

whereccdenotes a*corruption type*andℓ\\elldenotes a*severity level*\. Corruption types specify the mechanism of degradation, while severity levels control the strength of the degradation\. Following prior robustness benchmarks\[[9](https://arxiv.org/html/2606.14965#bib.bib1),[18](https://arxiv.org/html/2606.14965#bib.bib2)\], each corruption type is associated with a set of severity levels that control the strength of the degradation\. Throughout this work, we use severity levelsℓ∈\{1,3,5\}\\ell\\in\\\{1,3,5\\\}, corresponding to mild, moderate, and severe corruption\.

#### II\-C1Image Corruptions

For image data, we consider six corruption families commonly used in robustness benchmarks\[[9](https://arxiv.org/html/2606.14965#bib.bib1),[18](https://arxiv.org/html/2606.14965#bib.bib2)\]\.

- •Noisecorruptions perturb pixel values through random disturbances\. Representative corruption types includegaussian\-noise\(additive Gaussian noise\),shot\-noise\(Poisson noise caused by the discrete nature of light\),impulse\-noise\(salt\-and\-pepper style corruption caused by sparse pixel errors\), andspatter\(random blotches that partially occlude the image\)\.
- •Blurcorruptions reduce spatial detail and edge information\. Representative corruption types includedefocus\-blur\(out\-of\-focus lens blur\),glass\-blur\(local pixel shuffling resembling frosted glass\),motion\-blur\(blur caused by camera or object motion\), andzoom\-blur\(blur induced by rapid zooming\)\.
- •Geometriccorruptions alter the spatial structure of the image\. Representative corruption types includeelastic\(local elastic deformation of image regions\),rotation\(in\-plane rotation\),shear\(slanting the image along one axis\),translation\(shifting the image spatially\), andscaling\(resizing the image content\)\.
- •Weathercorruptions simulate environmental effects\. Representative corruption types includefog\(haze that reduces visibility\),frost\(ice\-crystal artifacts on the image\), andsnow\(snow\-like visual occlusion\)\.
- •Digitalcorruptions mimic acquisition and post\-processing artifacts\. Representative corruption types includebrightness\(global intensity shifts\),contrast\(changes in intensity separation\),jpeg\(lossy compression artifacts\), andpixelate\(loss of resolution through blocky upsampling\)\.
- •Structuralcorruptions introduce structured overlays or shape distortions\. Representative corruption types includecanny\-edges\(edge\-map transformation using the Canny detector\),dotted\-line\(superimposed dotted strokes\),stripe\(a vertical stripe overlay\), andzigzag\(superimposed zigzag strokes\)\.

These corruption families span a diverse range of visual degradations, from low\-level pixel perturbations to large\-scale structural transformations\.

#### II\-C2Tabular Corruptions

For tabular data, we adopt corruption operators inspired by the Jenga benchmark\[[27](https://arxiv.org/html/2606.14965#bib.bib22)\]\. Unlike image corruptions, tabular corruptions target the quality and completeness of attributes\. We consider two broad families:

- •Missingcorruptions remove information by replacing observed values with NaN\. We consider three standard missingness mechanisms: Missing Completely At Random \(mcar\), where values are removed independently with a fixed probability; Missing At Random \(mar\), where the missingness probability depends on other observed attributes; and Missing Not At Random \(mnar\), where the missingness probability depends on the value being removed\.
- •Valuecorruptions alter observed values through perturbations such as noise injection, scaling errors, unit inconsistencies, swapped values, and other feature\-level distortions\.

These corruptions reflect common data\-quality issues in real\-world tabular datasets\. The exact corruption types and severity settings used for each benchmark instantiation are described in Section[III\-B](https://arxiv.org/html/2606.14965#S3.SS2)\.

## IIIBenchmark Construction

Our goal is to generate datasets in which label uncertainty arises from controlled and observable ambiguity mechanisms\.CILNachieves this by combining controlled input corruptions with a pool of voters\. Section[III\-A](https://arxiv.org/html/2606.14965#S3.SS1)presents the general generation procedure, while Section[III\-B](https://arxiv.org/html/2606.14965#S3.SS2)describes dataset\-specific instantiations\.

### III\-ABenchmark Generation Algorithm

LetD=\{\(xi,yi\)\}i=1ND=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}denote a clean labeled dataset,τ=\(c,ℓ\)\\tau=\(c,\\ell\)a corruption setting, andV=\{f1,…,fM\}V=\\\{f\_\{1\},\\ldots,f\_\{M\}\\\}the voter pool\. For a fixed corruption settingτ\\tau,CILNproduces a benchmark instance

Bτ=\{\(xi,x~i,yi,p¯i,τ\)\}i=1N,\\displaystyle B\_\{\\tau\}=\\left\\\{\\bigl\(x\_\{i\},\\tilde\{x\}\_\{i\},y\_\{i\},\\bar\{p\}\_\{i\},\\tau\\bigr\)\\right\\\}\_\{i=1\}^\{N\},\(2\)
wherex~i\\tilde\{x\}\_\{i\}denotes the corrupted input, andp¯i\\bar\{p\}\_\{i\}denotes the aggregated voter distribution\. The benchmark\-generation procedure is summarized in Algorithm[1](https://arxiv.org/html/2606.14965#alg1)\.

Algorithm 1CILNBenchmark Generation\.1:Input:Clean dataset

D=\{\(xi,yi\)\}i=1ND=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}, corruption setting

τ=\(c,ℓ\)\\tau=\(c,\\ell\), voter pool

V=\{f1,…,fM\}V=\\\{f\_\{1\},\\ldots,f\_\{M\}\\\},

cleanstart∈\{false,true\}\\text\{cleanstart\}\\in\\\{\\text\{false\},\\text\{true\}\\\}
2:Output:Benchmark instance

BτB\_\{\\tau\}
3:

ℐ←\{1,…,N\};\\mathcal\{I\}\\leftarrow\\\{1,\.\.\.,N\\\};
4:ifcleanstartthen

5:for

i∈ℐi\\in\\mathcal\{I\}and

m=1,…,Mm=1,\\ldots,Mdo

6:

y^\(m\)←arg⁡maxc⁡fm​\(xi\)​\[c\];\\hat\{y\}^\{\(m\)\}\\leftarrow\\arg\\max\_\{c\}f\_\{m\}\(x\_\{i\}\)\[c\];
7:if

y^\(m\)≠yi\\hat\{y\}^\{\(m\)\}\\neq y\_\{i\}then

ℐ←ℐ∖\{i\};\\mathcal\{I\}\\leftarrow\\mathcal\{I\}\\setminus\\\{i\\\};
8:

Bτ←∅;B\_\{\\tau\}\\leftarrow\\emptyset;
9:for

i∈ℐi\\in\\mathcal\{I\}do

10:

x~i←C​\(xi;τ\);\\tilde\{x\}\_\{i\}\\leftarrow C\(x\_\{i\};\\tau\);
11:for

m=1,…,Mm=1,\\ldots,Mdo

12:

pi\(m\)←fm​\(x~i\)p\_\{i\}^\{\(m\)\}\\leftarrow f\_\{m\}\(\\tilde\{x\}\_\{i\}\)
13:

p¯i←1M​∑m=1Mpi\(m\);\\bar\{p\}\_\{i\}\\leftarrow\\frac\{1\}\{M\}\\sum\_\{m=1\}^\{M\}p\_\{i\}^\{\(m\)\};
14:

Bτ←Bτ∪\{\(xi,x~i,yi,p¯i,τ\)\};B\_\{\\tau\}\\leftarrow B\_\{\\tau\}\\cup\\\{\(x\_\{i\},\\tilde\{x\}\_\{i\},y\_\{i\},\\bar\{p\}\_\{i\},\\tau\)\\\};
15:return

BτB\_\{\\tau\}

The algorithm consists of three components: a corruption operator, a voter pool, and a label aggregation mechanism, together with an optional filtering step applied before benchmark generation\. Whencleanstartis enabled, the algorithm first restricts the index setℐ\\mathcal\{I\}to examples whose clean inputxix\_\{i\}is unanimously assigned its ground\-truth labelyiy\_\{i\}by all voters\. Consequently, any disagreement observed after corruption can be attributed to the corruption operatorC​\(⋅;τ\)C\(\\cdot;\\tau\)rather than pre\-existing voter disagreement on the clean example\. Since examples exhibiting uncertainty before corruption are removed, the resulting benchmark typically exhibits a lower aggregate noise rate and isolates corruption\-induced label uncertainty\. We usecleanstartwhen the goal is to study the effect of a corruption settingτ\\tauin isolation; users may disable it \(cleanstart=false\\text\{cleanstart\}=\\text\{false\}\) when retaining the full dataset or preserving class balance is more important\.

For each retained instance, the corruption operatorC​\(⋅;τ\)C\(\\cdot;\\tau\)generates a corrupted versionx~i\\tilde\{x\}\_\{i\}according to the specified corruption type and severity level \(Line[10](https://arxiv.org/html/2606.14965#alg1.l10)\)\. The corrupted instance is then evaluated by the voter pool, producing predictive distributionspi\(1\),…,pi\(M\)\{p\_\{i\}^\{\(1\)\},\\ldots,p\_\{i\}^\{\(M\)\}\}\(Line[12](https://arxiv.org/html/2606.14965#alg1.l12)\)\. Disagreement among these distributions captures the uncertainty induced by the applied corruption\.

The voter predictions are subsequently aggregated into a single label distributionp¯i\\bar\{p\}\_\{i\}\(Line[13](https://arxiv.org/html/2606.14965#alg1.l13)\), which serves as the benchmark soft label\. Following Section[II\-B](https://arxiv.org/html/2606.14965#S2.SS2), we use arithmetic mean aggregation, sop¯i\\bar\{p\}\_\{i\}corresponds to the expected label distribution obtained by selecting a voter uniformly at random from the pool\.

Some downstream learning methods require a single training label per instance\. For those experiments, we derive hard labels from the voter predictions using the same uniform\-rater sampling protocol employed by PL\-IDN\[[6](https://arxiv.org/html/2606.14965#bib.bib4)\]\. Specifically, for each instance, we select one voter uniformly at random and use its prediction as the observed label\.

The algorithm returns a benchmark instanceBτB\_\{\\tau\}consisting of tuples\(xi,x~i,yi,p¯i,τ\)\(x\_\{i\},\\tilde\{x\}\_\{i\},y\_\{i\},\\bar\{p\}\_\{i\},\\tau\)for all retained examples\. Thus, each benchmark instance contains the clean input, corrupted input, ground\-truth label, aggregated label distribution, and corruption metadata\. Different corruption settingsτ\\tauproduce different benchmark instances, while applying the algorithm to a different dataset, corruption family, or voter pool produces a new benchmark suite\.

### III\-BDatasets and Benchmark Instantiations

We instantiate the algorithm of Section[III\-A](https://arxiv.org/html/2606.14965#S3.SS1)on three datasets spanning image and tabular domains:CIFAR10,MNIST, andAdult\. For each dataset, we construct a diverse voter pool and generate benchmark instances using corruption settingsτ=\(c,ℓ\)\\tau=\(c,\\ell\), where corruption typesccare selected from the families described in Section[II\-C](https://arxiv.org/html/2606.14965#S2.SS3)and severity levels are fixed toℓ∈1,3,5\\ell\\in\{1,3,5\}\. Table[I](https://arxiv.org/html/2606.14965#S3.T1)summarizes the resulting benchmark suites\.

TABLE I:Summary of the benchmark instantiations used in this work\.*Label noise rate \(δ\\delta\)*denotes the average disagreement between voter predictions and ground\-truth labels\. The reported range spans the released settings within each benchmark\.- •CIFAR10: The voter pool consists of ResNet\-20\[[8](https://arxiv.org/html/2606.14965#bib.bib28)\], WRN\-28\-10\[[31](https://arxiv.org/html/2606.14965#bib.bib29)\], DeiT3\-Small\[[28](https://arxiv.org/html/2606.14965#bib.bib30)\], and CLIP ViT\-B/32\[[24](https://arxiv.org/html/2606.14965#bib.bib31)\], spanning convolutional networks, vision transformers, and multimodal foundation models\. Corruption types are drawn from CIFAR\-C\[[9](https://arxiv.org/html/2606.14965#bib.bib1)\]: 3 Noise \(gaussian,shot,impulse\), 4 Blur \(defocus,glass,motion,zoom\), 3 Weather \(fog,frost,snow\), 1 Geometric \(elastic\), and 4 Digital \(brightness,contrast,jpeg,pixelate\) corruptions\. Combining 15 corruption types with three severity levels yields 45 benchmark settings\.
- •MNIST: The voter pool consists of LeNet\-5\[[12](https://arxiv.org/html/2606.14965#bib.bib32)\], ResNet\-20\[[8](https://arxiv.org/html/2606.14965#bib.bib28)\], a multilayer perceptron \(MLP\), and DeiT3\-Small\[[28](https://arxiv.org/html/2606.14965#bib.bib30)\]\. Corruption types are drawn from MNIST\-C\[[18](https://arxiv.org/html/2606.14965#bib.bib2)\]: 3 Noise \(shot,impulse,spatter\), 2 Blur \(glass,motion\), 4 Geometric \(rotation,shear,translation,scaling\), 2 Weather/Digital \(fog,brightness\), and 4 Structural \(canny\-edges,dotted\-line,stripe,zigzag\) corruptions\. The Structural corruptions are binary operators and therefore contribute one setting each rather than three severity levels\. Consequently, the 11 continuous\-severity corruption types contribute 33 settings, while the 4 Structural corruptions contribute 4 additional settings, yielding 37 candidate settings in total\. In practice, several low\-impact corruption settings produce negligible label uncertainty and are excluded from the final benchmark suite, resulting in 30 released settings\.
- •Adult: The voter pool consists of XGBoost\[[4](https://arxiv.org/html/2606.14965#bib.bib33)\], CatBoost\[[23](https://arxiv.org/html/2606.14965#bib.bib34)\], an RTDL\-tuned MLP\[[5](https://arxiv.org/html/2606.14965#bib.bib26)\], FT\-Transformer\[[5](https://arxiv.org/html/2606.14965#bib.bib26)\], and TabPFN\[[10](https://arxiv.org/html/2606.14965#bib.bib35)\]\. Corruption types include 3 Missing \(mcar,mar,mnar\) and 2 Value \(gaussian\-noise,scaling\) corruptions\. Combining five corruption types with three severity levels yields 15 benchmark settings\. Formar, corruption probabilities depend on the observedsexattribute, while formnarthey depend on the values being corrupted\. Severity controls the fraction of affected rows\. For value corruptions,gaussian\-noiseadds zero\-mean Gaussian noise withσ∈2\.0,3\.5,5\.0\\sigma\\in\{2\.0,3\.5,5\.0\}andscalingmultiplies numerical values by factors of10,100,1000\{10,100,1000\}for severity levelsℓ∈1,3,5\\ell\\in\{1,3,5\}\.

For each corruption settingτ\\tau, the algorithm produces a benchmark instanceBτB\_\{\\tau\}as defined in Section[III\-A](https://arxiv.org/html/2606.14965#S3.SS1)\. Consequently, each benchmark suite consists of benchmark instances generated under different ambiguity mechanisms and severity levels\. Severity is implemented in a corruption\-specific manner, with higher levels corresponding to stronger degradation\. For image datasets, we adopt the severity parameterizations provided by CIFAR\-C\[[9](https://arxiv.org/html/2606.14965#bib.bib1)\]and MNIST\-C\[[18](https://arxiv.org/html/2606.14965#bib.bib2)\]; for tabular datasets, we follow the corruption operators introduced by Jenga\[[27](https://arxiv.org/html/2606.14965#bib.bib22)\]\. Complete corruption parameters, benchmark construction code, and reproducibility artifacts are available at[https://github\.com/sh\-islam/ciln\-bench](https://github.com/sh-islam/ciln-bench)\.

### III\-CLabel Distributions and Confusion Structure

We next examine how controlled input corruptions change the resulting label distributions\. This analysis characterizes the type of benchmark instances produced byCILN: corruptions may preserve the original class balance, introduce moderate distributional shifts, or concentrate noisy labels around a small number of*attractor classes*\. Figures[2](https://arxiv.org/html/2606.14965#S3.F2)and[3](https://arxiv.org/html/2606.14965#S3.F3)show representative examples, and Figure[4](https://arxiv.org/html/2606.14965#S3.F4)summarizes the induced imbalance through sorted class\-frequency profiles and label entropy\.

![Refer to caption](https://arxiv.org/html/2606.14965v1/x13.png)\(a\)CIFAR10brightness
![Refer to caption](https://arxiv.org/html/2606.14965v1/x14.png)\(b\)CIFAR10pixelate
![Refer to caption](https://arxiv.org/html/2606.14965v1/x15.png)\(c\)MNISTbrightness
![Refer to caption](https://arxiv.org/html/2606.14965v1/x16.png)\(d\)CIFAR10gaussian

Figure 2:Examples of corruption\-induced label distributions for image datasets across severity levels\.![Refer to caption](https://arxiv.org/html/2606.14965v1/x17.png)\(a\)Adultmcar
![Refer to caption](https://arxiv.org/html/2606.14965v1/x18.png)\(b\)Adultmnar

Figure 3:Examples of corruption\-induced label distributions forAdultunder different missing\-data mechanisms\.The main observation is that corruption\-induced label noise inCILNis structured rather than arbitrary\. Corruption type determines the*direction*of the shift: some corruptions leave the label distribution close to the clean baseline, while others pull probability mass toward specific classes\. For example, Figure[2](https://arxiv.org/html/2606.14965#S3.F2)[2\(a\)](https://arxiv.org/html/2606.14965#S3.F2.sf1)shows thatbrightnesshas little effect on theCIFAR10label distribution, whereas Figure[2](https://arxiv.org/html/2606.14965#S3.F2)[2\(b\)](https://arxiv.org/html/2606.14965#S3.F2.sf2)shows thatpixelatecreates a strong shift toward a small set of classes\. Severity then controls the*magnitude*of this effect\. As severity increases, distributions often become more concentrated, entropy decreases, and class imbalance becomes stronger\. Thus, corruption type determines the structure of the induced noise, while severity controls its intensity\.

The effect of corruption is also dataset\-dependent\. The same corruption mechanism can produce substantially different distributional shifts on different datasets\. For instance, Figures[2](https://arxiv.org/html/2606.14965#S3.F2)[2\(a\)](https://arxiv.org/html/2606.14965#S3.F2.sf1)and[2](https://arxiv.org/html/2606.14965#S3.F2)[2\(c\)](https://arxiv.org/html/2606.14965#S3.F2.sf3)show thatbrightnessproduces little imbalance onCIFAR10, but substantially concentratesMNISTlabels toward a small subset of classes\. Similarly, Figures[3](https://arxiv.org/html/2606.14965#S3.F3)[3\(a\)](https://arxiv.org/html/2606.14965#S3.F3.sf1)and[3](https://arxiv.org/html/2606.14965#S3.F3)[3\(b\)](https://arxiv.org/html/2606.14965#S3.F3.sf2)show that different missing\-data mechanisms can have markedly different effects on class balance inAdult\. These examples demonstrate that corruption effects are not universal properties of the corruption alone; they depend on how the corruption interacts with the underlying structure of the dataset\.

Attractor classes provide an extreme manifestation of this dataset\-dependent behavior\. Under some corruption–dataset pairs, many different true classes are mapped toward the same predicted class\. InCIFAR10, high\-severity noise corruptions often increase the share of*frog*, while blur corruptions frequently increase the share of*cat*\. InMNIST, pixel\-degradation corruptions such asimpulse\-noiseandbrightnesstend to pull labels toward digit 8, whereas geometric corruptions such asrotationtend to pull labels toward digit 2\. InAdult, missingness often pushes predictions toward the majority class, whilescalingcan shift predictions in the opposite direction\. These attractor patterns reveal that corruption\-induced noise changes not only the amount of label noise, but also its class\-level geometry\.

Figure[4](https://arxiv.org/html/2606.14965#S3.F4)provides a complementary view of these effects through sorted class\-frequency profiles\. Consistent with the severity trends observed above, settings such asCIFAR10pixelate,CIFAR10gaussian\-noise, andMNISTbrightnessbecome progressively more skewed as severity increases, producing lower\-entropy label distributions\. In contrast, settings with weaker distributional effects remain closer to the clean profile\. This visualization reinforces that severity acts as a control knob for the strength of the induced imbalance, while the corruption–dataset pair determines its specific form\.

These results show thatCILNprovides control over both the amount and the structure of label noise\. Corruption type determines the direction of the distributional shift, severity determines its strength, and the dataset governs how these effects are realized\. This characterization helps explain the experimental results in Section[IV](https://arxiv.org/html/2606.14965#S4): methods that perform well on rate\-matchedPL\-IDNcan fail onCILNbecause the challenge is not merely higher noise rates, but fundamentally different noise directions and imbalance structures\.

![Refer to caption](https://arxiv.org/html/2606.14965v1/x19.png)\(a\)CIFAR10pixelate
![Refer to caption](https://arxiv.org/html/2606.14965v1/x20.png)\(b\)MNISTrotation
![Refer to caption](https://arxiv.org/html/2606.14965v1/x21.png)\(c\)CIFAR10gaussian
![Refer to caption](https://arxiv.org/html/2606.14965v1/x22.png)\(d\)MNISTbrightness

Figure 4:Sorted class\-frequency profiles for selected corruption settings\. Each panel compares clean labels with low\- and high\-severity corruptions, with label entropy reported in the legend\. Lower entropy indicates stronger concentration and greater induced imbalance\.

## IVExperimental Evaluation

Our evaluation has three objectives\. First, we investigate whether controlled corruptions produce genuine instance\-dependent label noise and whether the degree of instance dependence can be systematically influenced through the choice of corruption type and severity\. Second, we examine how closely the resulting label distributions resemble human annotation uncertainty, assessing whether corruption\-induced ambiguity can serve as a realistic proxy for human disagreement\. Third, we evaluate the usefulness of the benchmark as a stress test for noisy\-label learning by studying how state\-of\-the\-art methods behave under corruption\-induced label noise\. Together, these experiments assess the realism, controllability, and practical value of the proposed benchmark\.

### IV\-AExperimental Setup

We first describe the competing benchmark used for comparison and then introduce the evaluation measures used throughout the experiments\.

#### IV\-A1Competing Benchmark Generation Methods

We compareCILNagainstPL\-IDN\[[6](https://arxiv.org/html/2606.14965#bib.bib4)\], a widely used benchmark for instance\-dependent label noise\.PL\-IDNis available only forCIFAR10and therefore serves as our primary reference benchmark for evaluating instance dependence, similarity to human annotation uncertainty, and downstream learning behavior\. No comparable benchmark release exists forMNISTorAdult, so for those datasets we report the absolute characteristics of the generated benchmarks\.

The two benchmarks differ in how label noise is generated\.PL\-IDNproduces noisy labels by using imperfect model raters on clean images, whereasCILNgenerates label uncertainty by applying controlled corruptions to the input and aggregating predictions from a voter pool\. Consequently,PL\-IDNcontrols noise through the properties of the raters, whileCILNcontrols noise through the type and severity of corruption\.

PL\-IDNis released in three versions corresponding to low\-, medium\-, and high\-noise regimes, which we denote byPL\-IDN\-L,PL\-IDN\-M, andPL\-IDN\-H, respectively\. Unless otherwise stated,PL\-IDNrefers to the benchmark generation method without reference to a particular noise level\. In contrast,CILNprovides a broader collection of benchmark settings spanning multiple corruption mechanisms and severity levels, allowing users to generate benchmarks with varying degrees and structures of label noise\. When comparingCILNagainstPL\-IDN, we compare eachPL\-IDNrelease with theCILNsettings having the closest observed noise rates\. For reference,PL\-IDN\-L,PL\-IDN\-M, andPL\-IDN\-Hcorrespond to approximately 11%, 19%, and 48% label noise, respectively\.

In addition to the default benchmark release,CILNsupports an optional*clean\-start*configuration \(Algorithm[1](https://arxiv.org/html/2606.14965#alg1)\), which retains only instances that are classified correctly by all voters before corruption\. We denote the default benchmark byCILN\-Sand the clean\-start variant byCILN\-C\. Unless stated otherwise,CILNrefers toCILN\-S\. The clean\-start variant isolates corruption\-induced uncertainty from pre\-existing ambiguity in the original data\. Unless otherwise stated, results are reported onCILN\-S, and selected experiments are repeated onCILN\-Cto assess the robustness of the findings\. BecauseCILN\-Cremoves instances that already exhibit voter disagreement before corruption, it typically produces lower noise rates than the correspondingCILN\-Ssetting\.

#### IV\-A2Evaluation Measures and Evaluation Scenarios

We evaluate the generated benchmarks from three perspectives: \(i\) the degree of instance dependence in the induced label noise, \(ii\) similarity to human annotation uncertainty, and \(iii\) downstream learning performance\.

To quantify instance dependence, we introduce*Noise Transition Heterogeneity*\(nth\), which measures the variability of instance\-specific noise transition distributions within each class\. Under class\-conditional noise, all instances belonging to the same class share the same transition distribution, resulting innth=0\\textit\{nth\}=0\. Larger values indicate stronger instance\-dependent variation\.

Letp¯i​\(y~∣y=yi\)\\bar\{p\}\_\{i\}\(\\tilde\{y\}\\mid y=y\_\{i\}\)denote the aggregated transition distribution of instanceii, and let

p¯​\(y~∣y=k\)=1\|Ik\|​∑i∈Ikp¯i​\(y~∣y=k\)\\bar\{p\}\(\\tilde\{y\}\\mid y=k\)=\\frac\{1\}\{\|I\_\{k\}\|\}\\sum\_\{i\\in I\_\{k\}\}\\bar\{p\}\_\{i\}\(\\tilde\{y\}\\mid y=k\)denote the average transition distribution for classkk, whereIk=\{i∣yi=k\}I\_\{k\}=\\\{i\\mid y\_\{i\}=k\\\}\. We definenthas

nth\(Bτ\)=1N∑k=1K∑i∈Ik∥p¯i\(y~∣y=k\)−p¯\(y~∣y=k\)∥22\.\\textit\{nth\}\(B\_\{\\tau\}\)=\\frac\{1\}\{N\}\\sum\_\{k=1\}^\{K\}\\sum\_\{i\\in I\_\{k\}\}\\left\\\|\\bar\{p\}\_\{i\}\(\\tilde\{y\}\\mid y=k\)\-\\bar\{p\}\(\\tilde\{y\}\\mid y=k\)\\right\\\|\_\{2\}^\{2\}\.
Intuitively,nthmeasures how much the transition behavior of individual instances deviates from the average transition behavior of their class\. A value of zero indicates class\-conditional noise, while larger values indicate stronger instance\-dependent label noise\.

To evaluate similarity to human annotation uncertainty, we compare the generated label distributions against CIFAR\-10H using total variation \(TV\) distance\. Lower TV values indicate greater similarity to human label distributions\.

For downstream evaluation, we train classifiers using standard empirical risk minimization \(ERM\), Co\-Teaching\[[7](https://arxiv.org/html/2606.14965#bib.bib7)\], and DivideMix\[[14](https://arxiv.org/html/2606.14965#bib.bib18)\], and report clean\-test accuracy\. Section[V\-C](https://arxiv.org/html/2606.14965#S5.SS3)provides a brief overview of Co\-Teaching and DivideMix\. To ensure a fair comparison withPL\-IDN, we follow the same protocol for both benchmarks: a single voter \(or rater\) is sampled uniformly at random for each instance, and its predicted label is used as the training label\. We consider two training scenarios\. In*clean\-img*, learners are trained on clean images paired with corruption\-induced labels\. In*noisy\-img*, learners are trained on the corrupted images paired with the same labels\. Comparing the two scenarios isolates the effect of label noise from the additional effect of corrupted inputs\.

### IV\-BExperimental Results

We present the results according to the three evaluation objectives: instance dependence, similarity to human annotation uncertainty, and downstream learning performance\.

![Refer to caption](https://arxiv.org/html/2606.14965v1/x23.png)\(a\)Blur
![Refer to caption](https://arxiv.org/html/2606.14965v1/x24.png)\(b\)NoiseandGeometric
![Refer to caption](https://arxiv.org/html/2606.14965v1/x25.png)\(c\)Weather
![Refer to caption](https://arxiv.org/html/2606.14965v1/x26.png)\(d\)Digital

Figure 5:CIFAR\-10 nth across corruption super\-families and severities\. Each panel groups corruption sub\-families within a super\-family; bars are shown for levels 1, 3, and 5\.#### IV\-B1Strength of Instance Dependence

We evaluate instance dependence usingnth\. Recall thatnthmeasures the variability of instance\-specific transition distributions within each class\. A value of zero corresponds to class\-conditional noise, while larger values indicate stronger instance dependence\.

Figure[5](https://arxiv.org/html/2606.14965#S4.F5)summarizesnthacross the 45CIFAR10corruption settings\. All settings produce positiventhvalues, indicating that the induced label noise is instance\-dependent rather than class\-conditional\. The amount of instance dependence varies substantially across corruption families and severity levels, showing thatCILNprovides explicit control over the strength of the induced IDN\. For many corruption families,nthincreases with severity at low and moderate levels, but eventually saturates or decreases at high severity\. Once a corruption becomes sufficiently strong, many examples exhibit similar prediction patterns, reducing variability across instances\. As a result, higher noise rates do not necessarily imply stronger instance dependence\.

Table[II](https://arxiv.org/html/2606.14965#S4.T2)comparesCILNandPL\-IDNat comparable noise rates\. At low noise rates,PL\-IDNgenerally exhibits stronger instance dependence\. At medium noise rates, the two benchmarks are comparable, with someCILNsettings exceedingPL\-IDN\. At high noise rates, representativeCILNsettings match or exceed thenthvalues observed inPL\-IDN\. Beyond matchingPL\-IDN,CILNprovides a direct mechanism for controlling instance dependence through corruption type and severity\. In contrast,PL\-IDNincreases noise by progressively degrading the quality of its rater pool\. Consequently, the instance dependence observed inCILNis driven by controlled input ambiguity, whereas the instance dependence observed inPL\-IDNmay also reflect variation in rater quality\.

NoPL\-IDNreference benchmark exists forMNISTorAdult, so we report the absolute behavior ofCILN\. For both datasets, all evaluated settings produce positiventhvalues, indicating consistent instance\-dependent behavior\. OnMNIST, the strongest instance dependence is observed forGeometric, such asrotateandshear, whose effects depend heavily on the shape of the underlying digit\. OnAdult, the strongest instance dependence is observed for thescalingcorruption family\. These results demonstrate thatCILNextends beyond image data and remains effective across different data modalities\.

Table[II](https://arxiv.org/html/2606.14965#S4.T2)comparesnthunder the standard benchmark and the clean\-start variant\. As expected, clean\-start generally produces smallernthvalues because it removes examples that already exhibit uncertainty before corruption and therefore reduces the effective noise rate\. However, the qualitative conclusions remain unchanged\. In particular,nthremains positive across all evaluated settings, and the strongest instance\-dependent settings remain the strongest under both benchmark variants\. This indicates that the observed instance dependence is driven by the corruption process rather than by pre\-existing ambiguity in the underlying data\.

TABLE II:Comparison ofPL\-IDNand representativeCILNsettings onCIFAR10at comparable noise rates\. Results are shown for bothCILN\-SandCILN\-C\. Empty cells reflect deprioritized runs: underCILN\-S, ERM and DivideMix were not re\-run forshot\(5\) since theNoisefamily was already covered bygaussian\(3\); underCILN\-C, ERM and DivideMix were only run on the three closest noise\-rate matches toPL\-IDN’s high tier, and DivideMix was not re\-run underCILN\-Csince Co\-Teaching already confirmed the same small\-loss failure\.![Refer to caption](https://arxiv.org/html/2606.14965v1/x27.png)\(a\)Blur
![Refer to caption](https://arxiv.org/html/2606.14965v1/x28.png)\(b\)NoiseandGeometric
![Refer to caption](https://arxiv.org/html/2606.14965v1/x29.png)\(c\)Weather
![Refer to caption](https://arxiv.org/html/2606.14965v1/x30.png)\(d\)Digital

Figure 6:Total variation distance toCIFAR10\-Hacross corruption families and severity levels\. Lower values indicate greater similarity to human annotation uncertainty\.![Refer to caption](https://arxiv.org/html/2606.14965v1/x31.png)Figure 7:The 10CILNsettings closest toCIFAR10\-Hin PCA space\. The referenceCIFAR10\-Hdistribution is shown as a black times sign and the closetsPL\-IDNrelease as a red plus\.
#### IV\-B2Similarity to Human Annotation Uncertainty

We useCIFAR10\-Has a reference for human annotation uncertainty and measure the similarity between benchmark\-generated label distributions and human label distributions using TV distance\. Lower TV values indicate greater similarity to human labeling behavior\.

Table[II](https://arxiv.org/html/2606.14965#S4.T2)shows thatCILNis consistently closer toCIFAR10\-HthanPL\-IDNacross all threePL\-IDNnoise levels\. The gap increases as the noise rate grows, indicating that the label distributions produced byCILNremain closer to human disagreement patterns even in high\-noise settings\.

Figure[6](https://arxiv.org/html/2606.14965#S4.F6)shows how similarity to human labeling varies across corruption families and severity levels\. Human\-likeness is not uniform across corruption settings\. In general, mild corruptions produce label distributions that are closer to human annotation uncertainty, whereas severe corruptions tend to move further away from theCIFAR10\-Hreference\. Weather and Digital corruptions consistently achieve the lowest TV distances, while severe Noise and Blur corruptions are less human\-like despite still outperformingPL\-IDN\.

Figure[7](https://arxiv.org/html/2606.14965#S4.F7)provides a complementary view by ranking corruption settings according to their proximity toCIFAR10\-Hin PCA space\. The ten closest settings are all low\-severity Weather or Digital corruptions, includingbright,frost,snow,fog,contr, andjpeg\. Notably, none of the three releasedPL\-IDNdatasets appears among the ten closest settings\. This suggests that mild corruption\-induced ambiguity provides a particularly realistic approximation of human labeling uncertainty\.

Table[II](https://arxiv.org/html/2606.14965#S4.T2)also shows that the main conclusion is robust to the clean\-start variant\. Although clean\-start slightly increases TV distance in some low\-noise settings by removing naturally ambiguous examples,CILNremains consistently closer toCIFAR10\-HthanPL\-IDN\. At high noise rates, the two benchmark variants produce nearly identical results\.

TABLE III:Representative high\-noise settings\. The attractor class is the class receiving the largest increase in label frequency relative to the clean distribution; the value in parentheses reports the increase in percentage points\. Purity is the fraction of examples assigned to the attractor class that truly belong to that class\. Co\-T reports Co\-Teaching clean\-test accuracy \(clean\-img / noisy\-img\)\. For comparison, Co\-Teaching achieves65\.8%65\.8\\%accuracy onPL\-IDN\-H\.![Refer to caption](https://arxiv.org/html/2606.14965v1/x32.png)\(a\)frost\(1\)
![Refer to caption](https://arxiv.org/html/2606.14965v1/x33.png)\(b\)contrast\(5\)
![Refer to caption](https://arxiv.org/html/2606.14965v1/x34.png)\(c\)glass\(3\)
![Refer to caption](https://arxiv.org/html/2606.14965v1/x35.png)\(d\)frost\(1\)
![Refer to caption](https://arxiv.org/html/2606.14965v1/x36.png)\(e\)contrast\(5\)
![Refer to caption](https://arxiv.org/html/2606.14965v1/x37.png)\(f\)glass\(3\)

Figure 8:Per\-sample loss distributions for clean and noisy samples under three representativeCILNsettings\. The top row shows the distributions at the end of the warmup stage, while the bottom row shows the distributions at the final training epoch\.
#### IV\-B3Downstream Learning Behavior

We evaluate whether different benchmark generation mechanisms lead to different conclusions about noise\-robust learning\. We compare ERM, Co\-Teaching\[[7](https://arxiv.org/html/2606.14965#bib.bib7)\], and DivideMix\[[14](https://arxiv.org/html/2606.14965#bib.bib18)\]onPL\-IDNandCILNat comparable noise rates\. ForCILN, we consider two training scenarios:*clean\-img*, where learners are trained on clean images paired with corruption\-induced labels, and*noisy\-img*, where learners are trained on the corresponding corrupted images and labels\. Results are reported as clean\-test accuracy averaged over three random seeds\.

Table[II](https://arxiv.org/html/2606.14965#S4.T2)shows thatPL\-IDNandCILNproduce similar downstream behavior at low and medium noise rates\. Across these settings, ERM, Co\-Teaching, and DivideMix achieve comparable performance on both benchmarks\. The differences become apparent at high noise rates\. While Co\-Teaching and DivideMix remain reasonably effective onPL\-IDN\-H, both methods experience substantial performance degradation on several high\-noiseCILNsettings\. In contrast, ERM remains comparatively stable\.

The results suggest that benchmark behavior depends not only on the amount of label noise, but also on its structure\. InPL\-IDN, noisy labels originate from imperfect raters and are largely independent of the input seen by the learner\. InCILN, noisy labels are induced by corruptions that alter the input itself\. As a result, some incorrect labels become consistent with the corrupted image and are therefore easier for the learner to fit\. This weakens the small\-loss assumption underlying methods such as Co\-Teaching and DivideMix, which rely on noisy examples producing larger training losses than clean examples\.

Table[III](https://arxiv.org/html/2606.14965#S4.T3)provides evidence for this effect\. Several high\-noiseCILNsettings exhibit attractor classes that collect examples from many different true classes\. These concentrated error patterns reduce class purity and can make noisy examples appear easier to learn than clean examples\. Consequently, methods that identify noisy samples using training loss may incorrectly retain mislabeled instances while discarding informative clean ones\.

We further evaluate the clean\-start variant,CILN\-C, on representative high\-noise settings\. AlthoughCILN\-Cremoves examples that already exhibit voter disagreement before corruption and therefore produces slightly lower noise rates, the same qualitative behavior remains\. Co\-Teaching continues to perform poorly, while ERM remains relatively stable\. This indicates that the observed failures are not caused by pre\-existing ambiguity in the data, but by the structure of the corruption\-induced label noise itself\.

Overall, the results show that methods that perform well on existing IDN benchmarks do not necessarily generalize to corruption\-mediated label noise\. At low and medium noise rates,CILNbehaves similarly toPL\-IDN\. At high noise rates, however,CILNexposes failure modes that are not apparent under conventional benchmark generation procedures, highlighting the importance of evaluating noise\-robust learning methods under multiple noise\-generation mechanisms\.

### IV\-CDiscussion and Takeaways

The results show thatCILNprovides a mechanism\-controlled way to generate instance\-dependent label noise\. Unlike rater\-based benchmarks, where the source of uncertainty is implicit in the rater pool,CILNmakes the ambiguity mechanism explicit through the corruption type and severity\. The benchmark covers9090settings acrossCIFAR10,MNIST, andAdult, and each setting includes the clean input, corrupted input, ground\-truth label, aggregated voter distribution, individual voter distributions, and corruption metadata\. This allows users to work with soft labels, sampled hard labels, or argmax hard labels under the same construction\.

A key takeaway is that corruption\-induced IDN is structured\. Corruption type controls the direction of the distributional shift, often producing attractor classes, while severity controls the strength of the shift\. This structure is not arbitrary label flipping: it reflects how a specific corruption interacts with the data\. The positiventhvalues across settings show that the resulting noise is instance\-dependent, and the variation across corruption families shows that this dependence can be controlled\.

The downstream results show that noise rate alone is not enough to characterize benchmark difficulty\. At low and medium noise rates,CILNandPL\-IDNlead to similar conclusions for ERM, Co\-Teaching, and DivideMix\. At high noise rates, however, severalCILNsettings expose failure modes that are not visible onPL\-IDN\. In particular, attractor\-heavy corruptions can make wrong labels consistent with the corrupted inputs, so mislabeled examples may no longer have high loss\. This weakens the small\-loss signal used by Co\-Teaching and DivideMix and explains why these methods can fail even when the nominal noise rate is comparable toPL\-IDN\-H\.

The clean\-start results strengthen this interpretation\.CILN\-Cremoves examples that already exhibit voter disagreement before corruption, so the remaining label errors are more directly attributable to the corruption mechanism\. The same qualitative failures still appear underCILN\-C, indicating that the downstream difficulty comes from the structure of corruption\-induced noise rather than from pre\-existing ambiguity in the original data\.

These findings suggest several uses forCILN\. It can be used to test whether noise\-robust methods handle attractor\-class effects, to study how performance changes as severity increases within a fixed corruption family, and to compare robustness across image, digit, and tabular settings under a shared benchmark construction\. More broadly,CILNcomplements rater\-based IDN benchmarks by providing a controlled way to study how observable data degradation creates label uncertainty\.

The benchmark also has limitations\. The induced noise depends on the chosen voter pool, so different voters may shift which classes become attractors\. The downstream learning experiments are currently limited toCIFAR10, and extending them toMNISTandAdultwould help determine whether the same failure modes appear in other modalities\. Finally,CIFAR10\-His used as a reference for human annotation uncertainty, but it should not be interpreted as ground truth for how humans would label severely corrupted images\.

## VRelated Work

This work relates to benchmark construction for noisy\-label learning, corruption\-based robustness evaluation, and learning under noisy labels\. Our framework connects the first two directions by using controlled input corruptions to generate realistic and interpretable instance\-dependent label noise\.

### V\-ABenchmark Construction for Label Noise

A central challenge in noisy\-label research is constructing benchmarks that balance realism and experimental control\.

#### V\-A1Symmetric and Random Label Noise

The most common synthetic benchmark strategy introduces label noise through random label flipping\. Under symmetric noise, each label is independently replaced with an incorrect label according to a fixed probability\[[19](https://arxiv.org/html/2606.14965#bib.bib5)\]\. These benchmarks provide precise control over noise rates and are widely used for evaluation, but they fail to capture realistic annotation behavior because all examples are equally likely to be mislabeled regardless of their difficulty\.

#### V\-A2Class\-Conditional Noise

Class\-conditional noise \(CCN\) allows corruption probabilities to depend on the true class\[[21](https://arxiv.org/html/2606.14965#bib.bib6)\]\. Label flips are governed by a class\-transition matrix, enabling semantically plausible mistakes such as confusing cats with dogs more often than cats with airplanes\. Although more realistic than random flipping, CCN still assumes that all examples within a class share the same corruption process and therefore ignores instance\-specific ambiguity\.

#### V\-A3Instance\-Dependent Noise

Instance\-dependent label noise \(IDN\) models settings in which the probability of mislabeling depends on the input instance itself\[[29](https://arxiv.org/html/2606.14965#bib.bib14),[3](https://arxiv.org/html/2606.14965#bib.bib13)\]\. This more closely reflects practical annotation processes, where difficult examples are naturally more likely to receive incorrect labels\.

Several synthetic IDN generation methods have been proposed\. Xia et al\.\[[29](https://arxiv.org/html/2606.14965#bib.bib14)\]assign each example an individual corruption probability using feature\-space transformations\. Gu et al\.\[[6](https://arxiv.org/html/2606.14965#bib.bib4)\]generate IDN through disagreement among a pool of classifier models acting as synthetic annotators\. By varying model architectures and training conditions, their framework produces controllable noise levels and label distributions that resemble human uncertainty datasets such as CIFAR10\-H\[[22](https://arxiv.org/html/2606.14965#bib.bib9)\]\.

Despite their differences, existing IDN benchmarks primarily generate ambiguity through latent feature transformations, instance difficulty, or annotator variation\. Consequently, the source of ambiguity remains implicit\. In contrast, our framework generates ambiguity through controlled data corruptions, providing explicit control over both the source and severity of the induced uncertainty\.

Several datasets provide naturally occurring noisy labels, including Clothing1M\[[30](https://arxiv.org/html/2606.14965#bib.bib15)\], Food101\-N\[[13](https://arxiv.org/html/2606.14965#bib.bib16)\], WebVision\[[15](https://arxiv.org/html/2606.14965#bib.bib17)\], and CIFAR10\-H\[[22](https://arxiv.org/html/2606.14965#bib.bib9)\]\. While these datasets capture realistic annotation uncertainty, they offer limited control over the underlying noise\-generation process\.

### V\-BCorruption\-Based Robustness Benchmarks

Corruption benchmarks evaluate model robustness under degraded inputs\. Representative examples include CIFAR\-C and ImageNet\-C\[[9](https://arxiv.org/html/2606.14965#bib.bib1)\], which apply common corruptions such as blur, noise, weather effects, compression artifacts, and geometric distortions across multiple severity levels\. Subsequent work has extended this paradigm to broader forms of distribution shift and robustness evaluation\[[17](https://arxiv.org/html/2606.14965#bib.bib3)\]\.

Our goal differs: rather than evaluating prediction accuracy under corruption \(the standard use of CIFAR\-10\-C and ImageNet\-C\), we use corruptions as a mechanism for generating label uncertainty\. The corrupted inputs are used during benchmark construction, while downstream learners can be trained on either clean or corrupted inputs paired with the generated labels\. The objective is therefore benchmark construction rather than robustness evaluation\.

### V\-CLearning under Noisy Labels

A large body of work studies learning from incorrect labels\. Existing methods are commonly grouped into sample\-selection, label\-correction, and hybrid approaches\. Sample\-selection methods exploit the observation that neural networks fit clean examples faster than noisy ones\[[1](https://arxiv.org/html/2606.14965#bib.bib36)\]\. Co\-Teaching\[[7](https://arxiv.org/html/2606.14965#bib.bib7)\]trains two networks that exchange small\-loss samples, reducing confirmation bias while relying on the assumption that clean examples occupy the low\-loss region\. Label\-correction methods attempt to repair suspicious labels rather than discard them\. Examples include transition\-matrix methods\[[21](https://arxiv.org/html/2606.14965#bib.bib6)\], Bootstrapping\[[25](https://arxiv.org/html/2606.14965#bib.bib38)\], and Confident Learning\[[20](https://arxiv.org/html/2606.14965#bib.bib39)\], which relabel or remove examples based on model predictions and estimated noise patterns\. Hybrid approaches combine both ideas\. DivideMix\[[14](https://arxiv.org/html/2606.14965#bib.bib18)\]fits a two\-component Gaussian Mixture Model \(GMM\) to the loss distribution to separate likely clean and noisy samples\. The noisy subset is then treated as unlabeled and incorporated into a MixMatch\-style semi\-supervised pipeline\[[2](https://arxiv.org/html/2606.14965#bib.bib37)\]\.

Both Co\-Teaching and DivideMix depend on noisy examples being harder to fit than clean ones, creating a bimodal loss distribution\. This assumption generally holds in conventional synthetic benchmarks\. However, Section[IV\-B3](https://arxiv.org/html/2606.14965#S4.SS2.SSS3)shows that corruption\-mediated noise can violate it: when an incorrect label remains plausible for a corrupted input, noisy examples become as easy to fit as clean ones, causing the bimodal structure to collapse\. Consequently, both methods fail on the same high\-noise settings for the same underlying reason\. Since noisy\-label methods are often evaluated on synthetic benchmarks, benchmark design can strongly affect conclusions about algorithm performance\. Our work provides a complementary benchmark\-generation framework for studying how different sources of ambiguity influence noisy\-label learning\.

## VIConclusion and Future Work

We introducedCILN, a benchmark\-generation framework for instance\-dependent label noise based on controlled input corruptions\. By making the source and severity of ambiguity explicit,CILNproduces controllable instance\-dependent noise across image and tabular datasets\. Our experiments show that the generated benchmarks exhibit genuine instance dependence, can produce label distributions closer to human uncertainty than existing synthetic IDN benchmarks, and expose failure modes of noise\-robust learning methods that remain hidden under traditional rater\-fallibility benchmarks\. Overall, our results suggest that benchmark difficulty depends not only on the amount of label noise, but also on its underlying structure, making corruption\-mediated and rater\-mediated IDN complementary evaluation settings\.

Several directions remain for future work\. First, the downstream evaluation could be extended beyond CIFAR\-10 to verify whether the observed failure modes generalize to other modalities\. Second, the benchmark could be expanded with additional corruption families and domain\-specific degradations\. Finally, the benchmark opens opportunities for developing mechanism\-aware noisy\-label learning methods that explicitly exploit information about the source and structure of ambiguity rather than relying solely on loss\-based heuristics\.

## VIIAcknowledgments

##### AI\-Generated Content Acknowledgment

Claude Opus 4\.7 was used for code generation from author\-provided specifications and GPT 5\.2 was used for language editing\. All generated content was reviewed and validated by the authors\. The research design, experiments, analysis, conclusions, and all claims regarding prior work are the authors’ own intellectual contributions\.

## References

- \[1\]D\. Arpit, S\. Jastrzębski, N\. Ballas, D\. Krueger, E\. Bengio, M\. S\. Kanwal, T\. Maharaj, A\. Fischer, A\. Courville, Y\. Bengio, and S\. Lacoste\-Julien\(2017\)A closer look at memorization in deep networks\.InICML,Cited by:[§V\-C](https://arxiv.org/html/2606.14965#S5.SS3.p1.1)\.
- \[2\]D\. Berthelot, N\. Carlini, I\. Goodfellow, N\. Papernot, A\. Oliver, and C\. Raffel\(2019\)MixMatch: a holistic approach to semi\-supervised learning\.InNeurIPS,Cited by:[§V\-C](https://arxiv.org/html/2606.14965#S5.SS3.p1.1)\.
- \[3\]A\. Berthon, B\. Han, G\. Niu, T\. Liu, and M\. Sugiyama\(2021\)Confidence scores make instance\-dependent label\-noise learning possible\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p1.1),[§I](https://arxiv.org/html/2606.14965#S1.p2.1),[§V\-A3](https://arxiv.org/html/2606.14965#S5.SS1.SSS3.p1.1)\.
- \[4\]T\. Chen and C\. Guestrin\(2016\)XGBoost: a scalable tree boosting system\.InKDD,Cited by:[3rd item](https://arxiv.org/html/2606.14965#S3.I1.i3.p1.3)\.
- \[5\]Y\. Gorishniy, I\. Rubachev, V\. Khrulkov, and A\. Babenko\(2021\)Revisiting deep learning models for tabular data\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[3rd item](https://arxiv.org/html/2606.14965#S3.I1.i3.p1.3)\.
- \[6\]K\. Gu, X\. Masotto, V\. Bachani, B\. Lakshminarayanan, J\. Nikodem, and D\. Yin\(2022\)An instance\-dependent simulation framework for learning with label noise\.Machine Learning\.Note:arXiv:2107\.11413Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p2.1),[§I](https://arxiv.org/html/2606.14965#S1.p7.1),[§III\-A](https://arxiv.org/html/2606.14965#S3.SS1.p6.1),[§IV\-A1](https://arxiv.org/html/2606.14965#S4.SS1.SSS1.p1.1),[§V\-A3](https://arxiv.org/html/2606.14965#S5.SS1.SSS3.p2.1)\.
- \[7\]B\. Han, Q\. Yao, X\. Yu, G\. Niu, M\. Xu, W\. Hu, I\. Tsang, and M\. Sugiyama\(2018\)Co\-teaching: robust training of deep neural networks with extremely noisy labels\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p1.1),[§I](https://arxiv.org/html/2606.14965#S1.p8.1),[§IV\-A2](https://arxiv.org/html/2606.14965#S4.SS1.SSS2.p6.1),[§IV\-B3](https://arxiv.org/html/2606.14965#S4.SS2.SSS3.p1.1),[§V\-C](https://arxiv.org/html/2606.14965#S5.SS3.p1.1)\.
- \[8\]K\. He, X\. Zhang, S\. Ren, and J\. Sun\(2016\)Deep residual learning for image recognition\.InCVPR,Cited by:[1st item](https://arxiv.org/html/2606.14965#S3.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2606.14965#S3.I1.i2.p1.1)\.
- \[9\]D\. Hendrycks and T\. Dietterich\(2019\)Benchmarking neural network robustness to common corruptions and perturbations\.InICLR,Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p4.1),[§II\-C1](https://arxiv.org/html/2606.14965#S2.SS3.SSS1.p1.1),[§II\-C](https://arxiv.org/html/2606.14965#S2.SS3.p1.1),[§II\-C](https://arxiv.org/html/2606.14965#S2.SS3.p2.3),[1st item](https://arxiv.org/html/2606.14965#S3.I1.i1.p1.1),[§III\-B](https://arxiv.org/html/2606.14965#S3.SS2.p3.2),[§V\-B](https://arxiv.org/html/2606.14965#S5.SS2.p1.1)\.
- \[10\]N\. Hollmannet al\.\(2025\)Accurate predictions on small data with a tabular foundation model\.Nature\.Cited by:[3rd item](https://arxiv.org/html/2606.14965#S3.I1.i3.p1.3)\.
- \[11\]L\. Jiang, D\. Huang, M\. Liu, and W\. Yang\(2020\)Beyond synthetic noise: deep learning on controlled noisy labels\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p1.1)\.
- \[12\]Y\. LeCun, L\. Bottou, Y\. Bengio, and P\. Haffner\(1998\)Gradient\-based learning applied to document recognition\.Proceedings of the IEEE86\(11\),pp\. 2278–2324\.Cited by:[2nd item](https://arxiv.org/html/2606.14965#S3.I1.i2.p1.1)\.
- \[13\]K\. Lee, X\. He, L\. Zhang, and L\. Yang\(2018\)CleanNet: transfer learning for scalable image classifier training with label noise\.InCVPR,Cited by:[§V\-A3](https://arxiv.org/html/2606.14965#S5.SS1.SSS3.p4.1)\.
- \[14\]J\. Li, R\. Socher, and S\. C\. H\. Hoi\(2020\)DivideMix: learning with noisy labels as semi\-supervised learning\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p8.1),[§IV\-A2](https://arxiv.org/html/2606.14965#S4.SS1.SSS2.p6.1),[§IV\-B3](https://arxiv.org/html/2606.14965#S4.SS2.SSS3.p1.1),[§V\-C](https://arxiv.org/html/2606.14965#S5.SS3.p1.1)\.
- \[15\]W\. Li, L\. Wang, W\. Li, E\. Agustsson, and L\. Van Gool\(2017\)WebVision database: visual learning and understanding from web data\.InCVPR,Cited by:[§V\-A3](https://arxiv.org/html/2606.14965#S5.SS1.SSS3.p4.1)\.
- \[16\]Y\. Lu, Y\. Zhang, B\. Han, Y\. Cheung, and H\. Wang\(2023\)Label\-noise learning with intrinsically long\-tailed data\.InProceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 1369–1378\.Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p1.1)\.
- \[17\]N\. Mu, J\. Gilmer, I\. Goodfellow, and D\. Duvenaud\(2021\)On the validity of imagenet\-c and imagenet\-p\.InNeurIPS Workshop on Distribution Shifts,Cited by:[§V\-B](https://arxiv.org/html/2606.14965#S5.SS2.p1.1)\.
- \[18\]N\. Mu and J\. Gilmer\(2021\)MNIST\-C: a robustness benchmark for computer vision\.arXiv preprint arXiv:1906\.02337\.Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p4.1),[§II\-C1](https://arxiv.org/html/2606.14965#S2.SS3.SSS1.p1.1),[§II\-C](https://arxiv.org/html/2606.14965#S2.SS3.p1.1),[§II\-C](https://arxiv.org/html/2606.14965#S2.SS3.p2.3),[2nd item](https://arxiv.org/html/2606.14965#S3.I1.i2.p1.1),[§III\-B](https://arxiv.org/html/2606.14965#S3.SS2.p3.2)\.
- \[19\]N\. Natarajan, I\. S\. Dhillon, P\. Ravikumar, and A\. Tewari\(2013\)Learning with noisy labels\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p1.1),[§V\-A1](https://arxiv.org/html/2606.14965#S5.SS1.SSS1.p1.1)\.
- \[20\]C\. G\. Northcutt, L\. Jiang, and I\. Chuang\(2021\)Confident learning: estimating uncertainty in dataset labels\.Journal of Artificial Intelligence Research \(JAIR\)\.Cited by:[§V\-C](https://arxiv.org/html/2606.14965#S5.SS3.p1.1)\.
- \[21\]G\. Patrini, A\. Rozza, A\. K\. Menon, R\. Nock, and L\. Qu\(2017\)Making deep neural networks robust to label noise: a loss correction approach\.InIEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p1.1),[§V\-A2](https://arxiv.org/html/2606.14965#S5.SS1.SSS2.p1.1),[§V\-C](https://arxiv.org/html/2606.14965#S5.SS3.p1.1)\.
- \[22\]J\. C\. Peterson, R\. M\. Battleday, T\. Griffiths, and O\. Russakovsky\(2019\)Human uncertainty makes classification more robust\.InIEEE International Conference on Computer Vision \(ICCV\),Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p7.1),[§I](https://arxiv.org/html/2606.14965#S1.p8.1),[§V\-A3](https://arxiv.org/html/2606.14965#S5.SS1.SSS3.p2.1),[§V\-A3](https://arxiv.org/html/2606.14965#S5.SS1.SSS3.p4.1)\.
- \[23\]L\. Prokhorenkovaet al\.\(2018\)CatBoost: unbiased boosting with categorical features\.InNeurIPS,Cited by:[3rd item](https://arxiv.org/html/2606.14965#S3.I1.i3.p1.3)\.
- \[24\]A\. Radfordet al\.\(2021\)Learning transferable visual models from natural language supervision\.InICML,Cited by:[1st item](https://arxiv.org/html/2606.14965#S3.I1.i1.p1.1)\.
- \[25\]S\. Reed, H\. Lee, D\. Anguelov, C\. Szegedy, D\. Erhan, and A\. Rabinovich\(2015\)Training deep neural networks on noisy labels with bootstrapping\.InICLR Workshop,Cited by:[§V\-C](https://arxiv.org/html/2606.14965#S5.SS3.p1.1)\.
- \[26\]D\. Rolnick, A\. Veit, S\. Belongie, and N\. Shavit\(2017\)Deep learning is robust to massive label noise\.arXiv preprint arXiv:1705\.10694\.Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p1.1)\.
- \[27\]S\. Schelter, T\. Rukat, and F\. Bießmann\(2021\)JENGA: a framework to study the impact of data errors on the predictions of machine learning models\.InEDBT,Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p4.1),[§II\-C2](https://arxiv.org/html/2606.14965#S2.SS3.SSS2.p1.1),[§II\-C](https://arxiv.org/html/2606.14965#S2.SS3.p1.1),[§III\-B](https://arxiv.org/html/2606.14965#S3.SS2.p3.2)\.
- \[28\]H\. Touvronet al\.\(2022\)DeiT iii: revenge of the vit\.ECCV\.Cited by:[1st item](https://arxiv.org/html/2606.14965#S3.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2606.14965#S3.I1.i2.p1.1)\.
- \[29\]X\. Xia, T\. Liu, B\. Han, N\. Wang, M\. Gong, H\. Liu, G\. Niu, D\. Tao, and M\. Sugiyama\(2020\)Part\-dependent label noise: towards instance\-dependent label noise\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§I](https://arxiv.org/html/2606.14965#S1.p1.1),[§I](https://arxiv.org/html/2606.14965#S1.p2.1),[§V\-A3](https://arxiv.org/html/2606.14965#S5.SS1.SSS3.p1.1),[§V\-A3](https://arxiv.org/html/2606.14965#S5.SS1.SSS3.p2.1)\.
- \[30\]T\. Xiao, T\. Xia, Y\. Yang, C\. Huang, and X\. Wang\(2015\)Learning from massive noisy labeled data for image classification\.InCVPR,Cited by:[§V\-A3](https://arxiv.org/html/2606.14965#S5.SS1.SSS3.p4.1)\.
- \[31\]S\. Zagoruyko and N\. Komodakis\(2016\)Wide residual networks\.BMVC\.Cited by:[1st item](https://arxiv.org/html/2606.14965#S3.I1.i1.p1.1)\.