Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels

arXiv cs.LG Papers
loss-functions robust-training noisy-labels neural-networks symmetrization multi-class unhinged-loss

Summary

This paper studies symmetrization of loss functions for robust training under label noise, introducing SGCE and alpha-MAE loss functions that interpolate between multi-class unhinged loss and Mean Absolute Error, with theoretical guarantees and competitive empirical performance.

arXiv:2605.20347v1 Announce Type: new Abstract: Labeling a training set is often expensive and susceptible to errors, making the design of robust loss functions for label noise an important problem. The symmetry condition provides theoretical guarantees for robustness to such noise. In this work, we study a symmetrization method arising from the unique decomposition of any multi-class loss function into a symmetric component and a class-insensitive term. In particular, symmetrizing the cross-entropy loss leads to a linear multi-class extension of the unhinged loss. Unlike in the binary case, the multi-class version must have specific coefficients in order to satisfy the symmetry condition. Under suitable assumptions, we show that this multi-class unhinged loss is the unique convex multi-class symmetric loss. We also show that it has a fundamental local role: the linear approximation of any symmetric loss around score vectors with equal components is equivalent to the multi-class unhinged loss. We then introduce SGCE and alpha-MAE, two loss functions that interpolate between the multi-class unhinged loss and the Mean Absolute Error while allowing control of the beta-smoothness of the loss. Experiments on standard noisy-label benchmarks show competitive performance compared with existing robust loss functions.
Original Article
View Cached Full Text

Cached at: 05/21/26, 06:23 AM 

# Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels
Source: [https://arxiv.org/abs/2605.20347](https://arxiv.org/abs/2605.20347)
[View PDF](https://arxiv.org/pdf/2605.20347)

> Abstract:Labeling a training set is often expensive and susceptible to errors, making the design of robust loss functions for label noise an important problem\. The symmetry condition provides theoretical guarantees for robustness to such noise\. In this work, we study a symmetrization method arising from the unique decomposition of any multi\-class loss function into a symmetric component and a class\-insensitive term\. In particular, symmetrizing the cross\-entropy loss leads to a linear multi\-class extension of the unhinged loss\. Unlike in the binary case, the multi\-class version must have specific coefficients in order to satisfy the symmetry condition\. Under suitable assumptions, we show that this multi\-class unhinged loss is the unique convex multi\-class symmetric loss\. We also show that it has a fundamental local role: the linear approximation of any symmetric loss around score vectors with equal components is equivalent to the multi\-class unhinged loss\. We then introduce SGCE and alpha\-MAE, two loss functions that interpolate between the multi\-class unhinged loss and the Mean Absolute Error while allowing control of the beta\-smoothness of the loss\. Experiments on standard noisy\-label benchmarks show competitive performance compared with existing robust loss functions\.

## Submission history

From: Alexandre Lemire Paquin \[[view email](https://arxiv.org/show-email/95b3122a/2605.20347)\] **\[v1\]**Tue, 19 May 2026 18:03:40 UTC \(90 KB\)

Similar Articles

Sparse and robust geometric twin support vector machine via asymmetric RoBoSS loss function

arXiv cs.LG

This paper proposes a new asymmetric robust bounded sparse smooth (aR) loss function for l1-norm penalized geometric twin support vector machine (aRSGTSVM) to handle classification and regression tasks with label and feature noise, achieving feature selection and robustness. Experiments on synthetic and UCI datasets plus China stock market index tracking demonstrate superiority.

Fast Rates for Swap-Agnostic Learning of Proper Losses

arXiv cs.LG

This paper studies swap-agnostic learning of proper losses, showing that prediction-level comparisons can be controlled jointly via second-order multicalibration, achieving tight rates for finite hypothesis classes and families of losses.

Making Optimization Work When Labels Are Scarce [R]

Reddit r/MachineLearning

Gnosys Labs introduces an autonomous model engineering method that improves classifiers under label scarcity, outperforming standard optimizers like GEPA on the ToxicChat benchmark.

Loss Smoothing for Stable Adaptation Under Distribution Shift

arXiv cs.LG

Loss smoothing interpolates between source and target objectives during adaptation, preserving useful features while enabling specialization. Experiments across supervised shifts, RL, and language model fine-tuning show consistent improvements.