Skip to content

Gradient-Modulated Cross-Entropy Loss for Robust Learning With Label Noise.

Oct 2026 · IEEE Transactions on Neural Networks and Learning Systems · Vol PP · 0 citations
Medicine

Abstract

Deep neural networks (DNNs) trained with the standard cross-entropy (CE) loss typically achieve strong performance on clean datasets but suffer from severe overfitting when confronted with label noise. To alleviate this issue, numerous robust loss functions have been proposed and shown to improve noise tolerance. However, most existing methods focus predominantly on the target class, overlooking informative class-structural cues embedded in nontarget predictions. In this article, we propose a simple yet effective loss function, named gradient-modulated CE (GMCE), which enhances robustness by incorporating uncertainty-aware gradient modulation across the entire predicted distribution. GMCE comprises two synergistic components: negative-shifted CE (NSCE) loss and uncertainty-aware calibration mechanism (UACM). Specifically, NSCE attenuates the gradient magnitude for highly uncertain predictions by applying a convexity-preserving negative shift to the CE loss, thereby flattening its curvature in low-confidence regions. In parallel, UACM adaptively reweights the loss magnitude based on the entropy and cumulative probability over nontarget classes, assigning greater emphasis to samples with peaked and confident output distributions. Furthermore, we provide a theoretical analysis demonstrating the robustness of GMCE under label noise. Extensive experiments on benchmark datasets with both synthetic and real-world label noise, including MNIST, CIFAR-10, CIFAR-100, WebVision, and Clothing1M, demonstrate that the proposed method consistently outperforms the state-of-the-art approaches.

View source

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.