Jun 2026· arXiv.org· Vol abs/2606.28654· 0 citations· 45 references
Computer Science
TL;DR
This work proposes FedLAS: Feature-Modulated Bidirectional Label Smoothing, a plug-and-play algorithm for label smoothing-based losses that consistently improves calibration compared to modern baselines, reducing Expected Calibration Error (ECE) and Adaptive ECE while maintaining Top-1 accuracy.
Abstract
Deep Neural Network (DNN) classifiers suffer from poor calibration when their softmax outputs (predictive confidence) deviate from the empirical likelihoods. This manifests itself as either overconfident incorrect predictions or under-confident correct predictions. Label smoothing (LS) enhances model calibration by introducing entropy regularization during training through redistributing probability mass from the ground-truth label to the remaining classes. LS, including Margin-based LS (MbLS), have restrictive assumptions: they rely on predefined, uniform smoothing rules and only tackle overconfidence. In reality, samples exhibit diverse characteristics, such as difficulty/ambiguity, that interact with the evolving nature of the model being trained. In training, samples may have various degrees of under- or overconfidence. To overcome this, a mechanism that identifies the specific confidence state of each sample and determines the appropriate degree of smoothing in each training step is needed, tailoring the adjustment to the individual sample. We propose FedLAS: Feature-Modulated Bidirectional Label Smoothing, a plug-and-play algorithm for label smoothing-based losses. In FedLAS, we introduce a Feature Norm-based Confidence Indicator (NCI) to control smoothing and a Bidirectional Calibration Gating (BCG) module to detect both over and under-confidence. Our algorithm can be integrated with LS and MbLS based losses when applied to standard DNNs, enhancing performance. Extensive experiments on standard and fine-grained high-resolution vision benchmarks show that FedLAS consistently improves calibration compared to modern baselines, reducing Expected Calibration Error (ECE) and Adaptive ECE while maintaining Top-1 accuracy. Code: github.com/nadarasarbahavan/FEDLAS
The current state-of-the-art (SOTA) deep randomized neural networks, such as deep Random Vector Functional Link (dRVFL) and ensemble deep RVFL (edRVFL), treat all training samples uniformly, which limits their robustness and effectiveness when applied to real-world datasets containing noise and outliers. Furthermore, the propagation of contaminated features across hidden layers negatively influences the decision-making capability of these models. To overcome these limitations, we propose intuitionistic fuzzy dRVFL (IF-dRVFL) and intuitionistic fuzzy edRVFL (IF-edRVFL) frameworks that enhance model robustness. The proposed models unify intuitionistic fuzzy theory to exploit sample neighborhood information in the kernel space by jointly considering membership and non-membership degrees for each sample. Membership degrees are computed based on the distance of samples from their respective class centroids, while non-membership degrees quantify sample heterogeneity within local neighborhoods. These measures are employed to assign adaptive weights to training samples, enabling effective discrimination among clean, noisy, and outlier data points. Extensive experiments conducted on UCI and KEEL benchmark datasets, with and without the presence of Gaussian noise, demonstrate the superiority of the proposed IF-dRVFL and IF-edRVFL models over existing SOTA fuzzy and non-fuzzy approaches. The source code is available at https://github.com/mtanveer1/IF-edRVFL.
M. Sajid, A. Quadir, A. Rahaman et al.· 0 citations
Deep neural networks often suffer significant accuracy degradation when exposed to real-world image corruptions and distribution shifts. To overcome the limitations of fixed, input-agnostic test-time augmentation (TTA), an adaptive framework is proposed that learns per-sample transformations via reinforcement learning. Augmentation selection is cast as a Markov decision process and proximal policy optimization (PPO) agents are trained to choose sample-specific transforms under a composite reward combining classifier confidence gains with a self-consistency KL-divergence penalty on the model's own softmax outputs, thereby preserving overall belief stability. On clean CIFAR-10 (1 000 samples), the adaptive ensemble raises accuracy from 88.5% (baseline) and 87.3% (static TTA) to 90.0% (+1.5 pp). On CIFAR-10-C (15 corruptions × 5 severities; 1 000 images per condition), pooled top-1 accuracy improves from 75.7% (baseline) and 74.3% (static TTA) to 76.4% (+0.7 pp), and exceeds a TENT entropy-minimization baseline (75.9%) while operating in a strictly label-free regime that updates no model weights. Per-corruption gains are consistently positive across noise, blur, weather, and compression distortions, with the adaptive policy outperforming TENT on texture and compression corruptions where input-space transforms are most effective. These findings demonstrate that learned, per-sample augmentation policies improve robustness and reliability of deep vision models under diverse image conditions, against a strong baseline classifier.
Tushar Mittal, A. Dubey, Dharmender Saini et al.· Scientific Reports· 0 citations
Deep learning models lack intrinsic interpretability and require manual tuning of information-theoretic hyperparameters. This paper presents HIFN-Transformer (HIFN-T), a framework extending the Variational Information Bottleneck through four jointly learnable per-layer parameters: information retention, entropy budget, magnitude scaling, and global information gates. The retention and budget parameters are co-adapted with the network weights via a stop-gradient decoupling that stabilizes training and substantially reduces the manual hyperparameter burden by eliminating per-layer $\beta $ grid search; the global gates learn per-neuron importance weights, and a Transformer encoder captures complex feature interactions via multi-head self-attention. We prove HIFN generalizes standard VIB as a special case and characterize the role of the entropy budget as an early-training stabilizer that prevents variance explosion before convergence. We evaluate HIFN-T on five benchmarks spanning network intrusion detection (CICIDS-2017, NSL-KDD, GothamDataset2025), image classification (MNIST), and speech emotion recognition (EMO-DB), comparing against XGBoost, TabNet, FT-Transformer, and four additional baselines. HIFN-T delivers consistent improvements across all five. On the official NSL-KDD KDDTrain $+ \,\, \rightarrow $ KDDTest+ benchmark, HIFN-T with a 4-method consensus feature-selection front-end achieves 80.22% accuracy and 0.6898 macro F1, lying within the established 78–87% band of published deep models on this cross-distribution split. On the large-scale GothamDataset2025 IoT benchmark, we evaluate HIFN-T on the full 35,126,611-row release under natural class prevalence as a scalability demonstration on this recently published dataset. MNIST is included as a domain-generalization sanity check rather than a competitive benchmark. Gate-derived feature importance correlates strongly with SHAP analysis across all five datasets, validating intrinsic interpretability. We report all results as mean ± standard deviation over ten independent runs with Friedman and Wilcoxon signed-rank statistical significance tests.
Learning with noisy labels (LNL) remains challenging, especially when the identification of clean samples relies heavily on the predictions of the model being trained. In such cases, early-stage selection errors may be reinforced during iterative optimization, leading to unreliable supervision. To alleviate this issue, a two-stage framework, termed CANNE, is proposed by combining Contrastive Language–Image Pre-training (CLIP)-based conservative offline cleaning with Adaptive Nearest Neighbors and eigenvector-based sample selection (ANNE)-based online refinement. Specifically, a high-confidence clean seed set is first constructed using two complementary probability sources derived from frozen CLIP representations and reliability criteria, including class-wise loss modeling and prediction consistency. This seed set is then used as a set of reliable anchors during the subsequent ANNE training process, where online feature- and neighborhood-based refinement further recovers and adjusts sample partitions. In this way, CANNE uses external vision–language priors to provide conservative and persistent guidance while preserving the adaptive recovery ability of online noisy-label learning. Experimental results on CIFAR-10, CIFAR-100, Animal-10N, and Mini-WebVision, together with additional evaluation under open-set noise, show that the proposed method achieves competitive performance across diverse noisy-label settings. In particular, CANNE achieves 96.6% and 96.3% best accuracies on CIFAR-10 under 80% and 90% symmetric noise, respectively, and 81.0% and 79.0% on CIFAR-100 under 20% and 50% symmetric noise. Additional repeated-run, threshold-sensitivity, and runtime analyses further indicate that the CLIP-based seed set provides stable guidance with only moderate computational overhead.
Ge Jin, Qian Zhang, Li Huang et al.· Entropy· 0 citations
This tutorial covers recent developments in OOD detection from both theoretical and practical perspectives, including four major categories: (1) post-hoc methods, (2) training-based methods with auxiliary outliers, (3) training methods without auxiliary outliers, and (4) foundation model-based approaches, along with recent advances in each area.
Suranga Seneviratne, Dishanika Denipitiyage, Sanjay Chawla et al.· Proceedings of the 32nd ACM...· 0 citations
Artificial neural networks (ANNs) make predictions based on patterns learned during training; however, their reliability may deteriorate when the data distribution or the trained model changes. This paper proposes an Embedded One-Class Classification (EOCC) framework for monitoring task-informed neural network representations. The predictive network is first trained using its original classification or regression objective and then fixed. Embedded Support Vector Data Description (ESVDD) or Embedded Least Squares Support Vector Data Description (ELS-SVDD) is subsequently fitted to embeddings extracted from a selected hidden layer. The framework is evaluated through classification and regression simulations involving mean, covariance, and mixed distributional shifts. Additional experiments examine embedding layer and activation choices, direct neural network weight perturbations, and model changes induced by altered training conditions. Comparisons with depth-based, density-based, covariance-based, isolation-based, and end-to-end deep one-class methods show that EOCC is competitive and frequently achieves low Type II error while maintaining the nominal in-control acceptance probability, although no method is uniformly superior across all settings. Illustrative applications involving the Internet Firewall, ELEC2, and SINE1 datasets demonstrate how the framework can identify changes reflected in neural network representations. The present framework performs change detection only; adaptation and automatic model updating remain directions for future research.
Edgard M. Maboudou-Tchao, Poorna Sandamini Senaratne, Randyll Pandohie et al.· Mathematics· 0 citations