Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· pp. 13360-13364· 0 citations· 35 references
TL;DR
This tutorial covers recent developments in OOD detection from both theoretical and practical perspectives, including four major categories: (1) post-hoc methods, (2) training-based methods with auxiliary outliers, (3) training methods without auxiliary outliers, and (4) foundation model-based approaches, along with recent advances in each area.
Abstract
Despite their remarkable success, Deep Neural Networks (DNNs) remain challenging to deploy in critical applications due to their inability to handle out-of-distribution (OOD) data. DNNs are trained assuming training and deployment data follow the same distribution, but this assumption is often violated in practice. When encountering unfamiliar inputs, models may produce highly confident yet incorrect predictions, and in high-stakes decision-making environments, such incorrect predictions can be costly. OOD detection has thus become fundamental across deep learning, affecting applications from computer vision and NLP to security, autonomous systems, and generative models. This tutorial covers recent developments in OOD detection from both theoretical and practical perspectives, including four major categories: (1) post-hoc methods, (2) training-based methods with auxiliary outliers, (3) training methods without auxiliary outliers, and (4) foundation model-based approaches, along with recent advances in each area.
Reliable detection of out-of-distribution (OOD) samples is crucial for the safe deployment of machine learning models. Neural networks often produce overconfident predictions for inputs that deviate from their training data, leading to significant degradation in performance. While many OOD detection methods focus on the final output layer, they neglect the rich hierarchical information present in intermediate network layers. This paper introduces a novel approach that leverages sparse autoencoders (SAEs) to learn interpretable features from these intermediate activations. We find that in-distribution (ID) and OOD data activate distinct sets of these sparse features. We propose a new OOD score derived from the cosine similarity between the sparse feature activations of a test sample and the mean activations of ID classes. Our post-hoc detection method not only achieves state-of-the-art performance on standard OOD detection benchmarks, but yields interpretable insights into how distribution shift affects learned representations.
Ayush Karmacharya, Luke Luschwitz, Lucia Romero et al.· 0 citations
This comprehensive survey formally defines the CTTA problem, analyzes the diverse continual domain shift patterns that characterize different evaluation protocols, and proposes a hierarchical taxonomy that categorizes existing methods into three families: optimization-based strategies (entropy minimization, pseudo-labeling, parameter restoration), parameter-efficient methods (normalization layer adaptation, adaptive parameter selection), and architecture-based approaches (teacher-student frameworks, adapters, visual prompting, masked modeling).
The findings demonstrate that introducing prior structure, through strategic sparse supervision, pre-trained multimodal knowledge, or transferable representations, consistently compensates for limited labeled data across diverse vision tasks.
In this paper, we propose MuST-VAD, a mutual structured learning framework for weakly supervised video anomaly detection (VAD) in which an anomaly detector and a large vision-language model (LVLM) exchange their acquired knowledge. Detectors in weakly supervised VAD learn anomaly scores from features extracted by a fixed, task-agnostic backbone. These fixed features bound the achievable detection accuracy. Recent methods therefore transfer LVLM semantics into the detector as richer features. However, this transfer is one-way: what the detector learns about the target videos never returns to the LVLM. MuST-VAD extends the one-way transfer into a bidirectional learning loop. In this loop, the latest detector predictions supervise the LVLM adaptation, and the adapted LVLM returns updated representations that retrain the detector; the two models alternate these updates over small video groups. Both models train on detector-selected key clips, while confidence weighting and annotation-anchored question answering keep the exchanged supervision reliable. On UCF-Crime, our mutual learning improves the one-pass transfer baseline from 88.15% to 88.63% AUROC and from 37.25% to 42.46% average precision (AP), outperforming the state-of-the-art method in AP by 4.13 points.
Satoshi Hashimoto, Hitoshi Nishimura, Mori Kurokawa· 0 citations
The performance of deep neural networks (DNNs) is strongly influenced by the characteristics and quality of the underlying datasets. This Ph.D. dissertation addresses three pervasive data challenges-imbalance, quality degradation, and scarcity-that commonly hinder the effectiveness of DNNs in computer vision (CV) and natural language processing (NLP) applications.
Class imbalance remains one of the most frequent causes of degraded model generalization. While Focal Loss effectively mitigates inter-class imbalance by assigning higher weights to minority classes, it struggles with intra-class imbalance, particularly in video datasets where longer clips dominate feature representation. To address this, I implement and utilize the AD-CORRE Loss, which operates at the mini-batch level to balance intra-class feature contributions with minimal computational overhead. Combined with Focal Loss, this dual-loss framework yields more stable and robust training, as demonstrated by improved empirical results across multiple CV benchmarks.
Data quality represents another major factor influencing learning outcomes. Blurred or low-resolution videos and images reduce feature richness and lead to biased feature extraction. To account for this, I introduce the Combined-SSL (Self-Supervised Learning) mechanism, which jointly models video quality and classification. The integration of quality-aware supervision significantly enhances recognition performance on challenging datasets. Moreover, in an NLP setting, I apply a RoBERTa-CNN model to detect suicide intentions from well-cleaned social media posts, further demonstrating the critical role of data quality in reliable model prediction.
Data scarcity poses a fundamental limitation for supervised learning, especially when labeled samples are rare or sequences are short. To alleviate this, I design a multi-task learning framework that integrates auxiliary tasks-Masked Language Modeling (MLM) and S/TP prediction-within the ProtBERT backbone. Self-supervised and physics-informed tasks enrich the feature space and improve generalization. Experimental results on IDP datasets confirm that the proposed Multi-task ProtBERT effectively mitigates data scarcity and achieves state-of-the-art performance.
Overall, this dissertation provides a unified investigation into data imbalance, data quality, and data scarcity-three core bottlenecks of modern deep learning-and proposes principled solutions that improve robustness, interpretability, and efficiency across both CV and NLP domains.
Recognizing out-of-distribution (OOD) samples is essential for deploying robust machine learning systems in open-world environments. While conventional OOD detection approaches rely on feature representations from the penultimate layer of neural networks, they often overlook informative signals embedded in intermediate layers. In this paper, we present a straightforward feature mixing approach for pre-trained Transformers, which combines multi-layer representations via calculated importance weights, and identifies OOD samples using Mahalanobis distance in the blended feature space. When in-distribution samples are accessible, we show that parameter-efficient fine-tuning strategies effectively balance classification accuracy and OOD detection performance. We conduct extensive empirical analyses to validate the superiority of our proposed method under zero-shot, and fine-tuning settings using both class-balanced and long-tailed datasets. The source code is available at https://github.com/SEUML/X-Maha .
Tong Wei, Bolin Wang, Jiang-Xin Shi et al.· Neural Information Processin...· 8 citations· ⚡2