Pseudo-label distillation for discriminative anomalous sound detection
TL;DR
Pseudo-label distillation not only transfers the performance of SSL models to a compact model but also further improves performance by leveraging available coarse labels and data augmentation.
Abstract
Discriminative anomalous sound detection (ASD) methods train a feature extractor through a classification task using machine-information labels. They then detect anomalies in the resulting feature space based on distances to normal samples. The discriminative feature space effectively captures machine characteristics, leading to high ASD performance. However, this approach benefits from detailed labels, which are costly to obtain. An alternative is a self-supervised learning (SSL)-based label-free approach. This approach directly uses SSL features for ASD and has shown competitive performance. However, SSL models are typically large and computationally expensive. To address these problems, we propose a simple pseudo-label distillation framework. The proposed method generates pseudo labels from SSL features and trains a compact discriminative feature extractor using these pseudo labels. To suppress the effect of noise on pseudo-label generation, we also propose lightweight noise-robust feature transformation (NRFT) methods utilizing a small amount of clean machine-sound data or isolated noise data. We conducted comprehensive evaluations and analyses on the DCASE 2020-2025 Task 2 datasets using four SSL models. The results demonstrate that pseudo-label distillation not only transfers the performance of SSL models to a compact model but also further improves performance by leveraging available coarse labels and data augmentation. Also, our NRFT methods provide further gains.