DAMR: Dual Adaptive Multi-Head Representation Learning for Multivariate Time Series Anomaly Detection

Abstract

Multivariate Time Series Anomaly Detection (MTSAD) has multiple applications in real-world environments, e.g., vehicles, industrial plants, and monitoring systems. Most existing MTSAD approaches mainly focus on Transformer-based temporal modeling and representation-based learning, yet they still ignore two key limitations, i.e., sensitivity to noise perturbations and limited diversity of learned temporal representations, which hinder fine-grained discrimination between normal and anomalous samples. To address these issues, we introduce DAMR, a novel dual adaptive multi-head representation learning framework for the MTSAD task. Specifically, we design a feature extraction module based on the Multi-Scale Adaptive Spectral Block (MSASB) to capture multi-scale temporal patterns in both time and frequency domains. Then, we adopt multiple projection heads to generate diverse representations and capture variations induced by multiple data augmentations. Furthermore, to improve feature representation flexibility, we introduce an independent adaptive temperature for each projection head to re-weight similarity scores between time series pairs. Finally, extensive experiments and comparisons are conducted on five dominant datasets. The detailed experiments show that, compared with the current promising works, our DAMR achieves significant performance improvements and strong noise robustness, e.g., +5.7% on WADI, +4.7% on MSL. Notably, compared with Transformer-based temporal modeling works, our DAMR requires far less FLOPs and GPU memory cost. The code is available at https://github.com/wyn404/DAMR.