Anomaly Detection in Long-term Time Series with Transformer and Variational Autoencoder

Abstract

Anomaly detection is crucial for ensuring system reliability and providing early warnings of potential failures. However, long-term time-series anomaly detection remains challenging because anomalous events are rare, anomaly patterns are diverse, and temporal dependencies can extend across multiple scales. Traditional methods and existing deep learning models still struggle to jointly represent fine-grained temporal variation, long-range contextual information, and the distribution of normal patterns. This work presents Transformer with Variational AutoEncoder (TransVAE), a framework that combines a dual-embedding representation pipeline, a Transformer encoder for long-range dependency modeling, and VAE-based latent regularization of encoded features. Specifically, local position-aware embeddings and feature-wise global embeddings are integrated into a shared representation before attention modeling, and the VAE is applied to the Transformer-encoded representation to regularize the latent space of normal patterns. Evaluations on benchmark time-series datasets show that TransVAE outperforms several state-of-the-art baselines, achieving improvements in the area under the receiver operating characteristic by 6.7% and 13.6%, respectively. Ablation and sensitivity analyses further support the contributions of each component of the framework. These results demonstrate that TransVAE provides an effective and robust solution for long-term time-series anomaly detection.