PatchCLE: Breaking the Linear Representation Bottleneck in Time Series Forecasting via Soft Contrastive Learning

Abstract

Although Patch-based Transformers have achieved remarkable success in time series forecasting, the conventional linear embedding layer remains a critical bottleneck, as it fails to map patches into a structured latent space. To address this issue, a novel representation-enhanced forecasting model, termed PatchCLE (Patch Contrastive Learning Encoder), is proposed, which innovatively integrates soft contrastive learning into multivariate time series forecasting in order to improve representation quality. Meanwhile, patch-based soft contrastive learning effectively mitigates the susceptibility of conventional time-step-based methods to noise and abrupt mutations. Furthermore, a three-level contrastive learning module named PCL (Patch Contrastive Learning) is proposed, featuring a newly designed channel level alongside the temporal level and instance level, thereby enriching the semantic information of the latent space. In addition, a new soft weight assignment strategy, namely Decomp, is introduced, which effectively resolves the computational complexity challenges associated with instance-level soft weight assignment. Finally, the representations are fused through a Channel Mixture module to aggregate inter-channel information, and are subsequently fed into Transformer blocks to produce the final forecasting outputs. Extensive experiments on forecasting tasks across diverse domains demonstrate state-of-the-art performance, with an average improvement of 4.5%, while also validating the representation capabilities of the PCL module as a generic plug-in component that boosts various baseline models by 2.3%. Our code is available at https://github.com/Ahouse-Mao/PatchCLE.