Experiments show that HyBDM outperforms state-of-the-art methods in both forecasting accuracy and computational efficiency, demonstrating its effectiveness in bridging global-local dependencies for multivariate TSF.
Abstract
Time series forecasting (TSF) is vital to many applications, yet existing models often struggle to capture the heterogeneous long-range global patterns and short-range local variations in multivariate time series. While some approaches partially model these dependencies, they often do not jointly exploit temporal and feature-wise information. To address this challenge, we propose HyBDM, a multi-scale hybrid model that decomposes temporal dynamics into global patterns and local variations, which are modeled by two specialized experts. The Global Patterns Expert employs an enhanced BiConv-Mamba module that integrates bidirectional convolutions, an M-SSM layer, a forgetting mechanism, and a GDD-MLP module for cross-channel modeling. The Local Variations Expert uses a Local Window Transformer (LWT) to perform efficient locality-aware attention with reduced computational complexity. In addition, a Multi-Scale Patcher and a Long-Short Router enable multi-resolution representations and adaptive fusion of the two experts. Experiments on six benchmark datasets show that HyBDM outperforms state-of-the-art methods in both forecasting accuracy and computational efficiency, demonstrating its effectiveness in bridging global-local dependencies for multivariate TSF.
The Decomposed Recurrent Neural Network (DeRNN) is proposed, which decouples global trend modeling from local fluctuation extraction via an asymmetric dual-track architecture and exhibits superior robustness against noise and distribution shifts.
Shanyun Qian· Poster Volume 0008 The 2026...· 0 citations
Multivariate time series forecasting requires modeling both hierarchical temporal dynamics and complex inter-variable dependencies, a dual requirement that often degrades predictive performance and incurs high computational costs in standard Transformer architectures. Unlike current channel-independent models that ignore vital cross-variable synergies, or dense-attention frameworks that suffer from quadratic computational noise, our approach extracts structurally sparse dependencies. To address these specific limitations, this study introduces the FCA-Transformer. The proposed framework integrates a Feature Pyramid Network (FPN) to isolate macroscopic trends from high-frequency localized fluctuations via hierarchical downsampling. Concurrently, a structured Transformer-based Cross-Attention (TCA) mechanism employs Dimensional Segmentation with Weighting (DSW) and a Two-Stage Attention (TSA) layer to map topological variable interactions, effectively extracting robust cross-variable pathways and mitigating distributional noise. Extensive empirical evaluations across three real-world multivariate benchmarks (ETTh1, Electricity, and Exchange Rate) demonstrate that the FCA-Transformer achieves an average reduction of up to 4.39% in MSE and 5.11% in MAE compared to leading baselines. These findings indicate that the proposed architecture successfully reconciles multi-scale feature extraction with lightweight dependency modeling, enhancing structural generalization and providing a scalable framework for real-time temporal analysis in complex industrial environments.
We study the problem of multivariate time-series forecasting, where existing Transformer-based models often treat time-domain dynamics, frequency-domain structure, and cross-channel relationships as separate components, limiting effective multi-view coordination and causing susceptibility to redundant information and spurious correlations. To address this, we propose HMV-Former, a forecasting architecture that integrates multi-view hierarchical patching with lag-coupled channel attention. Specifically, we construct a Hierarchical Patching Stack to build a semantic feature pyramid across multiple scales, design Lag-Coupled Channel Attention to dynamically disentangle asynchronous lagged dependencies from time-varying coupling relations, and adopt an integrated multi-view encoder with attention-logit regularization to constrain information pathways and suppress spurious dependencies in high-dimensional spaces. Experiments on 10 real-world datasets demonstrate that HMV-Former delivers substantial improvements in forecasting accuracy and stability compared to existing methods. These results highlight its advantages in multi-scale feature abstraction and dynamic dependency governance, and suggest its potential to support intelligent decision-making in complex systems. Code and datasets are publicly available at:
https://github.com/FangXinbang/HMV-Former
.
Xinbang Fang, Tao Ma, Ke Lu et al.· Frontiers in Applied Mathema...· 0 citations
Long-term multivariate time series plays a significant role in many application areas such as power systems, trading, etc. However, their accurate prediction is quite difficult for conventional forecasting methods as they often exhibit high dimensionality and complex relationships. Recent works show that transformer-based approaches are quite effective for long-term forecasting thanks to their attention mechanism. However, in the presence of complex high-dimensional inputs, they show evidence of oversmoothing, limited capacity, and opacity. To this end, this paper introduces SETTer, a transformer-based model that addresses these challenges by incorporating novel techniques for decoupled self-attention and hybrid masking. The proposed techniques enable SETTer to effectively capture the dominant short- and long-term patterns across the temporal and channel dimensions. In addition, we enrich the model layers with simple explainable structures that indicate the discriminative pattern of SETTer. We show that with a single-layer transformer architecture, SETTer can effectively model long-term dependencies in the presence of varying data complexities. Extensive experiments on real-word benchmark datasets for long-term multivariate time series forecasting demonstrate that SETTer outperforms state-of-the-art models in 88% of the scenarios.
Abraham Ezema, C. Eze, F. Ponci et al.· 0 citations
Time-series analysis is important for various scientific and industrial fields, such as remote sensing where observations may be disturbed by clouds, have irregular revisits and different sensors. Current transformer-based approaches have enhanced the modeling of distant time relationships, but they are still constrained by specific task designs and poor adaptability to various time-series patterns. To solve these problems, we present TimeHome, a universal sparse transformer basic model for handling heterogeneous time series. TimeHome incorporates a Heterogeneous Mixture-of-Experts (H-MoE) component, where different expert types are chosen dynamically based on a low-rank temperature-controlled gating mechanism to fit various sequence features. Moreover, a hybrid local–global attention mechanism is designed to consider both short-term variations and long-distance correlations, while specific heads are used for unified prediction, missing value estimation and abnormal event detection. TimeHome is pretrained on TS-200B, a huge database of time series including various temporal patterns from different domains. Comprehensive tests on several benchmark datasets and remote sensing extended evaluations show that TimeHome performs well in long-term prediction, missing value replacement and abnormal event detection. The model also exhibits good zero-shot adaptation ability and fast inference speed by adjusting experts dynamically. The source code and pre-training data will be released publicly.
Tao Zhang, Xiaobo Wu, Xing-Guo Li et al.· Remote Sensing· 0 citations
Forecasting time series over long horizons is essential for proactive decision-making in many systems. Recent research has focused on transformer-based architectures, which capture long-range dependencies in sequential data. However, several studies show that simpler linear models can outperform transformers by avoiding overfitting during training. In this context, we present NeuroFlexMLP, a deep learning model for multivariate time series forecasting tasks. NeuroFlexMLP's key distinct feature is the adaptability to the diverse complexity of real-world time series, which is achieved, from the architecture standpoint, by adding non-linear residual blocks to a first linear block. This architectural design simplifies hyperparameter optimization, leading to accurate forecasts for various time series data types regardless of the lookback or prediction horizons, outperforming state-of-the-art (SOTA) models on challenging real-world datasets. Its Multi-Layer Perceptron (MLP) design ensures high computational efficiency, making it scalable for longer input sequences than transformer-based models. We validate NeuroFlexMLP for the LEO satellite beam hopping use case, where its lightweight design enables on-board deployment, and on state-of-the art AI datasets. Across all these benchmarks, NeuroFlexMLP achieves competitive accuracy over state-of-the-art models while providing an adaptive architecture that significantly reduces computational overhead. On the LEO beam hopping task, it achieves up to 35.9% MSE reduction over Informer, which translates into up to 28% lower provisioning cost under asymmetric cost models that penalize under-allocation more heavily than over-allocation.
P. F. Pérez, Claudio Fiandrino, Marco Fiore et al.· La Main· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
