TriFusion: A Unified CNN-Transformer Framework for Multiscale and Spectral-Spatial Learning in Hyperspectral Image Classification

Abstract

Hyperspectral image (HSI) classification remains challenging due to two fundamental issues: first, severe spectral redundancy and mixed spectral–spatial noise that degrade discriminability, and second, the difficulty of simultaneously capturing fine-grained local patterns and long-range contextual dependencies under limited labeled data.Existing convolutional neural network (CNN)-based methods excel at local feature extraction but fail to model global interactions, while Transformer-based models provide strong global modeling but are data-hungry and often overlook local spectral–spatial cues. To address these problems, we propose TriFusion, a unified CNN–Transformer hybrid framework that systematically integrates multiscale and spectral–spatial learning within a coherent fusion architecture. Specifically, an efficient spectral–spatial convolutional block is introduced to disentangle and enhance local spectral–spatial features while suppressing redundancy. A multiscale context module is further employed to aggregate contextual information. In addition, a deep feature pyramid progressively fuses hierarchical representations to preserve discriminative spectral–spatial semantics across scales. A lightweight Transformer encoder is finally incorporated to capture long-range dependencies. Extensive experiments on four widely used benchmarks (Pavia University, Salinas, WHU-Hi-Longkou, and Xuzhou) demonstrate that the proposed TriFusion framework consistently achieves the best overall accuracy among recent CNN-based, Transformer-based, and hybrid methods, with gains of 0.26–1.42 percentage points over the strongest competing baseline across the four datasets. Ablation studies and efficiency analyses further validate the effectiveness of each fusion component and the practical efficiency of the proposed design.