CroSpec-MT: A Compact Spectral State-Space Network for Joint Classification and Regression of Hyperspectral Forest Variables

Abstract

Hyperspectral imagery enables simultaneous estimation of categorical and continuous forest variables, yet most deep learning approaches address these tasks in isolation and rely on heavy single-branch encoders. We propose CroSpec-MT, a compact multitask network that learns from hyperspectral patches via three parallel, complementary branches: a spatial branch with residual convolution, dense atrous spatial pyramid pooling (DenseASPP), and convolutional block attention module (CBAM); a spectral branch built on a bidirectional selective state-space model (BiSSM) with a learnable forward–backward fusion gate; and a parameter-light frequency branch based on a nonlearnable Type-II discrete cosine tranform (DCT-II) of the spectral axis. Branch outputs are combined by element-wise summation, globally pooled, and routed through task-specific trunks followed by a task-affinity cross-decoder (TACD) that enables curriculum-gated bidirectional interaction between the classification and regression streams. Experiments on the TAIGA airborne hyperspectral dataset (three categorical and ten continuous forest variables), with five random seeds under a strict spatial-block split, show that CroSpec-MT achieves the best average accuracy (AA), Cohen’s Kappa, macro $F1$ , R2, root-mean-square error (RMSE), and normalized RMSE among five state-of-the-art multitask baselines, at only 1.39 M parameters.