SO(3)-based and structure-guided deformable registration for respiratory motion correction in thoracic PET

Abstract

Respiratory motion in thoracic positron emission tomography (PET) introduces spatially heterogeneous non-rigid deformation that can blur lesions, weaken local boundary definition, and reduce structural fidelity. To address this problem, we developed TLCE-morph, a Tri-Path Lie Convolution Encoder-based learning framework for deformable respiratory motion correction in thoracic PET. The framework combines an SO(3)-based group-aware convolution module with a Tri-Path Fusion Encoder to couple orientation-aware geometric modeling with structurally guided feature encoding at local, global, and cross-scale levels. TLCE-morph was evaluated on simulated respiratory motion datasets and a two-center clinical gated PET cohort using Dice, correlation coefficient, and 95th percentile Hausdorff distance. Additional analyses included lesion-level normalized PET uptake consistency, local line-profile and full width at half maximum measurements in motion-sensitive regions, architectural ablation, group-representation comparison, and computational profiling. Across the simulated datasets, TLCE-morph remained comparatively stable as deformation increased from relatively regular displacement to more heterogeneous and coupled motion. In the clinical gated PET cohort, it achieved the most favorable overall quantitative performance among the evaluated methods and showed more consistent local structural recovery in representative motion-sensitive regions. Additional comparisons of group representations and architectural ablation indicated that the observed advantage was associated with the joint contribution of 3D orientation-aware feature modeling and complementary structural constraints rather than with any single component alone. These findings suggest that stable respiratory motion correction in thoracic PET may benefit from coupling geometric sensitivity with structurally guided feature encoding under heterogeneous deformation, rather than relying on appearance matching alone.