EdgeFormerNet++: A Hybrid Attention-Guided Transformer Network for Accurate Retinal Layer Segmentation in OCT Images

Abstract

Accurate retinal layer segmentation in OCT is crucial for the early detection and monitoring of retinal and neurodegenerative diseases, yet it remains challenging due to the thin, low-contrast structures and closely packed layers in multi-class settings. We propose EdgeFormerNet++, a hybrid framework that integrates Res2Net-based multi-scale encoding, lightweight Transformer bottlenecks for global context, and dual attention (CBAM and scSE) for spatial-channel recalibration, complemented by an edge attention module to refine layer boundaries. The model is evaluated on two public datasets (NR206 and MGU) covering macular and peripapillary regions, achieving state-of-the-art Dice scores of 92.34% and 83.28%, respectively. Qualitative and quantitative results demonstrate accurate delineation of complex retinal structures, supporting the use of EdgeFormerNet++ for automated OCT analysis and computer-assisted diagnosis.