SegRWKV: Fast and Accurate Biomedical Image Segmentation via a Receptance-Weighted Key-Value Network

Abstract

Medical image segmentation is critical for clinical diagnosis, treatment planning, and prognosis. However, practical deployment of segmentation models is often hindered by two main challenges: (1) inefficient global dependency modeling, which slows down context integration and limits feature expressiveness; and (2) insufficient decoder feature reconstruction and multi-scale alignment, which prevents accurate recovery of fine anatomical details. These challenges are exacerbated in high-resolution medical imaging, where computational complexity and slow inference speed further limit model effectiveness. Current solutions, including CNN-based, Transformer-based, and state-space-based models, fail to simultaneously balance efficiency, fine-detail reconstruction, and scalability.To address these limitations, we propose SegRWKV, a hybrid architecture that integrates pretrained Vision-RWKV (VRWKV) modules in the encoder for efficient global dependency modeling, and a PixelRefinement module in the decoder to improve feature reconstruction and multi-scale alignment. By using recursive spatial mixing and channel-wise decoupled projections, SegRWKV reduces computational overhead while maintaining high feature expressiveness. A lightweight variant, SegRWKV-S, reduces model size by decreasing channel width and convolutional depth, achieving compression without sacrificing accuracy.Comprehensive experiments on several medical imaging datasets, including AbdomenMRI, Endoscopy, and Microscopy, demonstrate that SegRWKV consistently outperforms CNN-, Vision Transformer-, Mamba-, and RWKV-UNet-based baselines. Specifically, SegRWKV-B improves DSC/F1 scores by 4–22% over CNNs, 8–58% over Transformers, 1–8% over Mamba, and 10–27% over RWKV-UNet. Efficiency analysis shows that SegRWKV delivers 4.8 × faster inference and reduces GPU memory usage by 4.5 × compared to existing architectures, with stable computational scaling as input resolution increases. These results highlight SegRWKV’s superior ability to balance accuracy, efficiency, and scalability, particularly in high-resolution medical image segmentation tasks, where many attention-based models struggle to scale effectively. Code and pretrained models are available at: https://github.com/luminescentfrr/SegRWKV/.