SpikeRestormer is proposed, an energy-efficient SNN for AiOIR that performs event reasoning over internally generated spike cues that delivers competitive performance against ANN-based methods and establishes new state-of-the-art results among SNN-based methods with significantly lower energy consumption.
Abstract
ANN-based All-in-One image restoration (AiOIR) unifies diverse degradation handling but incurs high computational costs, limiting its real-time deployment. While Spiking Neural Networks (SNNs) offer a low-power alternative, applying them to static images remains challenging. This difficulty arises because explicit event signals are absent, and degradation cues are heavily entangled with scene structures, hindering the learning of reliable restoration-oriented spike events. To address these issues, we propose SpikeRestormer, an energy-efficient SNN for AiOIR that performs event reasoning over internally generated spike cues. Specifically, we propose a degradation-event perception process to extract spike-based degradation events through Subtractive Degradation Event Attention (SDEA). Moreover, we introduce Hierarchical Bayesian Skip Masking (HBSM) and Additive Restoration Event Attention (AREA) processes for event-reliability inference and restoration-event construction, respectively. By integrating these complementary processes, SpikeRestormer formulates restoration as a unified process of degradation-event perception, degradation-event reliability inference, and restoration-event construction, liberating the potential of SNNs for energy-efficient AiOIR. Extensive experiments show that SpikeRestormer delivers competitive performance against ANN-based methods and establishes new state-of-the-art results among SNN-based methods with significantly lower energy consumption.
Semantic segmentation is critical for intelligent robotics to understand complex environments. While CNN-based models on RGB images achieve high performance, their accuracy drops in fast-motion or low-light scenes. Fortunately, event cameras, with high temporal resolution and low latency, offer robust perception in such challenging conditions. Many event-image fusion methods attempt to combine the complementary strengths of both modalities, but most adopt simple fusion strategies without considering intermodal correlations or designing computationally expensive architectures, resulting in degraded accuracy and high energy costs. To overcome these limitations, we propose a lightweight spiking neural network (SNN)-based event-image fusion network (Spike-EIFNet) that leverages the complementary strengths of multimodal fusion and energy-efficient spike-driven computation. In particular, to reduce computation cost for lightweight, Spike-EIFNet adopts a dual-branch SNN encoder to process events and images in parallel. Then, to improve the segmentation accuracy with enhanced feature interaction, we introduce a spike-driven cross-modal fusion (SCMF) module, consisting of a modality-aware fine-grained extraction (MFE) stage to capture dynamic cues from events and spatial details from images, followed by a cross-modal interaction and fusion (CIF) stage for effective feature alignment. Finally, a lightweight feature enhancement (LFE) module is proposed to further refine feature representations and facilitate deep-shallow feature fusion. Extensive experiments demonstrate that Spike-EIFNet achieves 67.34% and 58.09% mean intersection over union (mIoU) on the DDD17 and DSEC-Semantic datasets while consuming $72.83\times $ and $100.26\times $ less energy, respectively. Compared with ANN-based methods, Spike-EIFNet significantly reduces energy consumption; among SNN-based methods, it achieves the highest segmentation accuracy with a favorable accuracy-efficiency tradeoff. Code is available at: https://github.com/Chensyfighting/Spike-EIFNet.
Siyu Chen, Qie Liu, Xianlei Long et al.· IEEE Transactions on Neural...· 0 citations
This work pioneers the synergistic integration of SNNs into Transformer architectures for LLIE, establishing a compelling pathway toward powerful, energy-efficient low-level vision on resource-constrained platforms.
Hongzhi Wang, Xiubo Liang, Jinxing Han et al.· Advances in Neural Informati...· 0 citations
A compact and configurable event-driven autoencoder that efficiently compresses neuromorphic data while preserving essential spatiotemporal structure for downstream inference and demonstrates the potential of compact event-driven models to advance environmentally conscious, low-power AI systems for high-speed perception in autonomous, mobile, and embedded computing environments.
Riadul Islam, Joey Mulé, Dhandeep Challagundla et al.· 0 citations
CIS-Fuse is proposed, a spiking network that performs cross-modal fusion directly at the membrane-potential level while inheriting the energy efficiency of spike-based computation, with roughly an order of magnitude lower inference energy than the similarly-sized ANN-based DCEvo.
Rui Zhao, Zhuoyuan Li, Wenrui Li et al.· 0 citations
Although deep learning-based object detectors have achieved remarkable performance on standard benchmarks, their reliability deteriorates significantly under adverse weather conditions (e.g., rain, snow, and fog) due to severe image degradation and the loss of critical structural information. To address this challenge, this paper proposes GPE-YOLO, a robust detection framework built upon the YOLOv11 architecture that explicitly integrates multiscale edge priors to enhance feature resilience. Specifically, we introduce three novel components: 1) Dual-Stream Gradient-Pooling Stem (DSGPS) serves as an efficient preprocessing unit, utilizing parallel 3D Sobel and spatial convolutions to capture comprehensive low-level features and mitigate noise propagation at the input stage. 2) The Pyramidal Edge Prior Generation Module (PEPGM) extracts multi-scale edge maps from shallow layers and injects them into the backbone network via max pooling, effectively suppressing background interference while preserving structural details. 3) The Dynamic Edge-Aware Feature Pyramid (DE-AFP) incorporates DySample and CSPStage to optimize feature upsampling and downsampling processes, thereby ensuring efficient semantic information transmission and robust multi-scale fusion across the network. Extensive experiments on the VOC2012-CWC dataset demonstrate the superiority of our method. GPE-YOLO achieves an mAP@0.5 of 32.84% and mAP@0.5:0.95 of 24.59%, outperforming the baseline YOLOv11 by substantial margins of 4.85% and 3.86%, respectively. These results validate the potential of GPE-YOLO for reliable deployment in real-world adverse weather scenarios.
Xiaojie Chen, Yifei Zhou, Yiming Zhou et al.· International Conference on...· 0 citations