A spiking neural network based on hierarchical residual quantization and temporal error compensation (HRQ-TEC-SNN) and used for remote sensing object detection tasks is proposed and the model’s detection accuracy has been further improved while reducing the number of model parameters and computational costs.
Abstract
Compared with traditional artificial neural networks (ANNs), spiking neural networks (SNNs) have lower computational complexity, lower energy consumption, and faster inference speed, making them more promising for practical deployment on edge devices. However, in SNNs, since the output spikes of neurons are discrete, the model may face information loss, especially when the membrane potential is quantized into binary spikes, where quantization errors can lead to model precision loss and information loss. To address these challenges, this study proposes a spiking neural network based on hierarchical residual quantization and temporal error compensation (HRQ-TEC-SNN) and used for remote sensing object detection tasks. Through hierarchical residual quantization and temporal error compensation design, higher resolution quantization of membrane potential can be performed and the quantization threshold of membrane potential can be dynamically adjusted to compensate for errors introduced during the quantization process, allowing for fine-tuning at each time step and reducing information loss caused by coarse quantization. In terms of network structure, by introducing depthwise separable convolution modules, channel attention and spatial attention mechanisms, and integrating fast spatial pyramid pooling based on pulse neural networks, the model’s detection accuracy has been further improved while reducing the number of model parameters and computational costs. Experimental results show that the HRQ-TEC-SNN achieves significant advantages in both accuracy and energy consumption on the DOTAv1.0, DOTAv1.5 and DIOR datasets.
Spiking Neural Networks (SNNs) are one of the most promising bio-inspired neural networks models and have drawn increasing attention in recent years. The event-driven communication mechanism of SNNs allows for sparse and theoretically low-power operations on dedicated neuromorphic hardware. However, the binary nature of instantaneous spikes also leads to considerable information loss in SNNs, resulting in performance degradation. To address this issue, multi-level spiking neuron models have been proposed to provide both low-quantization error and minimal inference latency while approaching the performance of full precision Artificial Neural Networks (ANNs). In this paper, we provide an in depth analysis of the different trade-offs between quantization, generalization performance, and energy efficiency between binary SNNs, multi-level SNNs and ANNs for two different applications scenarios: image classification and image denoising. Our goal is to determine analytical thresholds on SNNs activity to understand when SNNs can be more energy efficient than ANNs but also to explain why SNNs cannot provide energy gains in other scenarios. Our analysis is based on an hardware execution model, which allows us to take into account the energy costs related to synaptic operations as well as memory accesses. Our experimental results show that multi-level spiking neurons provide better information compression, allowing therefore a reduction in latency without performance loss for classification tasks. When compared to binary SNNs on image classification scenarios, multi-level SNNs indeed allow reducing by 2 to 3 times the energy consumption depending on the number of quantization intervals. On neuromorphic data, multi-level SNNs allows us to drastically reduce the inference latency to 1 timestep, which corresponds to a compression factor of 10 compared to previously published results on binary SNNs. However, for the image denoising task we observe that neither the binary nor the multi-level SNNs can match the ANNs energy efficiency and performance. Based on a hardware-aware execution model of a neuromorphic SNN accelerator, we then provide analytical upper-bounds on SNNs activity and energy-efficiency. These theoretical bounds are then compared against the experimental results and allows us to understand, from a system level point of view, the relation between energy efficiency, quantization schemes and finally the performance of the SNNs.
A. Castagnetti, Alain Pegatoquet, Benoît Miramond· IEEE Journal on Selected Are...· 0 citations
The second-generation neural networks have evolved in recent years, which have become more complex architectures such as spiking neural networks and quantum neural networks. However, the computational resource restriction of neural networks on edge devices is still challenging. The thesis investigates stable learning and compute-resource efficiency on spiking neural networks and hybrid classical-quantum neural networks. Other common qualities like high performance (e.g. high accuracy, high reward), robustness, convergence, predictability, and fast running times were also considered in one or more studies. The contributions of the thesis have several folds. The first study was using audio data; one reason was to verify if a trend called temporal information concentration is present in the spiking neural network. I also gathered other findings, such as dataset complexity impacting Fisher information, related to temporal information dynamics. A second study on spiking neural networks revealed that temporal information concentration was not present in quantization aware training variants, but an increase in Fisher information was found in those variants. The third study on my Multimodal Simplified Spiking Neural Networks explored the effects of audio and image noise. The results show the multimodal model outperformed its unimodal counterparts, but certain configurations of image noises, audio noises, and noise levels performed better than others. In one of the hybrid classical-quantum neural network studies with reinforcement learning, in my quantization-aware training variant, I found higher initial reward growth, longer decreasing in standard deviation and policy entropy, and a few correlations as well related to average reward and policy entropy. In the second study on hybrid classical-quantum neural networks, structured pruning is found to sharpen decisiveness and reveal bad pruning paths, while overparameterization can help exploration. All these studies try to address maintaining or improving stable learning and if the models are computation-resource efficient enough to be realistic.
PTQ4SNN is proposed, a membrane-aware post-training quantization framework that jointly quantizes weights and recurrent membrane states using only a small calibration set and effectively preserves model accuracy under W4 quantization and approximately 4-bit membrane precision.
Hui Xie, Tong Shi, Haotong Qin et al.· 0 citations
Extensive experiments on static and neuromorphic benchmarks show that lower-bit BASC models match or outperform higher-bit baselines and retain this accuracy advantage after structured pruning, while further reducing model storage and synaptic operations.
Linliang Chen, Yan Zhong, Xin Liu et al.· 0 citations
This work proposes the use of ternary spike neurons, which enhance the information-carrying capacity in the spiking neural rendering model, and introduces Spik-NeRF (Spiking Neural Radi-ance Fields with Ternary Spike), which achieves rendering performance comparable to ANN-based NeRF models.
Gang Wan, Qinlong Lan, Zihan Li et al.· Neural Information Processin...· 1 citation
Spiking Neural Networks (SNNs) serve as core architectures for neuromorphic computing thanks to event-driven operation and ultra-low power consumption. Direct SNN training is hindered by non-differentiable spikes that induce vanishing gradients and unstable optimization. ANN-SNN conversion circumvents such issues by reusing well-trained ANN weights for low-latency, energy-efficient inference. Nevertheless, existing conversion schemes suffer from severe accuracy drops at small timesteps, large inference delays and cumulative quantization errors, even with marginal performance loss at large $T$. To address these limitations, we first analyze flaws of conventional conversion pipelines from residual membrane potential statistics and propose a novel conversion strategy combining dynamic initial potential tuning and feature enhancement. We then introduce a regularization loss $\mathcal{L}_{\mathrm{RMPD}}$ to adapt initial potential of IF neurons and mitigate systematic truncation bias from boundary aggregation. A dedicated SCR-Conv2d competitive refinement layer with grouped convolution is further built to sharpen feature discrimination, eliminate redundant spikes and stabilize encoding under tiny time windows. Integrated with the state-of-the-art QCFS baseline, our approach delivers consistent low-latency performance gains and generalizes to ReLU CNNs, ANN Transformers, and multi-threshold SNN variants. Evaluations on CIFAR-10, CIFAR-100 and ImageNet verify prominent accuracy improvements at $T=2,4,8$, with negligible extra computation overhead. This work offers an effective conversion paradigm to facilitate real-world SNN deployment on neuromorphic chips.
Zirui Chen, Zihan Huang, Tong Bu et al.· 0 citations