Jun 2026· arXiv.org· Vol abs/2606.31695· 0 citations· 50 references
Computer Science
TL;DR
The Intrinsically Stable SNN (IS-SNN) architecture is proposed, which removes activation-normalization layers by enforcing signal homeostasis through topology-aware weight standardization and modified residual connections and removes the runtime statistics tracking and multiplications introduced by activation normalization, restoring an accumulation-oriented inference datapath.
Abstract
The performance of deep spiking neural networks (SNNs) often relies on batch normalization (BN). However, the advanced dynamic BN variants used in state-of-the-art models introduce runtime multiplications, which weaken the hardware-efficiency motivation of SNNs. To address this tension, we identify catastrophic firing-rate decay as a primary cause of severe performance degradation in normalization-free SNNs. Guided by this insight, this work proposes the Intrinsically Stable SNN (IS-SNN) architecture, which removes activation-normalization layers by enforcing signal homeostasis through topology-aware weight standardization and modified residual connections. By folding the standardization operations into static weights offline, IS-SNN removes the runtime statistics tracking and multiplications introduced by activation normalization, restoring an accumulation-oriented inference datapath. Comprehensive experiments show that IS-SNN achieves performance competitive with or superior to computationally expensive dynamic BN techniques across VGG, ResNet, and Transformer-based models. Notably, it achieves a competitive accuracy of 68.05\% on ImageNet and overcomes the severe depth limitations of prior BN-free attempts. Together with a 96.4\% reduction in FPGA lookup table resource consumption for neuron implementations, these results support IS-SNN as a practical framework for building accurate and hardware-friendly deep neuromorphic systems.
Spiking Neural Networks (SNNs) serve as core architectures for neuromorphic computing thanks to event-driven operation and ultra-low power consumption. Direct SNN training is hindered by non-differentiable spikes that induce vanishing gradients and unstable optimization. ANN-SNN conversion circumvents such issues by reusing well-trained ANN weights for low-latency, energy-efficient inference. Nevertheless, existing conversion schemes suffer from severe accuracy drops at small timesteps, large inference delays and cumulative quantization errors, even with marginal performance loss at large $T$. To address these limitations, we first analyze flaws of conventional conversion pipelines from residual membrane potential statistics and propose a novel conversion strategy combining dynamic initial potential tuning and feature enhancement. We then introduce a regularization loss $\mathcal{L}_{\mathrm{RMPD}}$ to adapt initial potential of IF neurons and mitigate systematic truncation bias from boundary aggregation. A dedicated SCR-Conv2d competitive refinement layer with grouped convolution is further built to sharpen feature discrimination, eliminate redundant spikes and stabilize encoding under tiny time windows. Integrated with the state-of-the-art QCFS baseline, our approach delivers consistent low-latency performance gains and generalizes to ReLU CNNs, ANN Transformers, and multi-threshold SNN variants. Evaluations on CIFAR-10, CIFAR-100 and ImageNet verify prominent accuracy improvements at $T=2,4,8$, with negligible extra computation overhead. This work offers an effective conversion paradigm to facilitate real-world SNN deployment on neuromorphic chips.
Zirui Chen, Zihan Huang, Tong Bu et al.· 0 citations
Extensive experiments on static and neuromorphic benchmarks show that lower-bit BASC models match or outperform higher-bit baselines and retain this accuracy advantage after structured pruning, while further reducing model storage and synaptic operations.
Linliang Chen, Yan Zhong, Xin Liu et al.· 0 citations
This work proposes Burst Spiking Neural Networks (BuSNNs), built upon Burst-enhanced Spiking Neurons and a Dynamic Weight Constraint (DWC) mechanism, which mitigates perturbation-induced transitions in activation states and thereby enhances robustness.
Jiahong Zhang, Sijun Shen, Man Yao et al.· 0 citations
Spiking neural networks (SNNs) offer an energy-efficient alternative to conventional deep neural networks by exploiting sparse event-driven computation, but their training remains challenging because the non-differentiable spike function requires surrogate gradients whose fixed shape may be suboptimal across layers and training stages. In this work, we introduce SAGE, an uncertainty-modulated surrogate-gradient mechanism for Transformer-based SNNs. SAGE estimates block-level uncertainty from normalized self-attention entropy and uses this signal to adapt the surrogate-gradient slope during training while leaving the inference model unchanged. By modulating only the training-time surrogate parameter, the proposed method preserves the original architecture and deployment cost while improving optimization flexibility. Experiments on CIFAR-10/100 demonstrate that SAGE achieves improved accuracy over fixed-surrogate baselines, with results up to 1-2\% consistent gains across multiple simulation time steps. These results highlight the potential of attention-derived uncertainty as a lightweight training signal for adaptive surrogate-gradient learning in transformer-based SNNs.
A Spiking Neural Network (SNN) is a kind of brain-inspired and event-driven network, which is becoming a promising energy-efficient alternative to Artificial Neural Networks (ANNs). In recent years, SNN methods have been successfully applied in the fields of electromagnetic signal processing and image signal processing, particularly in application scenarios that require low energy consumption. However, the performance of SNNs by direct training is far from satisfactory. In this paper, we study a novel learning method named SAD-SNN (Spatial-Activation Distillation for Spiking Neural Networks), which utilizes the ANN model to guide the SNN model learning. Unlike prior works that rely on element-wise feature alignment approaches, SAD-SNN aligns spatial-activation maps at different resolutions of the teacher and student networks. Specifically, we introduce a direct alignment approach, which defines a spatial-activation loss and normalizes the representation vectors of ANN and SNN, to alleviate the unexpected precision loss. This enables the knowledge of teacher ANNs to be effectively transferred to train student SNNs. On three image classification datasets, our proposed SAD-SNN outperforms other SNN training methods no matter whether homogeneous or heterogeneous teacher ANNs are used. Furthermore, we apply SAD-SNN to the electromagnetic signal detection task, demonstrating strong generalization ability and superior performance. In conclusion, the experimental results on various tasks and SNN architectures demonstrate that our method is a general and effective solution that significantly improves the learning of student SNNs with only two time steps.
Chongxiao Qu, Qian Zhang, Chenxiao Dou et al.· Italian National Conference...· 0 citations
Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task. Holding architecture fixed and swapping only the hidden unit (continuous vs. leaky-integrate-and-fire), plus a two-sided target-firing-rate probe, we measure how far activity can be pushed down before quality breaks. Low-load feed-forward perception sparsifies to 5% firing at no accuracy cost; a recurrent language model cannot go below ~50% -- the recurrent state must stay active to carry information. A spiking Transformer, by contrast, sparsifies freely to 2% (3 seeds) -- so the ceiling is a property of recurrent compression, not sequence modeling. Attention escapes the floor only by storing the full key-value cache, trading a firing floor for a memory wall: on neuromorphic hardware, recurrence and attention pay on different axes, neither escapes. We formalize the ceiling with an information-theoretic bound rho>= H_b^{-1}(log2 M / H) and confirm its predictions: the floor rises with memory load, falls with state width, and (refuting a naive memory-only reading) rises with task difficulty. A layer-wise input floor further caps op reduction under dense input, isolating event-driven perception as where neuromorphic hardware wins.