TL;DR
This work proposes two simplified attention mechanisms, Simplified Spiking Attention (SSA) and Ultra-Simplified Spiking Attention (USSA), which replace matrix multiplication with Hadamard products and eliminate hardware-unfriendly components such as multi-head attention and scaling.
Extensive experiments on static and neuromorphic benchmarks show that lower-bit BASC models match or outperform higher-bit baselines and retain this accuracy advantage after structured pruning, while further reducing model storage and synaptic operations.
PNTAFormer, which incorporates PNT-Attention, enhances token attention by integrating two critical components: (1) 2D local neighborhood aggregation through depthwise convolution to capture spatial context, and (2) excitatory-inhibitory competition driven by membrane polarity in spiking neurons, enabling the model to e...
This work introduces a minimal predictive coding framework for SNNs with two layer variants sharing a predictor block: error units, which transmit signed spiking residuals, and predictive suppression, which uses residual magnitude to dynamically gate and forward only unpredictable,"surprising"activity.
Spiking Transformers merge the energy-efficiency of spiking neural networks (SNNs) with the representational power of self-attention, creating a promising architecture for high-performance, energy-efficient computation. However, a performance gap persists versus its counterparts in artificial neural networks (ANNs). Un...
This work analyzes flaws of conventional conversion pipelines from residual membrane potential statistics and proposes a novel conversion strategy combining dynamic initial potential tuning and feature enhancement, which generalizes to ReLU CNNs, ANN Transformers, and multi-threshold SNN variants.
An in depth analysis of the different trade-offs between quantization, generalization performance, and energy efficiency between binary SNNs, multi-level SNNs and ANNs for two different applications scenarios: image classification and image denoising and results show that multi-level spiking neurons provide better info...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
