Efficient Transformer Attention for SNNs via Hadamard Simplification
This work proposes two simplified attention mechanisms, Simplified Spiking Attention (SSA) and Ultra-Simplified Spiking Attention (USSA), which replace matrix multiplication with Hadamard products and eliminate hardware-unfriendly components such as multi-head attention and scaling.
Ting-Ting Jiang, Jiangrong Shen, Long Chen et al.
· 1 citation
· ⚡1