SpikeOPD: Stable On-Policy Distillation for Autoregressive Spiking Language Models
Spiking neural networks (SNNs) offer a path to energy-efficient language modeling through sparse encoding and event-driven computation, but training capable spiking language models from scratch remains difficult. A practical alternative is ANN-to-SNN migration through knowledge distillation (KD), where a pretrained art...
En-Qiao Lu, Xingrui Yu, Yi-Wei Fu et al.
· 0 citations