Jul 2026· Annual International Computer Software and Applications Conference· pp. 2903-2908· 0 citations· 21 references
Abstract
Efficient text embedding is critical for large-scale natural language processing (NLP) applications, where storage, computation, and scalability are key concerns. While binary representations offer a compact alternative to real-valued embeddings, existing approaches typically rely on fixed or heuristic thresholding schemes that fail to capture task-relevant structure in the data. In this paper, we propose a spiking neural network (SNN) based framework for learning binary representations of pretrained NLP embeddings, where binary spiking codes are learned directly from continuous transformer embeddings using surrogate gradient optimization. Unlike conventional threshold based binarization methods such as min-max scaling, Otsu thresholding, and hybrid heuristics, the proposed approach predicts the projection parameters and these are optimized jointly with the downstream task, enabling adaptive and task-aware binarization. The binary embedding behaves like a text DNA. We evaluate our method on a wide range of benchmark NLP datasets, including IMDb, AG News, SNLI, and additional text classification tasks, and compare it against multiple heuristic binarization techniques as well as real-valued embeddings. Extensive experimental results demonstrate that the proposed spiking binary representations consistently achieve superior classification performance, outperforming existing thresholding methods and, in several cases, exceeding the performance of real-valued embeddings.
Dynamic graph learning aims to capture evolving structural and semantic patterns in real-world systems, such as fraud detection and recommender systems. Due to the scarcity of labeled data in real-world dynamic graphs, recent studies have introduced generative or contrastive paradigms (e.g., masked graph autoencoders or graph contrastive learning) to generate task-agnostic graph embeddings. However, these methods typically rely on complex edge-level reconstruction objectives and tailored graph augmentation strategies. This incurs substantial computational overhead when scaling to large-scale dynamic graphs. In this paper, we propose SG-JEPA, a joint spiking embedding predictive architecture for large-scale dynamic graphs. In contrast to existing self-supervised methods, SG-JEPA partitions nodes into context and target sets along the temporal dimension to learn embeddings that are predictive of each other via additional spatial-temporal information. Furthermore, through encoding sequential inputs into coarse-to-fine spike count embeddings, spiking neurons enable SG-JEPA to adapt to the varying computational constraints of downstream tasks. Extensive experiments demonstrate that SG-JEPA achieves competitive or even superior performance over discriminative baselines on node classification, while effectively scaling to the dynamic graph with 13 million edges. SG-JEPA avoids the complex machinery (negative sampling, graph augmentations, edge-level reconstruction, etc.), resulting in superior training efficiency and memory scalability compared with prior self-supervised dynamic graph baselines.
Huizhe Zhang, Yuchang Zhu, Huazhen Zhong et al.· 0 citations
Parameter-efficient fine-tuning adapts pre-trained language models to downstream tasks with reduced training cost, but widely used methods such as Low-Rank Adaptation (LoRA) still rely on conventional multiply-accumulate computation in the adapter path. This paper proposes Hybrid Spiking LoRA, an asymmetric spiking adapter that replaces the LoRA down-projection with a one-bit spiking encoder while retaining a full-precision up-projection. The design introduces sparse event-driven computation into the low-rank adapter while preserving continuous reconstruction capacity for language model adaptation. We evaluate the method on Korean natural language understanding tasks covering topic classification, relation extraction, natural language inference, and extractive question answering. Across timestep, rank, membrane time constant, and bit-width ablations, the encoder-only spiking design consistently outperforms fully spiking alternatives and remains competitive with standard LoRA. Hybrid Spiking LoRA retains 95.9% of standard LoRA performance on KLUE topic classification at rank 16 and 96.8% in a matched five-seed KorQuAD 2.1 comparison, where Hybrid- $T{=}4$ achieves $79.92~\pm ~0.30$ F1. On relation extraction, it underperforms standard LoRA on average but clearly exceeds fully spiking variants, and single-seed bit-width gains over FP32 are interpreted as seed-dependent preliminary observations rather than a general performance claim. Operation-level analysis indicates substantial theoretical adapter-path energy-saving potential from sparse accumulate operations, although current graphics processing unit simulation introduces latency overhead. These results position asymmetric spiking adapters as a feasible neuromorphic-compatible direction for language model adaptation, while motivating future hardware-level validation.
Jae-Hwan Kim, Dae-Yeol Kim, Chae-Bong Sohn· IEEE Access· 0 citations
Extensive experiments on static and neuromorphic benchmarks show that lower-bit BASC models match or outperform higher-bit baselines and retain this accuracy advantage after structured pruning, while further reducing model storage and synaptic operations.
Linliang Chen, Yan Zhong, Xin Liu et al.· 0 citations
The robustness of deep learning-based semantic communication systems remains a major challenge due to the vulnerability of artificial neural networks (ANNs) to adversarial attacks and the lack of clear robustness mechanisms in such models. Existing approaches often rely on data augmentation, which is training-intensive and requires prior knowledge of the perturbation distribution. In this paper, we instead leverage the structural properties of spiking neural networks (SNNs), a class of energy-efficient, brain-inspired models that compute using binary spikes. We exploit their inherent resilience enabled by the discrete and thresholded nature of spikes and analyze robustness from a structural perspective. For both random and adversarial perturbations, we identify two key factors: 1) membrane potentials should be kept away from the firing threshold to reduce spike flipping; and 2) synaptic weights should be regularized to prevent the amplification of perturbations across layers. Based on these observations, we propose robustness-aware potential (RAP), a simple regularizer designed to exploit the inherent robustness of SNNs. We evaluate our method on both digital and event-based datasets. Experimental results show that SNNs trained with RAP significantly improve robustness and exhibit better resilience to distributional shifts. Moreover, SNNs demonstrate a clear robustness advantage over ANNs, particularly under adversarial attacks.
Yanzhen Liu, Geoffrey Y. Li· IEEE Transactions on Signal...· 0 citations
PTQ4SNN is proposed, a membrane-aware post-training quantization framework that jointly quantizes weights and recurrent membrane states using only a small calibration set and effectively preserves model accuracy under W4 quantization and approximately 4-bit membrane precision.
Hui Xie, Tong Shi, Haotong Qin et al.· 0 citations
A Spiking Neural Network (SNN) is a kind of brain-inspired and event-driven network, which is becoming a promising energy-efficient alternative to Artificial Neural Networks (ANNs). In recent years, SNN methods have been successfully applied in the fields of electromagnetic signal processing and image signal processing, particularly in application scenarios that require low energy consumption. However, the performance of SNNs by direct training is far from satisfactory. In this paper, we study a novel learning method named SAD-SNN (Spatial-Activation Distillation for Spiking Neural Networks), which utilizes the ANN model to guide the SNN model learning. Unlike prior works that rely on element-wise feature alignment approaches, SAD-SNN aligns spatial-activation maps at different resolutions of the teacher and student networks. Specifically, we introduce a direct alignment approach, which defines a spatial-activation loss and normalizes the representation vectors of ANN and SNN, to alleviate the unexpected precision loss. This enables the knowledge of teacher ANNs to be effectively transferred to train student SNNs. On three image classification datasets, our proposed SAD-SNN outperforms other SNN training methods no matter whether homogeneous or heterogeneous teacher ANNs are used. Furthermore, we apply SAD-SNN to the electromagnetic signal detection task, demonstrating strong generalization ability and superior performance. In conclusion, the experimental results on various tasks and SNN architectures demonstrate that our method is a general and effective solution that significantly improves the learning of student SNNs with only two time steps.
Chongxiao Qu, Qian Zhang, Chenxiao Dou et al.· Italian National Conference...· 0 citations