Efficient inference and computational optimization of large language models for intelligent signal processing
TL;DR
The paper outlines an efficient scheme of inference in LLM by synergistically using post-training quantization, key-value cache compression, speculative decoding, and Flash Attention to fill the gap between the state-of-the-art LLM capabilities and the latency constraints of the intelligent signal processing systems.
Abstract
The impressive performance of large language models (LLMs) in various artificial intelligence tasks has not yet been applied to real-time signal processing tasks due to the prohibitive memory and computation requirements. The paper outlines an efficient scheme of inference in LLM by synergistically using post-training quantization, key-value (KV) cache compression, speculative decoding, and Flash Attention. We look at the combined effect that these techniques have on reducing memory footprint by as much as 4x and enhancing decoding by up to 3x without drastically compromising the accuracy of signal interpretation. The results of experimental assessments on spectrum sensing, modulation classification, and channel estimation tasks indicate that our optimized pipeline can reach almost the baseline performance, and it can be implemented on the resource-constrained hardware accelerators, such as GPUs, TPUs, and edge neural processing units. The suggested methodology will fill the gap between the state-of-the-art LLM capabilities and the latency constraints of the intelligent signal processing systems, paving a viable way to LLM-based cognition in the next-generation wireless and IoT networks.