A Collaborative Multi-Compression Acceleration Mechanism for Neural Networks in Keyword Spotting
TL;DR
This study proposes a collaborative multi-compression framework for lightweight deployment that combines adaptive importance-aware structured pruning, mixed-precision quantization, and quantization-aware multi-stage knowledge distillation on LiteKWS-Net.
Abstract
To address the large model size, high computational cost, and limited deployment resources of keyword spotting models on edge platforms, this study proposes a collaborative multi-compression framework for lightweight deployment. Built on LiteKWS-Net, an attention-enhanced 2-D convolutional backbone, the framework combines adaptive importance-aware structured pruning, mixed-precision quantization, and quantization-aware multi-stage knowledge distillation. The retrained teacher reaches 97.90% (mean, 100,813 parameters, 0.385 MiB). MPDQ reaches 95.53 ± 1.16% at 8.27× theoretical weight compression. AIASP reaches 97.59% at a 30% target and 43.9% realized sparsity. The final joint model reaches 96.82% and, under ideal packed sparse mixed-precision storage, has a 51.55× theoretical weight-compression factor relative to the FP32 teacher; sparse-index overhead is excluded. On a Jetson Nano, the TensorRT FP16 network-body benchmark reports 2.86 ms latency and 0.69 mJ per inference.