DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization
DAMP uses both quantization-error energy and decay-based persistence to identify high-risk channels during offline calibration and stores these channels at higher precision and the remainder in INT8, the first to study post-training quantization of recurrent states in GDN and KDA based language models.
Tao Zhang, Jian-Chao Tan, Pingwei Sun et al.
· 0 citations