Bao Tan Duy Huynh

Distribution-aware Low-bitwidth Quantization for Large Language Models

A comprehensive PTQ framework is presented that addresses the problem of compressing LLM weights through three core innovations: a calibration process guided by Kullback-Leibler divergence minimization to preserve the original weight distribution, a learnable codebook optimization mechanism employing noise substitution...