Haida Yu

Quantization Degradation in Large Language Models: A Signal-Noise Perspective

This work systematically study weight-only post-training quantization across bit-widths, quantization methods, model scales and downstream tasks and establishes that quantization degradation is governed by how errors are introduced at the source and how they accumulate across the network.