A Unified Hessian-Guided Framework for Post-Training Quantization
Low-bit quantization enables efficient deployment of large language models by reducing memory footprint and computational cost. Achieving high accuracy at low precision requires selecting quantization ranges, linear transformations, and weight rounding decisions. Existing approaches typically optimize these components...
M. Toftrup, Marco Federici, Boris van Breugel et al.
· 0 citations