Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
Large Language Models (LLMs) demand efficient inference, where the multiply-accumulate (MAC) operations in linear projections dominate compute and energy cost. Post-training quantization (PTQ) reduces this cost by mapping weights and activations to low-bit integers. Power-of-Two (PoT) weight quantization promises effic...
Sunghyun Wee, Geunjae Choi, Hyeonjin Kim et al.
· 0 citations