EPTQ: Fast and Accurate 2-bit Post-Training Quantization via Factored E8 Lattice
EPTQ (E8) achieves accuracy competitive with or surpassing QTIP on Llama-2 models, while EPTQ (FE8) surpasses competing VQ methods in decode throughput on most models, achieving up to 38% faster inference than FP16, with both variants reducing quantization time by over 80% compared to competing approaches.