EFQ-Softmax: Exp-Free Quantization for Softmax
EFQ-Softmax (Exp-Free Quantization for Softmax), a low-bit probability-generation method that directly maps shifted attention scores to block-scaled E2M1 operands, is proposed and shown to replace the conventional exp-then-quantize path while preserving end-to-end model quality.
Hao-Hui Han, Yu-Ming Wan, Hong-Ning Wang et al.
· 0 citations