Precision As You Need: Stochastic Computing Is a Dense Adaptive Quantizer
Matrix multiplications dominate the inference cost of modern transformer-based vision models, yet existing efficiency techniques such as post-training quantization and mixed-precision inference are largely limited to the small set of fixed-width formats (INT4, INT8, BF16, and FP16) supported by conventional accelerator...
Hao-Ran Jin, Kang-Qi Zhang, Ji-Rong Yang et al.
· 0 citations