Negar Alihadi

From Attention Sensitivity to Layer Role: Revisiting Mixed-Precision Quantization of Transformers

Most post-training quantization pipelines fit each weight matrix to its pretrained counterpart, one matrix at a time. Whether that proxy tracks what an attention block actually computes, or how errors in the Q, K and V projections compound inside the softmax, is rarely checked. We write the objective on the attention o...