Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
SQD (SubQuadratic Disaggregation), a fine-grained heterogeneous disaggregation scheme that splits decode by quadratic and subquadratic attention rather than by operator type, and that applies across subquadratic attention variants is introduced.
Arya Tschand, Yao-Sheng Fu, Vikram Sharma Mailthody et al.
· 0 citations