Disaggregated Quantization: Specializing LLM Prefill and Decode
Assessment of accuracy under disaggregated serving in vLLM and further validate shared-weight format disaggregation through post-training quantization on models up to 2.8T parameters are evaluated.
Andrei Panferov, Maximilian Kleinegger, Sweta Priyadarshi et al.
· 0 citations