Wen-Jie Fan

Q-First: Most of Attention Needs Only the Query in Disaggregated LLM Decoding

Disaggregated LLM serving puts the KV-cache sweep on memory-optimised hardware and the projections and feed-forward on compute-optimised hardware, then inherits from the decoder block a dependency neither device wants: attention runs first and the feed-forward consumes its output, so within one sequence each side idles...