Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving
This work presents Crossflow, which makes this boundary elastic without changing node roles, and improves token throughput by 16.2-17.4% on geometric mean over static P/D, and by up to 43.4% at high load, while reducing mean TTFT at every evaluated point.
Yi Xu, Ehsan K. Ardestani, Wen-Yin Fu et al.
· 0 citations