Shiju Wang

PipeSwift: Revisiting Pipeline Parallelism for Large-Scale Completion-Oriented Agentic LLM Serving

It is shown that pipeline parallelism (PP), long overlooked because it offers little decode-latency advantage, can reduce JCT by providing a more favorable balance between prefill and decode efficiency, and PipeSwift is built, an optimized open-source pipeline-parallel runtime integrated with a tailored micro-batch par...