Ke-Qiang Wu

DualPace: Decoupled Scheduling and Cooperative GPU--CPU Decoding for LLM Serving

Large language model (LLM) serving relies on batching to improve inference throughput, but the key-value (KV) cache grows with sequence length and limits GPU-resident concurrency. CPU offloading can expand KV capacity using host memory, yet existing systems remain largely GPU-gated: CPU-resident requests make progress...