Preprint
Sep 2026
PackServe: SLO-Aware Request Scheduling for Agentic LLM Serving at Scale
Request scheduling is a key challenge in large-scale clusters serving agentic large language model (LLM) workloads. An effective scheduler must preserve key-value cache (KVC) reuse across long, shared prefixes, meet token-level latency service-level objectives (SLOs), and minimize GPU resource footprint. Existing sched...
Zhi-Yuan Tan, De-Jiang Zhu, Jing-Zhe Jiang et al.
· 0 citations