Xi-Shuo Li

Co-Optimizing Request Scheduling and KV Caching for Edge LLM Serving

Deploying large language models (LLMs) on edge nodes enables low-latency and privacy-preserving inference, but faces severe resource constraints under high-concurrence workloads. While existing inference systems leverage intranode key–value (KV) caching to improve efficiency, they largely neglect the unique complexitie...