HAServe: Horizon-Aware Scheduling and Memory Co-Design for Efficient LLM Serving
HAServe is presented, which turns HorizonSchema, a workload abstraction that connects exact prefix paths, permitted serializations, and released consumers as shared inputs to both decisions, into a scheduling--memory co-design method of systematic optimization for online LLM serving.
Xing-Yu Qu, Tian-Hao Lin, Yiqi Li et al.
· 1 citation