Zi-Nan Xiong

GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving

Long-output reasoning has made the key--value (KV) cache a critical memory bottleneck for efficient LLM serving. Existing KV compression methods usually rely on a predefined per-request budget and adjust only which KV states are retained, leaving the total capacity fixed throughout decoding. However, reasoning workload...