Dynamic Flow, Static Graph: KV Cache Reuse for Efficient LLM Serving on Mobile NPUs
This work proposes an intra-graph mechanism that maps selective KV recomputation onto static NPU graphs, reconciling algorithmic dynamicity with NPU staticity, and develops an inter-graph scheduler to optimize chunk merging and minimize padding with dynamic programming.
Zheng-Xiang Huang, Sheng-Heng Chen, Chao-Yue Niu et al.
· 0 citations