Jul 2026
SemPIC: Learning Semantic Position-Independent KV Caches
This work presents SemPIC, which trains a LoRA-enabled Writer to compile native per-layer document KVs through behavioral distillation while retaining the pretrained decoder as an unchanged Reader, and introduces KV Gradient Checkpointing, which reduces peak training memory without severing gradients through cached KVs.
Hui Xie, Peng Xiao, Yutong Deng et al.
· arXiv.org · 0 citations