Tianjun Shao

AdaK: adaptive KV cache budget estimation framework for analyzing long-context large language model inference

AdaK, an adaptive KV cache budget estimation framework with three strategies: entropy-based thresholding, task-aware lookup table, and a lightweight policy network, which enables safe budget estimation as a dynamic ceiling for downstream sparse attention kernels.