Aug 2026· IEEE International Conference on Embedded and Real-Time Computing Systems and Applications· pp. 230-231· 0 citations· 6 references
Abstract
Log-structured merge-tree (LSM-tree) key-value stores rely on caching to mitigate multi-component lookups and long tails, yet block and KP caches are prone to compactioninduced expiry, and all three cache types suffer from scan pollution under LRU eviction. We present AutoThermKV, a two-level in-memory architecture that decouples fast admission from hot-item protection. The front tier (HotFilter) uses a 64 Baligned bitmap/key/value page layout with Quick Space Reservation for near-constant admission cost; the back tier (HotCommitted) retains proven hot items via a lightweight LRU and a secondtouch promotion rule. Implemented atop RocksDB and evaluated on YCSB-like workloads with controllable temporal locality, AutoThermKV achieves up to 2.58 × throughput speedup, 2.60 × average-latency reduction, and tail-latency improvements of up to 20 × (p99) and 36 (p999) under modest memory budgets.
Modern serverless platforms create function instances based on user-specified memory limits, which often result in memory over-provisioning even when the average runtime footprint is small. This over-provisioning reduces warm container density under a fixed memory budget, leading to early pool saturation, frequent evic...
Dongjae Lee, Kyuli Park, Yeonwoo Jeong et al.· IEEE International Conferenc...· 0 citations
Results show that object-value signals rank what to retain, while persistent responsibility determines which group bears reclamation pressure, which shows that object-value signals rank what to retain, while persistent responsibility determines which group bears reclamation pressure.
The BASIC Prefetcher is proposed, which recovers application-level I/O patterns by classifying requests into size-based bins and modeling temporal bin transitions with a lightweight Markov-chain framework and achieves up to 89% prediction accuracy on sequential workloads and 70% on highly random workloads, including AI...
Han Jang, Dongjun Lee, Youngbin Jin et al.· Proceedings of the 19th ACM...· 0 citations
Large language model serving faces a critical memory bottleneck: the KV cache grows with sequence length and batch size. PagedAttention uses fixed-size memory blocks to reduce allocator-level fragmentation, but recent KV eviction algorithms operate at a token granularity finer than block-level management. This mismatch...
Yuanhang Gao, Xiangrui Yang, Yuan-Feng Chen et al.· 1 citation
KV-cache offload is widely used to stretch GPU memory for LLM serving, but its storage behavior has not been characterized at the block-device level. In this paper, we study LMCache through realworld multi-session workloads that span same/different context $\times$ same/different prompt, using over 100 stateless reques...
Ying He, Dingsen Shi, Yanbo Dai et al.· 2026 International Conferenc...· 0 citations
Minima-KV, a retention-preserving hierarchy for mixed-format paged attention, is presented, a practical mixed-format path for compressing long-context state without evicting live-request KV pages.
S. Kozyrev, AI DavydMaiborodaMinima, Inc.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
