Liang Guo

PMKS: Co-Designing Distributed Networking and Multi-Tier Storage for Ultra-Long Context LLM Inference

The massive memory footprint of the KV Cache is a critical bottleneck in scaling Large Language Model (LLM) inference to ultra-long contexts. Existing single-node or compression-based methods either compromise model fidelity or fail to scale efficiently. This paper introduces a four-tier heterogeneous storage architect...