Locality in Today's Content Caching: Why it Matters and How to Model it

CacheFlare: Optimizing Cold Content Performance in CDNs

Existing research on Content Delivery Networks (CDNs) predominantly focuses on optimizing the delivery of hot content—popular items that attract frequent and repeated access from large user bases. However, at Meta, we have identified that cold content, which is less popular and accessed infrequently, poses significant...

Serverless Function Latency Model in Edge Computing

A statistical model of the service latency of serverless functions, with particular reference to edge computing, based on the observation of experimental latency values and adapted to produce a statistical distribution that closely approximates the real one is illustrated.

FairCache: Demystifying Cache-Induced Unfairness in Multi-Tenant Large Language Model Serving

Large language models (LLMs) increasingly rely on context caching to enhance serving efficiency. However, this optimization inadvertently compromises fairness in multi-tenant LLM serving systems. Existing fair schedulers, which account only for compute resources, are unable to handle the multi-dimensional resource dema...

CacheRoute: Planned Prefix-Affinity Routing for Large-Scale LLM Serving

When affinity recovers too little KV work, its residual load skew reduces or erases the improvement, so gating any deployment with a shadow replay rather than enabling affinity from workload statistics alone is recommended.

RDPart: A reuse-based OS-level cache-partitioning policy for fairness optimization in cloud data centers

RDPart is proposed, an OS-level Reuse-Driven LLC Partitioning policy designed to improve fairness while preserving the QoS of cloud workloads, and adopts a black-box design, making it well-suited for public cloud environments where real-time QoS feedback from applications is unavailable.

Warming up Storage-Level Caches with Bonfire