Resource Efficiency and Performance Predictability in A Groupwise, Hardware-Prioritized Cache on NVMe SSDs

Abstract

Thanks to notable performance and capacity advantages, NVMe SSDs promise an effective cache tier for alleviating the load pressure of back-end storage servers. Compared with DRAM, an NVMe SSD exhibits up to hundreds of times larger capacity but delivers two orders of magnitude smaller bandwidth per gigabyte. This paper reveals that these hardware characteristics challenge fundamental design goals of caches: (1) storing tons of objects in a large-capacity SSD easily induces severe, persistent fragmentation, resulting in low cache space utilization; (2) intensive front-end requests contend for limited SSD bandwidth, causing unpredictable cache lookup latency. To tackle these challenges, this paper presents Gemini. The core of Gemini is heat-informed, group-based object management with a hardware-assisted I/O prioritization mechanism. Specifically, we introduce a tree-based prediction model for agile object grouping and fast reaction to hotspot shift. It provides SSD-friendly, bulk object eviction with a zero-write, remapping-based migration mechanism. In addition, to ensure request performance, we propose a novel I/O model that mitigates bandwidth congestion and prioritizes their processing at the hardware layer. Evaluation results show that Gemini delivers up to 3.8<inline-formula><tex-math notation="LaTeX">${\boldsymbol{\times}}$</tex-math><alternatives><mml:math><mml:mrow><mml:mo mathvariant="bold">×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="cai-ieq1-3711088.gif"/></alternatives></inline-formula> throughput improvement, 2<inline-formula><tex-math notation="LaTeX">${\boldsymbol{\times}}$</tex-math><alternatives><mml:math><mml:mrow><mml:mo mathvariant="bold">×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="cai-ieq2-3711088.gif"/></alternatives></inline-formula> tail latency reduction, and 2.7<inline-formula><tex-math notation="LaTeX">${\boldsymbol{\times}}$</tex-math><alternatives><mml:math><mml:mrow><mml:mo mathvariant="bold">×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="cai-ieq3-3711088.gif"/></alternatives></inline-formula> higher cache space utilization than state-of-the-art caches for both YCSB workloads and production applications.