Cloud OLAP workloads are bursty and memory-hungry. Per-machine DRAM caps and co-scaled CPU/memory provisioning are fragile. They lead to low utilization, slow autoscaling, and transient OOMs. Prediction-driven schedulers and bin-packing optimizers mitigate but struggle with rigid per-machine memory boundaries and forecasting errors, Existing RDMA or OS-level remote-memory approaches either assume single-tenant trusted clusters (lacking hardware isolation and QoS) or treat remote DRAM as opaque swap. We present Narwhal, an RDMA-co-designed analytical database. It jointly schedules local DRAM and a shared remote memory pool to trade consolidation against bounded remote borrowing. It enforces hardware-level multi-tenant isolation with per-tenant RDMA Protection Domains, quotas, and QoS. It uses a database-aware memory allocator with a three-tier cache and a pressure-threshold invariant to avoid OOMs and protect metadata. Integrated into a production engine, Narwhal eliminates OOMs, boosts memory overselling from 92.67% to
121.89% (+29.22)
and node utilization from 78.16% to
96.63% (+18.47)
, while limiting average remote memory borrowing to just
1.63%.
Jian Zhou, Jia-Chi Zhang, Yang Zhang et al.· Proceedings of the VLDB Endo...· 0 citations
Disaggregated memory architecture has gained wide adoption in cloud and high-performance systems [18, 31, 43] due to its decoupled resource model, elasticity, and low-latency access. In such architectures, transaction mechanisms must ensure atomic and consistent access to remote memory. Prior designs use array-based version structures to improve locality, but their fixed capacity introduces false conflicts, especially for long transactions that require accessing historical versions, limiting concurrency and throughput. We present MiTDM, a transaction system tailored for disaggregated memory environments. MiTDM introduces a hierarchical block version chain that combines the benefits of array and chained structures, supporting dynamic version expansion while maintaining low-latency access. A lightweight Block Index Table accelerates remote reads, and an asynchronous index update mechanism synchronizes indexes across nodes with minimal overhead. To manage version growth, MiTDM employs a hybrid garbage collection strategy aware of transaction states, enabling safe reclamation and reuse of version slots, particularly in hot spots. A one-sided optimistic transaction protocol ensures consistency and atomicity under high concurrency. Experimental results on a disaggregated memory cluster show that MiTDM outperforms state-of-the-art systems. Under the TPC-C workload with 128 clients, it achieves up to 80.1% higher throughput compared to FaRMV2 and 42.7% higher than Motor, while significantly reducing P50 and P99 latencies.
Ao-Xin Wei, Jin-Tian Wu, Jian Zhou et al.· Proceedings of the Internati...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
