Breaking the Tradeoff: Elastic and Isolated GPU Sharing with GVM

Abstract

GPUs are expensive resources but often underutilized. This inefficiency is exacerbated by the increasing diversity of AI workloads, where small or specialized models often fail to saturate high-capacity devices, necessitating GPU sharing. Existing sharing approaches force a hard tradeoff. They either partition hardware to provide strong isolation but sacrifice elasticity, or employ application-level resource sharing to provide flexibility at the expense of memory isolation or fault containment. We present GVM, an OS-level GPU virtualization layer integrated into the open-source GPU driver. GVM uses three mechanisms to break this hard tradeoff: (1) a GPU container abstraction with cgroup-like APIs for compute and memory control, (2) fine-grained per-container memory accounting and proactive swapping via GPU page tables and extended fault handlers, and (3) privileged hardware-level scheduling and preemption for dynamic compute resource management. Fully compatible with existing applications, GVM improves throughput by up to 2 times and reduces latency by up to 43 times across three representative AI pipelines. GVM is open-sourced at https://github.com/ovg-project/gvm.