BoxD: Managing GPU Managed Memory
Abstract
Performance-sensitive GPU applications require deterministic resource provisioning. NVIDIA’s Unified Virtual Memory (UVM) enables GPU memory oversubscription but relies on a device-centric, application-agnostic eviction mechanisms. While oversubscription improves co-location, global device-level evictions induce interference and non-deterministic memory provisioning. We demonstrate the severity of this effect with different oversubscription scenarios by quantifying the mean and p99 runtime inflation, increase in evictions and page fault amplification. Static compute and memory provisioning techniques do not preserve working-set stability or service-level differentiation. We present BoxD, a cgroup-backed mechanism that integrates GPU-managed memory into the Linux resource control framework. BoxD enforces hard and soft limits on managed memory residency, supports custom per-cgroup eviction policies, and enables deterministic multi-tenant GPU memory provisioning without modifying UVM’s core allocation mechanisms. Our evaluation, with a tiered isolation setup, shows that BoxD reduces the volume of eviction by up to 50 × and increases throughput by 1.5 × in a tier-based isolation setup, and preserves service-level performance targets.