Linux AGX: An Adaptive GPU eXtension to Linux Fair Scheduling for Physical AI and Robotic Systems
Abstract
Linux fair scheduling (EEVDF in kernel ≥ 6.6, CFS in older releases) is the default time-sharing policy on many production robots. ML-driven robotic pipelines, combining perception, planning, and on-device LLM/VLM inference, create workloads in which short CPU preparation steps gate GPU launches and the active task graph reconfigures at runtime as scenes change, humans intervene, or new behaviors are triggered. Because Linux fair scheduling observes only CPU service and sleep-wake events, it cannot identify GPU-gating feeder slices or adapt as the critical path shifts, leading to avoidable GPU idle time and inflated end-to-end latency on single-GPU embedded platforms. We present AGX (Adaptive GPU eXtension), a lightweight, scheduler-agnostic mechanism that preserves Linux fair scheduling's pick logic, virtual-deadline computation, and fairness semantics while making its weight interface GPU-, dependency-, and adaptation-aware. A user-space profiler periodically computes a bounded per-task importance score, which the kernel applies as an effective weight, requiring no new queues, locks, or vendor-specific paths. AGX tracks the evolving critical path online without manual retuning. We prove that AGX preserves proportional fairness under effective weights and bounds competition-induced GPU launch delay, yielding a K-independent bound when the bounded boost is not clipped and bounded coalescing otherwise. AGX instantiates on EEVDF, specializes to CFS, and generalizes to any proportional-share fair scheduler. Experiments on three NVIDIA platforms across fuzz-generated DAGs, realistic robotic pipelines, background CPU-pressure stress tests, and a physical TurtleBot stack show that AGX reduces makespan by 6.0-20.0% and raises average GPU utilization by 7.6-22.1% relative to vanilla EEVDF. Under background CPU pressure, AGX cuts p99 perception-to-control latency by 38.5% and deadline misses by 33.8 percentage points while preserving CPU-only fairness. AGX incurs minimal overhead and remains a drop-in extension that degrades cleanly to vanilla behavior.