Skip to content

Dissecting How Die Scaling Breaks GPU Fine-grained Scheduling

Sep 2026 · arXiv.org · Vol abs/2609.24270 · 3 citations · 33 references
Computer Science

TL;DR

This work develops lightweight characterization methods to uncover per-chip compute topology and memory affinity and uses the discovered information to make existing fine-grained scheduling asymmetry-aware, considering not only how many resources are allocated but also which physical resources are assigned.

Abstract

Modern GPUs are no longer physically symmetric. Die scaling leads to both manufacturing-driven floorsweeping and cache and memory partitioning. The former creates chip-specific compute topologies, while the latter causes non-uniform memory access. These asymmetries are substantial. Topology-oblivious compute unit allocation can lead to up to 1.33x performance variation, while remote accesses increase HBM latency by up to 67% and nearly double L2 latency. However, these asymmetries are hidden behind the GPU's logical resource abstractions and can vary across chips. We develop lightweight characterization methods to uncover per-chip compute topology and memory affinity. We then use the discovered information to make existing fine-grained scheduling asymmetry-aware, considering not only how many resources are allocated but also which physical resources are assigned. Across full-GPU kernel execution, intra-application multiplexing, and inter-application co-location, asymmetry-aware scheduling improves mainstream kernels by up to 1.22x, multiplexed LLM inference by up to 14.3%, and avoids up to 1.33x performance variation.

View source

Similar papers

Preprint Nov 2026

Uncovering the Memory Hierarchy of Modern NVIDIA GPUs for Accurate Simulation

Emerging artificial intelligence workloads, particularly Large Language Models (LLMs), place unprecedented demands on GPU memory systems. Consequently, architectural innovations increasingly target the memory system to alleviate these bottlenecks. However, existing works still rely on outdated memory hierarchy models t...

Zi-Wen Wang, Jia-Peng Zhou, Chang-Xi Liu et al. · 0 citations
Preprint Nov 2026

Characterizing CUDA Locality Domains on Blackwell GPUs

Low latency is critical to user experience. As NVIDIA Blackwell and Rubin GPUs advance multi-die scaling, data placement and cross-die communication become increasingly important. However, the latency implications and underlying costs of cross-die communication and die-aware execution remain largely unexplored. In this...

Rocky Wang, Joyce Zhou, Ke Jiang et al. · 0 citations
Book Open access Nov 2026

BoxD: Managing GPU Managed Memory

Performance-sensitive GPU applications require deterministic resource provisioning. NVIDIA’s Unified Virtual Memory (UVM) enables GPU memory oversubscription but relies on a device-centric, application-agnostic eviction mechanisms. While oversubscription improves co-location, global device-level evictions induce interf...

Utkarsh Tiwari, P. Kulkarni · 0 citations
Book Open access Nov 2026

GPipeANN: Taming Throughput-Latency Tradeoff in GPU-SSD ANNS

Billion-scale Approximate Nearest Neighbor Search (ANNS) increasingly adopts a GPU-SSD architecture, where GPU fetches and processes fine-grained, SSD-resident data on-demand. However, since GPUs lack native support for I/O scheduling, existing ANNS systems are trapped in a severe throughput-latency tradeoff. They eith...

Pu-Qing Wu, Min-Hui Xie, Yi-Heng Tong et al. · 0 citations
Preprint Nov 2026

SubGPU Pipelining: Intra-GPU Fine-Grained Spatial Parallelism for LLM Inference

Large language model (LLM) decode is dominated by short, predominantly memory-bound kernels that make bursty use of on-chip execution resources and memory bandwidth. This paper introduces SubGPU Pipelining, a fine-grained, intra-GPU spatial execution model that cuts an operator graph into ordered contiguous segments, b...

Muralidhar Andoorveedu, Marius Juston, Ting-Kai Liu et al. · 0 citations
Preprint Oct 2026

Demystifying the Impact of Physical Non-Idealities on Kernel Performance

Physical non-idealities in integrated circuit (IC) design, fabrication, and operation are concealed by conventional hardware abstractions, causing unexpected performance variations in kernels. For large-scale workloads, such as AI training and inference, performance unpredictability can lead to substantial losses. We i...

Supun Kuruppu, Pranav Dangi, Wei Li et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.