Context parallelism (CP) partitions long sequences across GPUs for LLM training and inference. For variable-length training and full prefill, attention-balanced placement can leave token loads uneven, while ring execution repeats overhead at each computation and communication step. In stage-serialized decode CP (DCP),...
Hong-Yu Chen, Yi-Kang Ruan, Yu-Hai Long et al.· 0 citations
Modern GPU kernels fuse increasingly more work into a single kernel, and intra-kernel tracing has become the mainstream method to profile them. Tracing inserts probes into the kernel to record its runtime states, and the fidelity of the trace determines the efficiency of performance optimization. Unfortunately, existin...
Zhuo-Bin Huang, Kai Zhang, Wei-Hao Cui et al.· 0 citations
This work develops lightweight characterization methods to uncover per-chip compute topology and memory affinity and uses the discovered information to make existing fine-grained scheduling asymmetry-aware, considering not only how many resources are allocated but also which physical resources are assigned.
X. Fan, Jia-Hao Wang, Wei-Hao Cui et al.· arXiv.org· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.