CUDA Graphs eliminate launch overheads by replaying tensor operations over static virtual addresses. However, FP8 pipeline training continuously alters the scaling states, microbatches, and deferred backward tasks that those fixed addresses represent. Split-backward schedules (e.g., 1F1B, Zero-Bubble) decouple input-gr...
SemBridge establishes consumer observation as a semantic layer between placement and collective execution at a boundary between vendor runtimes that cannot share a native communicator.
AccelPact, a parallel runtime enabling zero-I/O in-memory fault recovery for sharded distributed training, and operates directly on native communicator instances, providing an efficient foundation for resilient deep learning.
Gen-Lang Chen, Jun-Yi Zhu· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.