ATTUNE: Agentic Throughput Tuning for LLM Training
Training a large language model can occupy thousands of GPUs for weeks, and its speed depends on its recipe: how the model is split across GPUs, how memory is traded for compute, and which kernels and communication overlaps it uses. Finding a good recipe is still manual work for experts. Automated tuners search fixed l...
Leo Fan, Hong-Yang Wang, Wen-Bo Chen et al.
· 0 citations