Meta deploys a heterogeneous fleet comprising NVIDIA GPUs, AMD GPUs, and proprietary MTIA accelerators to reduce costs and mitigate vendor lock-in. However, the extensive history of NVIDIA-specific optimizations hinders performance portability to other hardware. This paper presents our methods for optimizing alternativ...
Hao-Yu Zhang, Arshad Rahman, Adnan Aziz et al.· 0 citations
This work presents Crossflow, which makes this boundary elastic without changing node roles, and improves token throughput by 16.2-17.4% on geometric mean over static P/D, and by up to 43.4% at high load, while reducing mean TTFT at every evaluated point.
Yi Xu, Ehsan K. Ardestani, Wen-Yin Fu et al.· 0 citations
This work presents the first production-scale application of Triton on a custom ML accelerator, MTIA-2i, developed by Meta, and develops a new compiler backend that targets it, introduces enhancements to TorchInductor code generation, and proposes minimal language extensions that expose MTIA-specific architectural feat...
Haishan Zhu, Domi Yan, Michael Levesque-Dion et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.