Adaptive Topology-Aware Algorithm Selection for MPI Collective Communications

Abstract

The Message Passing Interface (MPI) is a de-facto standard programming model for developing high-performance applications on large-scale supercomputers, where the performance of its collective communication operations over the underlying interconnection network is critical to the overall scalability of the applications. While the MPI defines the semantics of these operations, it leaves the algorithmic implementation to Open MPI, and selecting the algorithm typically relies on hard-coded decision logic. However, in real-world scenarios, this static approach often leads to sub-optimal performance, as it ignores the specific properties of the underlying system, particularly the complex and dynamic interconnection topologies. In this work, we address the challenge of selecting an efficient algorithm for a given collective, taking into account interconnection topologies. First, we demonstrate through a simulation study that interconnect topological features significantly influence the choice of the optimal collective algorithm for specific communication instances. Second, we propose a novel, machine learning-based, topology-aware framework for MPI collective algorithm selection. Third, we integrate this predictive framework directly into Open MPI, and show that our method is effective in selecting efficient collective algorithms for the Next Generation Tianhe Supercomputer, thereby unlocking the performance potential of the underlying interconnect.