One For All: Achieving Adaptive Graph Neural Networks via Mixture of Message Passing

Abstract

The message passing mechanism, which updates node representations by exchanging messages with their neighbors, plays a critical role in graph neural networks (GNNs) for capturing structural patterns. Since the single message passing mechanism lacks the flexibility to handle graphs with differing node feature quality, GNNs with a fixed message passing mechanism, e.g., GCN, GAT, and GraphSAGE, exhibit inconsistent performance across different graphs. To address this issue, we propose Mixture of Message Passing (MoMP). Code is made publicly available at https://github.com/Tinmomo/MoMP, a novel mechanism that incorporates the Mixture of Experts (MoE) paradigm directly into the message passing mechanism of GNNs, treating different message passing mechanisms as ''experts''. Specifically, MoMP employs a gating network to perform edge-level routing, adaptively selecting the most suitable expert to process the message for each edge. To ensure training stability and efficacy, we also introduce a partially-shared expert architecture, reframing the learning task for sparse experts into a residual from a globally-learned general mechanism. Extensive experiments on several homophilic graph benchmarks demonstrate that the proposed method not only outperforms strong baselines but also provides significant interpretability. To the best of our knowledge, our work is the first to fuse the MoE paradigm directly with the GNN message passing mechanism, paving the way for developing adaptive GNNs.