GEM-Distill: Graph Ensemble Multi-Level Distillation for MLP-Based Graph Classification

Abstract

Graph Neural Networks (GNNs) achieve strong performance on graph learning tasks, but their message-passing computation hinders deployment in resource-constrained settings. GNN-to-MLP (G2M) knowledge distillation improves inference efficiency by transferring GNN knowledge to lightweight MLP students; however, existing methods mainly target node classification and generalize less effectively to graph classification, where supervision is sparse, teacher global semantics are difficult to transfer, and MLP students have limited structural expressiveness. This paper presents GEM-Distill, a graph ensemble multi-level distillation framework for MLP-based graph classification. GEM-Distill assigns graph-, subgraph-, and node-level objectives to granularity-specific MLP experts and coordinates them through a sequential coarse-to-fine training strategy, mitigating cross-granularity optimization conflicts while preserving complementary structural supervision. It further introduces Virtual-Neighbor Guided Graph Distillation (VNGD), a training-only graph-level distillation module that enriches global supervision by mixing semantically related training graphs. Experiments on standard and large-scale graph classification benchmarks show that GEM-Distill achieves competitive or stronger performance than existing G2M baselines while retaining efficient MLP-style inference. Additional analyses on robustness, parameter-aligned comparisons, efficiency, and teacher architectures further support the effectiveness and practicality of the framework.