Lightweight Graph Contrastive Distillation for Node Classification

Abstract

Graph contrastive learning (GCL) has emerged as an effective paradigm for learning discriminative graph representations. However, applying GCL to large-scale graphs remains challenging, since conventional GCL frameworks often rely on message-passing GNN encoders and pairwise contrastive objectives, which may introduce considerable computational overhead. To alleviate these challenges, this paper proposes Lightweight Graph Contrastive Distillation (LGCD), a scalable contrastive distillation framework for node classification. LGCD employs a lightweight dual-path student that decouples feature transformation from structural soft-label propagation, avoiding costly message passing over high-dimensional node features while retaining access to structural signals. Based on the attribute- and structure-oriented representations produced by the student, LGCD performs augmentation-free contrastive learning. Dedicated projection heads map heterogeneous teacher and student representations into a shared metric space, supporting both cross-model and cross-branch alignment. To accommodate different graph scales, LGCD employs instance-level alignment on small graphs and global-summary alignment on large graphs, with the latter avoiding explicit all-pair node-to-node similarity construction. Furthermore, LGCD optionally applies Secondary Heat-up Inference (SHI), a teacher-prior-initialized post-training strategy that refines final predictions without updating model parameters. Extensive experiments across diverse graph benchmarks demonstrate the effectiveness and scalability of LGCD, as well as its competitive node-classification performance across different graph scales.