Where Do Crystal Graph Networks Fail? A Space-Group-Resolved Error Analysis of Band Gap Prediction with a Simple Edge-Aware GNN: Crystal-X

Abstract

Graph neural networks (GNNs) for crystal property prediction are typically evaluated by a single aggregate error, which can mask where, and for which classes of materials, these models fail. In this study, we present a space-group-, centering-type-, and band gap-resolved error analysis of GNN band gap prediction on the Materials Project dataset. As a channel for this analysis we use Crystal-X, a deliberately simple model: a standard graph convolutional backbone with two minor architectural modifications, an asymmetric edge convolution and a neighbor-feature transformation, that supplement bond information often treated as secondary in node-centric models. Crystal-X is not a state-of-the-art model: it reaches a band gap MAE of 0.256 eV on the MP 2018.6 dataset, behind ALIGNN (0.22 eV) and PotNet (0.20 eV), though ahead of older baselines such as CGCNN (0.39 eV), SchNet (0.415 eV), and MEGNet (0.33 eV) while using only the nine-property CGCNN atomic feature set. Its value here is as a controlled, low-complexity testbed for the error analysis. That analysis reveals systematic patterns that aggregate MAE conceals: errors concentrate in underrepresented band gap ranges and in low-symmetry and non-centrosymmetric space groups; per-group errors for sparsely populated space groups are dominated by sampling noise; and modest, as-yet-unverified gains from edge-aware convolutions appear in monoclinic and non-primitive-centered systems. We argue that this kind of granular, symmetry-resolved evaluation should accompany aggregate benchmarks when assessing crystal GNNs.