This study offers a practical recipe for node-level JEPA-style latent prediction on graphs, and clarifies when structural conditioning helps representation learning.
Abstract
Self-supervised learning on graphs is largely shaped by contrastive methods that depend on carefully designed augmentations, and by generative methods that reconstruct node attributes in the input space. Both paradigms can entangle representations with low-level input statistics rather than with relational structure. Joint-embedding predictive architectures (JEPA) instead learn by predicting latent targets rather than reconstructing inputs. Recent work has explored this idea for graph-level representation learning, but how to design JEPA-style objectives for node-level tasks, and which structural signals the predictor should condition on, remains less clear. We present NodeJEPA, a joint-embedding predictive architecture for node-level graph self-supervised learning. NodeJEPA masks structure-aware k-hop ego-subgraphs and trains a context encoder to predict the latent representations of the masked nodes. These targets come from an EMA-updated target encoder with stop-gradient. A structure-conditioned predictor integrates spectral and centrality descriptors through cross-attention. Variance, covariance, and Laplacian spectral regularizers help stabilize the embedding geometry, and an optional curriculum gradually increases masking difficulty during training. Because prediction occurs in latent space, NodeJEPA does not rely on input reconstruction or hand-crafted graph augmentations. We evaluate NodeJEPA on standard node classification benchmarks under linear probing and fine-tuning protocols, and conduct ablations on masking, prediction, and regularization design choices. Our study offers a practical recipe for node-level JEPA-style latent prediction on graphs, and clarifies when structural conditioning helps representation learning. Code, configurations, and evaluation scripts are publicly available at https://github.com/OliverZ-dot/Node-Jepa.
D2GSL constructs a semantic similarity channel and a spectral feature channel to model node relationships from both local semantic and global spectral views and introduces a hyperadjacency matrix that explicitly models inter-layer node correspondences and enables joint structural reconstruction across channels.
Jun-Chen Zhang, Xuhao Wei, Xiaolei Gu et al.· Computer Modeling in Enginee...· 0 citations
Graph self-supervised learning aims to learn transferable representations from large-scale unlabeled graph data. Joint-embedding predictive architectures (JEPAs) avoid explicit negative-pair construction and raw-input reconstruction by predicting masked targets directly in latent space. However, existing graph JEPAs typically rely on a single predefined graph partition, biasing the learned representations toward one structural granularity and limiting their ability to capture complementary patterns at different graph scales. To address this limitation, we propose HP-JEPA, a hierarchical partitioning framework for multi-resolution graph joint-embedding prediction. HP-JEPA organizes each graph into an ordered bank of coarse-to-fine partition resolutions and performs context-target latent prediction separately at each resolution using an online encoder, an exponential-moving-average target encoder, and a latent predictor. The resulting resolution-specific graph representations are subsequently integrated through concatenation or task-specific resolution weighting, allowing downstream models to combine complementary local, regional, and global structural information. Experiments on seven graph classification benchmarks and one graph regression benchmark show that HP-JEPA outperforms the fixed-resolution Graph-JEPA baseline on 6 of 8 tasks, improving upon Graph-JEPA on most evaluated benchmarks. Size-stratified analyses further show that HP-JEPA achieves higher accuracy than Graph-JEPA in most evaluated graph-size quartiles on three representative datasets. These results highlight the effectiveness of hierarchical multi-resolution partitioning for transferable graph representation learning.
Ruichen Xu, Jingxiang Qu, Wenhan Gao et al.· 0 citations
This framework introduces a dual-encoder architecture that separately processes structural and attribute information, incorporates node positional encoding to approximate Neighborhood Identity Distribution (NID), and employs dual reconstruction tasks for both edges and node attributes.
High-dimensional unlabeled data often contain complex latent structures that are easily obscured by redundant features, noise, and unreliable neighborhood relationships. Although graph-based learning provides an effective means of preserving sample relationships, most existing methods mainly rely on first-order neighborhoods and therefore fail to fully exploit multi-order dependencies revealed by multi-hop propagation. To address this limitation, we formulate unsupervised representation learning as a graph-guided structure-preserving projection problem and propose Unsupervised Representation Learning with Adaptive Multi-order Structural Graph Fusion (URL-AMGF). The proposed method constructs multi-order graphs to characterize structural relationships at different neighborhood orders and adaptively fuses them into a unified guidance graph. This graph is then integrated into projection matrix learning, enabling graph structure optimization and low-dimensional representation learning to be jointly performed within a unified framework. By integrating local neighborhood information with multi-order structural cues, URL-AMGF learns low-dimensional representations that better reflect the structural relationships among samples. Experiments on multiple benchmark datasets show that URL-AMGF achieves generally competitive clustering performance compared with representative unsupervised dimensionality reduction and graph-based learning methods. These results indicate that adaptive multi-order graph fusion can provide effective structural guidance for structure-preserving unsupervised representation learning.
Can-Yu Zhang, Yun-Jing Zhang, Jia-Wen Sun et al.· Applied Sciences· 0 citations
A multimodal model that combines two complementary ideas: a self-supervised method that enables a GNN encoder pretrained on one dataset to operate directly on another dataset with a different node-feature dimensionality, without rebuilding the model or realigning the data is investigated.
Fumiaki Kimino, Ryoma Sato Sokendai, National Institute of Informatics· 0 citations
Graph representation learning has largely focused on designing increasingly sophisticated models to transform graph topology into vector representations, or embeddings. However, the extent to which embedding quality depends on model learning, rather than on the underlying topological transformations, remains unclear. Here, we show that informative embeddings can be derived without complicated model design and gradient-based training. Propagating random features through implicit hierarchical structures induced by random walks and anonymous walks yields embeddings that capture node proximity and structural role, respectively. These two training-free embeddings preserve complementary aspects of graph organization and perform competitively with classic and recent methods across various node-, edge-, and graph-level tasks. They often require substantially less computation, resulting in a favorable quality-efficiency trade-off. Combining the two types of embeddings further improves inference quality of some tasks compared with using either embedding type alone. Our results suggest that informative graph embeddings can arise from carefully chosen topological transformations before any learning operation is applied.
Meng Qin, Jin-Qiang Cui, Hongwei Zheng et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
