This work proposes bioMoR, which is the first framework to apply MoR to gene-level and pathway-level learning, and identifies three locations for integrating structured biological knowledge within an MoR backbone: graph-based information sharing refines token embeddings, a structural bias guides self-attention toward biologically related tokens, and a graph-aware router uses neighborhood information to determine each token's recursion depth.
Abstract
Transformer models for high-dimensional omics analysis process thousands of genes or pathways, although only a subset requires deep computation. Mixture-of-Recursions (MoR) improves efficiency through adaptive token-choice or expert-choice routing. We propose bioMoR, which, to the best of our knowledge, is the first framework to apply MoR to gene-level and pathway-level learning. Our contributions include identifying three locations for integrating structured biological knowledge within an MoR backbone: graph-based information sharing refines token embeddings, a structural bias guides self-attention toward biologically related tokens, and a graph-aware router uses neighborhood information to determine each token's recursion depth. These techniques are centered on our insight that additional knowledge of token interaction can effectively help models construct embeddings and select which tokens should be learned more deeply. Across eight benchmarks spanning diverse omics data types and evaluated under a unified five-fold cross-validation protocol, bioMoR improves average macro-F1 by 8.2 percentage points and balanced accuracy by 7.1 percentage points over the strongest biology-agnostic MoR baseline while using 75 percent fewer parameters and up to 58 percent fewer FLOPs than a non-recursive Transformer. The selected marker genes or pathways provide biological interpretability, while their token-specific recursion depths reveal how computation is allocated.
Single-cell transcriptomes are sparse observations of coordinated biological programmes, yet most self-supervised models learn by reconstructing individual genes. Here we present BioM-JEPA, a joint-embedding predictive architecture that instead predicts aggregate representations of graph-connected gene blocks defined by protein-association and corpus-derived coexpression evidence. A student network infers each target-block representation from the remaining genes in a cell, while a slowly updated teacher supplies the corresponding target from the full observed gene set. Under the reported extraction procedure, block-level prediction produced embeddings with higher effective rank and weaker association with detected-gene depth in the tested diagnostics than token-prediction, random-block and reconstruction controls. Across CellBench tasks, frozen BioM-JEPA embeddings retained expression, pathway and neighbourhood information and achieved the lowest aggregate perturbation-response error among the evaluated models. Representation diagnostics were also consistent with canonical pancreatic programmes and compositional relationships between genetic perturbations. Linear attention avoids constructing a quadratic gene-by-gene attention matrix; in a matched one-epoch hPancreas experiment at batch size 8, BioM-JEPA provided 5.75-fold higher fine-tuning throughput and 3.76-fold higher held-out embedding throughput than scFoundation. Together, these results support graph-connected gene blocks as useful prediction units for JEPA-style representation learning in single-cell biology.
Yuhao Wang, Zelin Zang, Yuxuan Liu et al.· 0 citations
RNA-Protein Interactions (RPIs) are critical for regulating cellular functions. While traditional wet-lab experiments for RPI detection are costly and time-consuming, Deep Learning (DL) methods provide an efficient computational alternative for RPI Prediction (RPIP). In particular, Graph Neural Networks (GNNs) are promising, as they naturally model RPI networks. However, existing GNN-based methods often rely on homogeneous graphs or predefined meta-paths, which limit their ability to handle data sparsity and to generalize to cold-start scenarios involving unknown molecules. To address these limitations, we propose Edge Generation-guided Relation-aware Learning (EGRL), a novel framework with several key components: implicit meta-path learning to capture relational semantics without handcrafted paths; a multi-relation-aware attention mechanism for adaptive fusion of interaction patterns; a graph generator that predicts potential ("soft") edges to support cold-start nodes; and a multi-feature fusion predictor for final interaction scoring. EGRL is jointly trained with a primary task loss and an auxiliary generator loss. Comprehensive evaluations on four benchmark datasets demonstrate that EGRL achieves competitive overall performance. More importantly, it exhibits superior generalization in cold-start settings, achieving an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.867 and an Area Under the Precision-Recall curve (AUPR) of 0.861 on unknown molecules, corresponding to improvements of 8.6% in AUROC and 5.0% in AUPR over prior state-of-the-art methods. The code will be released soon.
Danyu Li, Ling Zhou, Rubing Huang et al.· 0 citations
A Framework for Learning Over REgulatory-Embedding Networks (FloREN), a supervised and interpretable sample representation method that enables improved sample stratification and biomarker discovery and supports downstream analyses that found specific immune network mechanisms in immune-mediated inflammatory diseases (IMIDs).
Iñigo Clemente‐Larramendi, S. Hillion, D. Cornec et al.· bioRxiv· 0 citations
Gene regulatory networks (GRNs) describe regulatory interactions between transcription factors and their target genes and are essential for understanding cellular processes and disease mechanisms. Recent advances in single-cell RNA sequencing (scRNA-seq) have enabled data-driven GRN inference at single-cell resolution. However, the high sparsity and noise inherent in scRNA-seq data pose substantial challenges for accurately recovering regulatory relationships. Existing graph neural network (GNN)-based approaches often rely on localized message passing, which can lead to over-smoothing and limited modeling of long-range regulatory dependencies. To address these limitations, a structure-aware interleaved-attention graph learning framework, termed IAGRN, is proposed for GRN inference from scRNA-seq data. Specifically, it interleaves topology-constrained local attention with distance-aware global attention, enabling effective integration of structural priors and long-range regulatory signals. Graph Laplacian positional encoding is further incorporated to preserve topological information and enhance node representations. Evaluations on seven public benchmark datasets demonstrate that IAGRN consistently improves GRN reconstruction under highly sparse conditions and achieves competitive performance compared with existing approaches.
Yue Wang, Sicheng Tian, Dan Li· International Journal of Mol...· 0 citations
Inferring gene regulatory networks (GRNs) from single-cell transcriptomic data is crucial for biological discovery, yet existing approaches suffer from a fundamental misalignment with real-world needs. Researchers typically seek a small set of high-confidence regulatory interactions for experimental validation, often involving previously unseen genes. However, current benchmarks rely on transductive splits with global classification metrics, while prevailing models struggle to generalize under inductive settings. To bridge this gap, we reformulate GRN inference as an inductive, ranking-centric graph completion problem and introduce \textbf{\benchmark}, a new benchmark that incorporates an inductive gene-holdout split together with knowledge graph completion metrics to better evaluate top-ranked predictions. Building on this, we propose \textbf{\method}, the first co-evolutionary discrete diffusion framework that jointly models biologically coherent discretized gene expression states and regulatory interactions for robust inductive generalization and improved top-ranked regulatory discovery. We further introduce TF-ALL Subgraph Sampling (TASS) for scalable training. Extensive experiments on {\benchmark} show that {\method} establishes new state-of-the-art performance, significantly outperforming existing methods in novel regulatory discovery, and ablation studies further verify the effectiveness of our design.
Jiaze Song, Runhao Zhao, Minghao Xu et al.· 0 citations
Self-supervised molecular representation learning can improve transfer on label-limited property prediction tasks, but contrastive objectives are sensitive to view construction and are often evaluated using protocols that report discrimination alone. We introduce M-JEPA (Molecular Joint Embedding Predictive Architectures), a predictive self-supervised method for molecular graphs based on connected-subgraph masking and an exponential-moving-average (EMA) teacher, and evaluate it with a compute-matched three-phase protocol that screens objectives on ESOL and tests transfer on Tox21 under Bemis-Murcko scaffold splits. Under matched compute, M-JEPA achieves lower ESOL proxy RMSE than a compute-matched InfoNCE baseline (2.12 vs 3.75; paired ΔRMSE 1.61, 95% CI 1.52-1.72; Wilcoxon p = 2.4 × 10-4) and shorter Phase-1 wall-clock time under the present implementation (24.09 ± 2.53 vs 44.21 ± 4.03 min on a single GPU). On Tox21, hybrid fine-tuning from the M-JEPA checkpoint improves mean ROC-AUC from 0.561 to 0.609 and reduces mean ECE from 0.279 to 0.064 relative to a supervised-from-scratch baseline under the same scaffold split, with paired cross-assay tests supporting systematic rather than assay-specific benefits (Wilcoxon p ≤ 5 × 10-3 for all four metrics). Motif-level Integrated Gradients attributions are stable across fine-tuning in most cases (median Spearman ρ = 0.90 across 62 molecule-assay pairs), although attribution stability and discrimination gains are end point-dependent. Scope. The conclusions in this study are specific to Tox21 scaffold-shift transfer with predictive versus contrastive self-supervision under matched compute; generalization to other MoleculeNet end points is a natural next step and is left for future work.
Karthik Iyer, Nasser R. Sabar· Journal of Chemical Informat...· 1 citation