This benchmark provides practical guidance for model selection and argues that future models should be judged by biological generalisation, interpretability and perturbation-grounded validity, not by scale or leaderboard performance alone.
Abstract
Single-cell and spatial foundation models promise transferable biological representations, yet their generality remains largely untested across modalities, biological domains and analytical tasks. We benchmarked six representative models, Nicheformer, CellPLM, scGPT-spatial, GenePT, scELMo and Novae, using a harmonised framework spanning scRNA-seq, spatial transcriptomics and Perturb-seq. We evaluated zero-shot and continually pretrained clustering, supervised annotation, marker-gene concordance and perturbation prediction. Model performance was strongly conditional: expression-trained cell-level transformers best resolved many cell-identity tasks, spatial and graph-aware models better preserved tissue architecture, and language-derived gene embeddings were competitive for selected perturbation-response metrics. No model dominated across tasks, and rankings shifted with modality, preprocessing, tokenisation, biological prior, domain shift and metric choice. This benchmark provides practical guidance for model selection and argues that future models should be judged by biological generalisation, interpretability and perturbation-grounded validity, not by scale or leaderboard performance alone.
This paper shows that latent-space predictive pretraining can provide a scalable route to foundation models for spatial transcriptomics. Existing spatial transcriptomics foundation models primarily reconstruct masked gene identities or expression values, potentially encouraging the reproduction of assay-specific technical variation and limiting representation transferability. To avoid directly reconstructing such variation, we shift the prediction target from observed gene measurements to latent cell representations and introduce CellWorld, which predicts the latent representations of masked cells from visible spatial context and a limited partial-expression hint. We pretrain four CellWorld variants, spanning 5.74M to 94.56M trainable parameters, on a corpus of 46 million human cells. Our controlled scaling experiments show that performance improves with model capacity, particularly on spatial tasks, while spatial transfer depends more on sufficient optimization and broad biological source diversity than on cell count alone. Across four held-out datasets, even CellWorld-Small, with 5.74M trainable parameters, outperforms every baseline on all 11 linear-probe benchmarks and all seven fine-tuned spatial benchmarks. Most notably, a frozen CellWorld-Large pretrained on only 5\% of the corpus with broad biological source coverage outperforms every fully fine-tuned baseline across all seven spatial benchmarks. Code is available at https://github.com/UoM-HealthAI/CellWorld.
Hai-Ping Liu, Qian Zhao, Lijing Lin et al.· 0 citations
Prior-enhanced Inference for Spatial Transcriptomic Cell Type Mapping (PRISM), a novel three-stage framework integrating biological prior construction, pseudo-label generation, and multi-level ST refinement, shows strong robustness to domain shift and platform heterogeneity.
Yiheng Xu, Xuehao Wang, Shuqi Liu et al.· Bioinform.· 0 citations
Single-cell transcriptomics has resolved cell-type-specific gene expression in plants, yet maize still lacks an integrated reference and species-specific foundation models. We present scMaize, combining scMaizeAtlas, an integrated atlas of 385,675 cells from 20 projects and 66 samples across seven tissues with hierarchical annotation, with two Transformer-based foundation models pretrained on this atlas. scMaizeExp serves as an expression-only baseline, while scMaizeGO incorporates Gene Ontology (GO) functional embeddings as an inductive bias. Although global expression-prediction accuracy was comparable, the GO prior improved rank-order prediction, strengthened attention toward functionally coherent gene modules, and enhanced embedding topology, with scMaizeGO achieving 86.0% cell-type and 97.1% tissue classification accuracy. Zero-shot evaluation demonstrated the cross-species generalizability of scMaizeGO representations, and few-shot fine-tuning enabled accurate cross-species classification with minimal labeled data. Root perturbation-condition analysis showed that the model encoded treatment-specific cellular states beyond cell-type identity, with the GO prior amplifying perturbation signals approximately threefold. Expression projection identified condition-responsive genes enriched for known stress pathways, and attention analysis revealed predominantly condition-specific changes in gene-gene attention that were weakly associated with expression-projection changes. An online platform (https://www.scmaize.com) provides atlas exploration, model access, and zero-code analysis tools. scMaize establishes a framework demonstrating that species-specific pretraining with functional priors enables transferable, perturbation-aware representations for crop single-cell genomics. HIGHLIGHTS scMaizeAtlas integrates 385,675 cells from 20 maize single-cell projects. scMaizeGO incorporates Gene Ontology priors into maize-specific pretraining. GO priors improve rank-order prediction, attention coherence and embeddings. Few-shot tuning enables cross-species cell-type classification with limited labels. Expression projection reveals stress-responsive genes in root cell states.
Single-cell foundation models have recently emerged as a promising approach for learning general- purpose representations from large-scale transcriptomic data. These models are trained on millions of cells and are designed to transfer their learned representations to a wide range of downstream tasks. However, their practical benefits compared to traditional approaches are still not fully understood. This study evaluates four foundation models, namely scGPT, SCimilarity, UCE, and Transcriptformer, across four downstream tasks: cell type annotation, human data integration, cross-species data integration, and protein expression prediction. Embeddings generated by each model were assessed using multiple public single-cell datasets and compared against conventional machine learning baselines. Performance was measured using task-specific evaluation metrics, including classification, integration, and regression metrics. The results showed that foundation model embeddings did not consistently outperform traditional approaches. In the cell type annotation task, baseline methods achieved the strongest performance across most datasets. For protein expression prediction, however, embeddings from the foundation models generally produced more accurate predictions than the baseline, with SCimilarity achieving the lowest prediction error and Transcriptformer obtaining the highest correlation scores. In the data integration task, all foundation models produced moderate results, while scVI (the baseline) achieved the strongest integration performance. Overall, the results suggest that current single-cell foundation models provide useful representations for some downstream tasks in zero-shot conditions but do not yet offer a universal replacement for task-specific methods. Their effectiveness remains dependent on the application and evaluation setting.
Yasmine Gaballa, Somaia K. Ahmed, T. Abdelaal· bioRxiv· 0 citations
Single-cell foundation models such as scGPT and Geneformer learn rich representations of gene expression programs, but whether these representations encode gene regulatory relationships beyond expression-level confounds remains unclear. Attention patterns in these models have been shown to capture co-expression rather than direct regulation, leaving open the question of whether deeper representations—particularly the residual stream—contain genuine regulatory information. We systematically investigated residual-stream geometry in scGPT and Geneformer across four tissue contexts from the Tabula Sapiens atlas, evaluating whether geometric proximity between gene vectors provides incremental predictive value for curated TRRUST transcription factor–target edges beyond expression confounds. Under repeated stratified cross-validation, geometric features provided significant incremental signal in kidney and immune settings, validated by label-permutation and geometry-shuffle null controls; centered-cosine similarity, PCA projection and multi-layer bundling recovered comparable signal in lung tissues, and the multi-layer bundle improved every domain (kidney ΔAUROC = + 0.122, immune + 0.042, lung + 0.028, external lung + 0.027; geometry-augmented AUROC 0.60–0.69). The effect was fully robust to leave-TF-out and leave-target-out cross-validation and to harder degree- and expression-matched negative edges, but under the stricter leave-both-out split—no transcription factor and no target shared between folds—it collapsed to near-zero (ΔAUROC at most + 0.003, and not statistically significant in kidney or immune), marking the ceiling of out-of-entity generalization. With a comparable per-layer residual-stream extraction applied to both models, the apparent Geneformer advantage mostly disappeared (small residual gaps remained in three of four domains), indicating it largely reflected representation-construction choices rather than a substantial architectural difference. Asymmetric geometric features predicted regulatory edge orientation (AUROC 0.80–0.90), and the geometric signal added incremental value on top of expression-based gene regulatory network (GRN) inference (GENIE3, co-expression). Foundation model residual streams carry incremental, regulatory-relevant geometric signal that is distributed across layers and that complements expression-based GRN inference for retrospective edge prioritization. The signal is statistical enrichment rather than a stand-alone regulatory classifier: absolute performance is modest and out-of-entity generalization is limited, so its practical role is as an orthogonal evidence channel for edge re-ranking and hypothesis prioritization in multi-evidence frameworks.
Spatial transcriptomics maps gene expression at cellular resolution, revealing how cells organize into multicellular niches. Yet computational analyses remain dataset-specific, without a transferable representation of tissue organization that generalizes across datasets, tasks and tissues or predicts how tissues behave under perturbation. We present TERRA, a foundation model pretrained on 112 million human cells profiled by spatial transcriptomics. From a single pretrained backbone, TERRA yields embeddings at the scale of cells, the genes they express and the neighborhoods in which they reside, and supports spatial in silico perturbation, all applied zero-shot to unseen tissues. At the cell level, in newly generated spatial data for developing pancreas, TERRA identified an islet-associated capillary state which we posit represents a developmental precursor of the mature islet microvasculature. At the gene level, in untreated kidney sections, in silico knockout of immune-checkpoint targets predicted a gene program of immune-checkpoint-blockade-associated nephrotoxicity, which we validated in treatment-exposed tissue and recovered in blood. At the neighborhood level, TERRA mapped macrophages across tissues to identify recurring cross-organ niches, which we term archetypes, including a tumor-boundary niche associated with poor prognosis in kidney cancer. Together, TERRA captures the spatial and multicellular logic of human tissue and predicts, in silico, its response to perturbation, providing a multi-scale framework for tissue biology, therapeutic development and clinical application.
Sebastian Birk, M. V. Sanian, Amirhossein Vahidi et al.· bioRxiv· 3 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
