Task-adapted biological foundation models uncover perturbation-centric representations
TL;DR
Fine-tuned scGPT, a transformer pre-trained on over 30 million single-cell transcriptomes, was fine-tuned on more than three million LINCS L1000 perturbation profiles using a supervised objective that predicts perturbation identity and spontaneously captured orthogonal biological relationships never provided during training.
Abstract
Foundation models have emerged as powerful tools for learning transferable representations of biological systems, yet their latent spaces are typically optimized to capture cellular state rather than the effects of perturbations. Here, we demonstrate that a biological foundation model can be repurposed to learn a fundamentally different representation by changing its learning objective. We fine-tuned scGPT, a transformer pre-trained on over 30 million single-cell transcriptomes, on more than three million LINCS L1000 perturbation profiles using a supervised objective that predicts perturbation identity. This transformed the latent space into a perturbation-centric representation that aligned transcriptional responses induced by the same chemical or genetic perturbation across heterogeneous experimental conditions. Fine-tuned embeddings substantially outperformed both gene expression profiles and the original pre-trained model, recovering 85–100% of perturbations within the top 100 nearest neighbors and increasing perturbation classification accuracy from 10–19% to 25–49%. Remarkably, although the model was trained exclusively to recognize perturbation identity, the learned representation spontaneously captured orthogonal biological relationships never provided during training, including chemical similarity (AUROC up to 0.81), mechanisms of action (Hit@10 up to 100%), compound–target relationships (AUROC up to 0.74), and functional relationships between genetic perturbations. The resulting embedding space enabled mechanism-of-action annotation of nearly 12,000 previously uncharacterized compounds, prioritization of target-related chemical–genetic associations, and contextualization of unseen perturbations and external transcriptomic datasets. Together, our results establish objective-driven adaptation as a general strategy for repurposing biological foundation models to learn reusable representations of complex biological phenomena.