ProteinJEPA: Latent prediction improves protein language model pretraining
Protein language models are trained primarily with masked language modeling (MLM), which predicts masked amino-acid identities, but joint-embedding predictive architectures (JEPA) instead predict latent representations, but have not been applied to proteins.
Dan Ofer, Dafna Shahaf, M. Linial
· 0 citations