Attention-Enhanced Bi-LSTM Ensembles with Frozen ESM-2 Embeddings Achieve Competitive Performance in Protein Subcellular Localization
TL;DR
Attention weight analysis shows the model identifies biologically relevant signals, including C-terminal membrane regions and N-terminal mitochondrial sequences, confirming that ESM-2 embeddings enable effective performance with simplified architectures.
Abstract
Predicting protein subcellular locations computationally is crucial for analyzing large protein datasets. A key issue is that similar sequences in training and test sets artificially inflate accuracy estimates. This study investigates whether Protein Language Model (PLM) features alone can achieve strong predictions using simple classifiers instead of complex architectures. We developed a streamlined deep learning framework combining pre-trained ESM-2 embeddings with an attention-enhanced Bi-LSTM network, deployed as a 3-fold ensemble with soft voting. Training used eukaryotic sequences with ≤40% similarity to ensure rigorous evaluation. The model achieved 86.81% accuracy (MCC = 0.825) on test data—a +22.27% improvement over an SVM baseline (64.54%, MCC = 0.530). On 86 newly released 2024 proteins, the system reached 88.37% accuracy (MCC = 0.827), surpassing DeepLoc 2.1 (80.23%, MCC = 0.714, p=0.007) and MULocDeep (77.91%, MCC = 0.682, p=0.019). However, the small validation set (N=86) and limited representation in categories like Mitochondrion (N=5) require cautious interpretation. The method only handles single-location assignments across four compartments, excluding multi-location proteins. Attention weight analysis shows the model identifies biologically relevant signals, including C-terminal membrane regions and N-terminal mitochondrial sequences, confirming that ESM-2 embeddings enable effective performance with simplified architectures.