Protein Language Models for Protein Classification: A Comparison with Sequence-Based Features

Abstract

Protein language models learn contextual representations from large sequence collections, but theirpractical advantage over simple sequence-pattern features depends on the task and the evaluationsetting. This exploratory study compares frozen representations from the 8-million-parameterESM-2 model (facebook/esm2_t6_8M_UR50D; mean-pooled, 320 features per protein) with acharacter-level 3-gram TF-IDF baseline (maximum 100,000 features) for seven-class proteinclassification, using the class_label field of the TAPE Remote Homology dataset. ESM-2 featureswere classified with logistic regression and 3-gram features with an SGD-trained logistic classifier.Both pipelines were evaluated on validation, fold-holdout, family-holdout, and superfamilyholdout partitions using accuracy, balanced accuracy, and macro-F1.ESM-2 scored higher than the baseline on all three metrics in all four partitions. On the validationpartition, ESM-2 reached an accuracy of 0.7731 and a macro-F1 of 0.7004, compared with 0.5788and 0.5052 for the baseline; on the superfamily-holdout partition, the accuracies were 0.7799 and0.5534. A supplementary analysis of validation sequences, grouped by maximum qualifyingMMseqs2 identity to the training set, showed higher ESM-2 accuracy in every group, although thesmallest group contained only four sequences. The results are point estimates from completefeature-and-classifier pipelines, reported without confidence intervals or significance tests. Theydo not establish evolutionary independence of the partitions or superiority over alignment-basedmethods. Within these limits, frozen ESM-2 features provided a stronger feature space than theevaluated 3-gram baseline for this dataset and pipeline, and the findings motivate evaluations withadditional baselines, verified label mappings, and homology-aware partitions.