Reinforcement learning (RL) is increasingly applied to Protein Language Models (PLMs), yet its effectiveness varies across tasks, and standard metrics such as pass@k can rise even when the model's solvable problem set is shrinking. We introduce two capability-level diagnostics. The Expansion-Shrinkage Ratio (ESR) measures how RL shifts the set of problems a PLM can solve, separating genuine gain from probability redistribution. Dual-Reward ESR reports ESR under both the training reward and an orthogonal evaluator; the gap ΔESR quantifies reward hacking as a single observable number. Applied across four protein design tasks, three RL algorithms (DPO, PPO, GRPO), and two PLM architectures, the diagnostics reveal that RL on PLMs is governed by two reward properties: verifiability, whether the reward is a fixed environment or a learned surrogate vulnerable to distribution shift, and coverage, the fraction of sequence space giving an informative gradient. The two axes produce three regimes with distinct ΔESR signatures: well-covered verifiable rewards yield genuine expansion; sparse verifiable rewards induce a coverage bottleneck; predicted rewards induce reward hacking. Controlled analyses isolate these two factors as operative, letting practitioners predict an RL run's outcome before committing to costly fine-tuning.
Hanqun Cao, Hongrui Zhang, Junde Xu et al.· Proceedings of the 32nd ACM...· 0 citations
Enzymatic reactions play an emerging role in a broad spectrum of scientific and industrial applications. The inherent complexity of enzymes, such as their substrate specificity, conformational flexibility, and the vast diversity of reactions involved, poses substantial challenges for the advanced computational prediction of enzymatic reactions with desirable accuracy. Moreover, existing approaches are mostly tailored for a specific sub-task, such as substrate prediction or binding site annotation, which limits their applicability. In this study, we introduce ERAM, a task-agnostic multimodal learning framework capable of addressing a broad range of downstream applications with both accuracy and efficiency. ERAM aligns pre-trained molecular representations from Protein Language Model with the knowledge of enzyme catalysis by modeling enzymatic reactions as multi-relational data. In enzyme retrieval tasks, ERAM achieves an improvement of 28.31% in mean average precision compared with the state-of-the-art (SOTA) method, CREEP. In substrate prediction tasks, ERAM outperforms the SOTA method ESP, achieving average improvements of 35.53% and 22.97% in Matthews correlation coefficient across two datasets. Additionally, ERAM exhibits commendable interpretability by assigning higher attention weights to binding sites, resulting in lower false-positive rates (42.36%) and higher overlap scores (70.59%) in the unsupervised binding site prediction task compared to RXNAA Mapper. By learning embeddings of substrates, enzymes, and products within a unified knowledge graph latent space, ERAM demonstrates its potential as a versatile and effective tool for enzyme catalysis research.