Aug 2026· Journal of Computer-Aided Molecular Design· Vol 40· 0 citations· 54 references
Medicine
TL;DR
This work investigates which chemical concepts, functional groups, and structural features are captured by ChemBERTa’s attention mechanisms across layers and heads, and introduces two complementary algorithms: AMPC-Chem-FG and AMPC-Struct, which assesses whether attention patterns reflect predefined chemical concepts and functional groups.
LARK is introduced, a unified framework for learning atom–molecule–reaction knowledge from chemical transformations that learns chemically informative representations of substructures involved in molecular transformations and establishes chemical transformations as a source of structured supervision for representation...
Jian-Bo Qiao, Yu-Hang Liu, Jun-Ru Jin et al.· bioRxiv· 0 citations
This work introduces Wide Encoded Extended Connectivity Fingerprints with Substructure Rotary Graph-distance Encoding (SuRGE), a tokenized hierarchical Morgan representation in which graph-distance-dependent rotations are applied at the input and within transformer self-attention.
Molecular structures are abundant, but measured properties are scarce and scattered across assays that each cover different compounds. Most molecular pretraining pipelines use only the structures, moving directly from self-supervision to endpoint-specific fine-tuning and leaving large, sparsely labelled bioactivity dat...
S. P. Srivastava, R. Gorantla, Sharath Krishna Chundru et al.· bioRxiv· 0 citations
This work introduces an automated and scalable method for interpreting SAE features in ESM-2 by using geometrically inspired features of the protein $\text{C}_\alpha$ backbone, providing a robust method of annotating proteins activated within SAE neurons at a residue level, providing a bridge between mechanistic interp...
S. Setlur, Djordje Mihajlovic, Darrick Lee· 0 citations
SPIRAL is presented, a layer-wise SAE analysis of BiRNA-BERT, a layer-wise SAE analysis of RNA language models where byte-pair tokenization breaks the one-token-one-nucleotide correspondence that nucleotide-level attribution assumes.
M. Hossain, MD. Roqunuzzaman Sojib, Md Toki Tahmid et al.· bioRxiv· 0 citations
This study comprehensively benchmarked these methods for predicting the properties of small organic molecules and showed that TFM consistently outperform XGBoost, CatBoost, multilayer perceptrons, and other descriptor-based methods, even with careful hyperparameter selection for the baselines.
D. Matyushin, A. Sholokhova· Journal of Chemical Informat...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
