Which chemical features are captured by ChemBERTa’s attention?
This work investigates which chemical concepts, functional groups, and structural features are captured by ChemBERTa’s attention mechanisms across layers and heads, and introduces two complementary algorithms: AMPC-Chem-FG and AMPC-Struct, which assesses whether attention patterns reflect predefined chemical concepts a...