Aug 2026· IEEE journal of biomedical and health informatics· Vol PP· 0 citations
Medicine
TL;DR
MolLingua, a token-centric, dual-modal framework designed for native molecular understanding, uses a dual-branch Residual Vector Quantization engine to discretize heterogeneous, high-dimensional spatial 2D and 3D features into compact code sequences rather than relying solely on continuous projections.
Abstract
Large Language Models (LLMs) have emerged as a powerful paradigm for scientific discovery, yet adapting them to natively comprehend complex molecular structures remains a fundamental challenge. To capture structural nuances, the community has increasingly shifted towards multimodal architectures. Existing molecular LLMs typically integrate 2D and 3D modalities via continuous projection, mapping pretrained molecular embeddings directly into the backbone LLM. However, mapping continuous embeddings from pretrained encoders into the discrete token space of LLMs creates an inherent modality gap, hindering effective semantic transfer and limiting their performance. To address these challenges, we introduce MolLingua, a token-centric, dual-modal framework designed for native molecular understanding. Specifically, we leverage a dual-branch Residual Vector Quantization (RVQ) engine to discretize these heterogeneous, high-dimensional spatial 2D and 3D features into compact code sequences rather than relying solely on continuous projections. By integrating them as learnable tokens within the LLM vocabulary, MolLingua translates structural chemistry into the native discrete language of LLMs. Extensive experiments demonstrate that our fully discrete approach effectively aligns spatial knowledge with language models, achieving state-of-the-art performance in complex chemical reasoning and competitive results in generative tasks. Our work contributes an effective, unified framework for propelling essential applications of LLMs in biomedicine.
UniLang is introduced, a unified generative framework that bridges the divide between language modeling and structured prediction by extending pretrained LLMs to treat machine-native symbols as first-class generative units alongside natural-language tokens.
Transformer models have revolutionized natural language processing (NLP), and text-based molecular representations like SMILES have successfully extended these architectures to chemistry. However, domain-adaptive pre-training often causes models to overfit to chemical syntax, catastrophically forgetting their foundational semantic capabilities. To address this challenge, we introduce CheMatE, a chemistry-oriented embedding model that jointly captures molecular structure and domain-specific natural language within the same representation space. Built on a ModernBERT backbone, CheMatE learns bi-semantic representations through a two-stage training procedure: continued masked language modeling (MLM) followed by a Matryoshka contrastive learning stage via Multiple Negative Ranking Loss (MNRL). First, we train the model using MLM on a novel, large-scale corpus of SMILES-annotated, long-context scientific documents that were constructed and curated from FineWeb and ChemPile (comprising 10.4B and 11.5B tokens, respectively). Subsequently, the model undergoes contrastive learning using a synthetic dataset of SMILES-text pairs algorithmically derived from our original training corpus. This design exposes the model to SMILES-enriched scientific literature, enabling bi-semantic understanding. We evaluate CheMatE across a range of downstream tasks covering molecular property prediction and scientific language understanding. Our results demonstrate that coupling our custom-curated datasets with this sequential training strategy yields robust, highly transferable representations. By effectively unifying structural and contextual signals within a single text-based framework, CheMatE achieves competitive performance across both specialized chemistry models and general-purpose language model baselines.
David Ming Segura, Jeremy Goumaz, Joshua W. Sin et al.· 0 citations
The results suggest that competitive MLLM can emerge from alignment alone, reducing multimodal extension to a lightweight projector-training problem that generalizes across modalities and adapts rapidly to each new LLM release.
Xuanru Zhou, Yiwen Shao, Jiahong Li et al.· 1 citation
3DTMC-LLM is introduced, the first multimodal LLM designed specifically for TMCs, and highlights the potential of multimodal approaches to accelerate research in TMCs and suggests broader opportunities for advancing the development of the general-purpose chemistry model.
Jingyuan Zhu, Farshad Shiri, Liren Xiao et al.· Journal of Chemical Informat...· 0 citations
This empirical research establishes the essential groundwork for predictably scaling multimodal foundation models by modeling the influence of data composition on compute laws and allocation exponents and derive an efficiency frontier specifying precise configurations of model size, token count, and data mixture.
Results demonstrate that MoLGE consistently outperforms dense multilingual baselines with a minimal increase in trainable parameters, and suggest that structured language specialization provides an effective pathway for massively scaling language coverage of multilingual ASR.
Sangmin Lee, Woo-Jin Chung, Woongjib Choi et al.· 0 citations