Jun 2026· Journal of Chemical Information and Modeling· Vol 66, pp. 7476-7488· 0 citations· 37 references
Medicine
TL;DR
3DTMC-LLM is introduced, the first multimodal LLM designed specifically for TMCs, and highlights the potential of multimodal approaches to accelerate research in TMCs and suggests broader opportunities for advancing the development of the general-purpose chemistry model.
Abstract
Large language models (LLMs) have demonstrated remarkable reasoning capabilities across various natural language tasks. However, comparable breakthroughs in scientific discovery remain limited, as understanding complex chemical and physical phenomena demands multidimensional representations that extend far beyond language alone. Transition metal complexes (TMCs) are a well-recognized paradigm, essential for the development of catalysts and functional materials. The exploration of their vast and intricate design space, characterized by diverse coordination geometries and topological structures, poses significant challenges within language-based representations interpretable by LLMs. To address this limitation, we introduce 3DTMC-LLM, the first multimodal LLM designed specifically for TMCs. 3DTMC-LLM achieves efficient alignment of structural and textual spaces through a pretrained 3D encoder trained on 12 million TMCs, combined with a lightweight single-token projection layer. In downstream tasks, including knowledge/description generation, property prediction, and the more challenging reactivity modeling, 3DTMC-LLM was benchmarked against state-of-the-art closed-source LLMs (e.g., GPT-5.2) as well as domain-specific machine learning models. It achieved competitive or improved performance on several tasks, particularly those with strong three-dimensional dependencies. This framework highlights the potential of multimodal approaches to accelerate research in TMCs and suggests broader opportunities for advancing the development of the general-purpose chemistry model.
We address a fundamental gap in 3D-LLMs: existing models focus on single-object/scene description, struggling with detailed, inter-object comparison. We propose a framework for detailed object-level reasoning across multiple objects with three components: (1) MO3D (Multi-Object in 3D), an instruction dataset requiring fine-grained multi-object comparison; (2) Multi-3DLLM, using a minimal Patch-Interaction Transformer (PIT) that models inter-/intra-object relationships while preserving local geometry; (3) Mini-apps, two application-driven benchmarks (Shape Mating, Change Captioning) that probe geometric understanding for practical use. Recent 3D-LLMs and 2D-VLMs perform poorly on these tasks, lacking both comparison-centric design and geometric awareness. In contrast, Multi-3DLLM trained on our mixture data learns geometric reasoning, surpasses all baselines on MO3D, and provides positive transfer to single-object classification.
K. Ide, Ryousuke Yamada, Yue Qiu et al.· 0 citations
MolLingua, a token-centric, dual-modal framework designed for native molecular understanding, uses a dual-branch Residual Vector Quantization engine to discretize heterogeneous, high-dimensional spatial 2D and 3D features into compact code sequences rather than relying solely on continuous projections.
Haoyang Liu, Xikang Feng, Fei Guo et al.· IEEE journal of biomedical a...· 0 citations
The proposed Chem-R, a general Chemical Reasoning model designed to emulate the deliberative processes of chemists, achieves state-of-the-art performance on comprehensive benchmarks, surpassing leading LLMs, including Gemini-3-Pro and Kimi-k2.5.
Weida Wang, Benteng Chen, Di Zhang et al.· Proceedings of the 32nd ACM...· 0 citations
Large Multimodal Models (LMMs) have achieved remarkable success on images and short videos, yet scaling them to long videos remains challenging due to frame-centric tokenization and limited context windows. 3D geometry provides a natural compression mechanism for visual streams: depth and camera pose enable observations from multiple views and time steps to be fused into a persistent, world-aligned representation. While recent 3D LMMs leverage geometry-aware representations to improve spatial reasoning, they continue to lag behind specialist 3D perception systems on grounding and segmentation tasks. We argue that a key limitation is geometry-aware decoding: existing methods communicate 3D predictions through language tokens, proposal selection, or lightweight grounding queries, creating a bottleneck between language reasoning and dense geometric prediction. Building on these insights, we introduce Qwen-3D, a geometry-aware LMM that compresses visual information within the Qwen backbone using multi-view geometric cues, enabling efficient long-horizon visual reasoning over static scenes. Qwen-3D augments visual tokens with 3D Rotary Positional Embeddings, allowing attention to operate directly in 3D scene space rather than across independent image frames and thereby facilitating scalable cross-view and temporal reasoning. To bridge language and geometry, Qwen-3D incorporates a query-based segmentation decoder that grounds language directly in the underlying 3D scene representation, unifying referential grounding, instance segmentation, and visual question answering across both images and videos. Across a diverse set of benchmarks, Qwen-3D surpasses existing 3D LMMs and outperforms several large proprietary 2D models. Notably, Qwen-3D achieves these improvements while maintaining strong performance on standard 2D vision-language benchmarks by jointly training on 2D and 3D data.
Lucy Lin, Ayush Jain, Yifan Liu et al.· 0 citations
Transformer models have revolutionized natural language processing (NLP), and text-based molecular representations like SMILES have successfully extended these architectures to chemistry. However, domain-adaptive pre-training often causes models to overfit to chemical syntax, catastrophically forgetting their foundational semantic capabilities. To address this challenge, we introduce CheMatE, a chemistry-oriented embedding model that jointly captures molecular structure and domain-specific natural language within the same representation space. Built on a ModernBERT backbone, CheMatE learns bi-semantic representations through a two-stage training procedure: continued masked language modeling (MLM) followed by a Matryoshka contrastive learning stage via Multiple Negative Ranking Loss (MNRL). First, we train the model using MLM on a novel, large-scale corpus of SMILES-annotated, long-context scientific documents that were constructed and curated from FineWeb and ChemPile (comprising 10.4B and 11.5B tokens, respectively). Subsequently, the model undergoes contrastive learning using a synthetic dataset of SMILES-text pairs algorithmically derived from our original training corpus. This design exposes the model to SMILES-enriched scientific literature, enabling bi-semantic understanding. We evaluate CheMatE across a range of downstream tasks covering molecular property prediction and scientific language understanding. Our results demonstrate that coupling our custom-curated datasets with this sequential training strategy yields robust, highly transferable representations. By effectively unifying structural and contextual signals within a single text-based framework, CheMatE achieves competitive performance across both specialized chemistry models and general-purpose language model baselines.
David Ming Segura, Jeremy Goumaz, Joshua W. Sin et al.· 0 citations
Rem3Di is introduced, a representation-learning framework that repurposes latent features from atomistic foundation models as transferable molecular descriptors for property prediction and virtual screening and provides a route from simulation-trained atomistic representations to transferable, chirality-aware molecular representations for chemical machine learning.
Steffen Wedig, Felix Burton, Rokas Elijošius et al.· 0 citations