Deep learning in materials electron microscopy: models, applications and emerging trends

Abstract

The intersection of deep learning (DL) and materials electron microscopy (EM) has significantly expanded since the early 2010 s; however, the diversity of model architectures and use cases makes it hard to identify literature gaps and determine which models are well-established in this field of application. This work provides a model-centric overview of DL applied to EM, with special focus on transmission EM, organized around three functional branches: representation (convolutional neural networks, residual networks, U-Net variants, and transformer-based architectures), analysis (object detectors such as Mask R-CNN and YOLO), and generation (variational autoencoders, generative adversarial networks, and denoising diffusion probabilistic models). The evolution of these model families is traced from AlexNet (2012) through ResNets and U-Nets (2015), vision transformers (2020–2021), and current hybrid architectures, with a characteristic lag of three to seven years between model conception and first application in materials science. A bibliographic analysis of more than 1 800 peer-reviewed articles, collected from the Web of Science and classified using an LLM-augmented regex methodology, provides co-occurrence matrices and publication trends that offer a quantitative context for these developments. The results indicate that U-Net and generic CNN variants dominate the current literature, while transformer-based and generative models remain underrepresented. The article also compiles a broad collection of DL applications in materials microscopy, examined through the lens of materials scientists, and discusses the specific challenges and perspectives that emerge, including standardized benchmark datasets, foundation models, and multimodal data fusion.