Developing and evaluating automated deep learning and human-in-the-loop vision–language systems for microplastic characterization

Abstract

Microplastic (MP) pollution poses escalating environmental risks, demanding efficient and reproducible tools for morphological characterization of plastic particles. Traditional manual microscopy is labour-intensive, operator-dependent, and poorly suited to large-scale monitoring. This study presents a comparative evaluation of two distinct artificial intelligence paradigms for the analysis of optical microscope images of microplastics. The first paradigm is a domain-specific, multi-task deep learning (DL) classifier based on EfficientNet-B0 with transfer learning, trained on an in-house dataset of approximately 700 annotated microscope images to simultaneously predict microplastic shape/type (five classes), color (10 classes), and surface texture (two classes). The second paradigm employs the Claude Vision API as a zero-shot vision–language model (VLM), augmented with a structured human-in-the-loop (HITL) mechanism allowing domain experts to provide targeted guidance for ambiguous particles. Both systems were evaluated on an identical, independent test set using accuracy, macro-averaged precision, recall, and F1-score. The DL classifier achieved F1-scores of 91.2%, 88.5%, and 85.1% for shape/type, color, and texture classification, respectively. In contrast, the VLM achieved raw F1-scores ranging between 72 and 81% across the evaluated tasks, which improved substantially to approximately 84–89% following expert-guided refinement. These results demonstrate that the trained DL model excels in high-throughput, reproducible screening, while the VLM-HITL system offers enhanced interpretability and flexibility for ambiguous cases. This comparative framework, deployed as a freely accessible web application via Hugging Face Spaces, provides practical insights into the deployment trade-offs between domain-specific and generalist AI approaches for environmental microplastic analysis.