Quantitative analysis of artificial intelligence-based detection of subregions from murine ear skin sections with application to quantifying drug-induced epidermal hyperplasia.

Abstract

The advent of artificial intelligence (AI) technologies is creating a paradigm shift in drug discovery and development. Veterinary pathology is an area that can significantly benefit from AI tools. The availability of AI algorithms in commercial software packages such as HALO and Visiopharm has generated interest in automating pathologists' workflows for the detection and quantification of histology endpoints from whole-slide images. As each software package provides a distinct set of AI models, the relative performance and equivalency of these models for lesion detection and quantification are poorly understood. Here, we present a systematic comparison of the performance of AI algorithms developed in HALO and Visiopharm. Specifically, we trained AI algorithms in HALO and Visiopharm by using the same ground truth training data to detect different subregions and quantify their areas from hematoxylin and eosin (HE)-stained images of murine skin sections. We also calculate the performance metrics (precision, recall, F1 score) for each algorithm using the same test dataset. Our analysis shows that both HALO and Visiopharm algorithms have comparable performance and are resilient to training data size and changes in the color profile of the HE images. As an application, we compared the results of the AI algorithms against pathologists' scores to quantify epidermal hyperplasia. Our analysis shows that the quantitative data from AI algorithms are consistent with pathologists' scores. These results provide a quantitative characterization of commercially available AI models in HALO and Visiopharm and offer practical guidelines for designing and validating AI algorithms using these software packages.