Utility of lay and clinical narratives for transparent autism diagnosis using BioBERT deep learning
TL;DR
It is demonstrated that lay behavioral descriptions can provide diagnostically valuable information comparable to clinical observations, although they are not readily summarized by AI.
Abstract
Introduction Early autism diagnosis remains challenging due to reliance on clinical observation and limited specialist availability. Addressing these barriers through automated diagnostic labeling and the integration of parental input may help mitigate the problem. Methods We trained a BioBERT machine learning model to label individual autism behavioral descriptions using the seven DSM-5 diagnostic criteria (A1-A3, B1-B4). This approach offers transparent clinical decision-making by providing detailed diagnostic information for individual behaviors and avoiding final case-level black-box decisions. We evaluated the model's performance on labeling lay (N = 35,971) and clinical (N = 145,603) behavior descriptions, as well as its transferability between the two. In addition, we compared the data sources by evaluating the diagnostic utility of lay and clinical examples across four dimensions, and of AI-generated summaries across two dimensions. Results We found that BioBERT can label both types of input, although it achieved higher precision (69%) on clinical descriptions and higher recall (83%) on lay descriptions. Sample size did not explain differences in performance. Transferring models from one data type to another results in a performance drop. Overall, training first on clinical data yielded the best-performing diagnostic models. When evaluating the examples from both data sources, the results show similar scores for the Utility, Specificity, Clinical Relevance, and Impact on Daily Life dimensions, and the cosine similarity analysis revealed substantial overlap (0.42) in vocabulary between the two. The utility of examples for A diagnostic behaviors was generally scored higher than that for B diagnostic behaviors. AI-generated summary scores showed a similar pattern between A and B examples but they were only moderately representative of these examples. Discussion These results demonstrate that lay behavioral descriptions can provide diagnostically valuable information comparable to clinical observations, although they are not readily summarized by AI. The integration of lay information into the diagnostic workflows could accelerate autism diagnosis without compromising clinical utility.