More signal versus more noise: comparing full text and abstract as inputs for large language model-based classification of oncology trial eligibility criteria

Abstract

Abstract Objectives Large language models (LLMs) offer significant potential for automating clinical trial classification by eligibility criteria. However, the optimal input data remain unclear: while abstracts provide a condensed signal, full-text articles contain substantially more information. Whether this additional signal improves performance or whether accompanying noise negatively affects the model’s reasoning capabilities remains unclear. Materials and Methods GPT-5 was applied to classify 200 randomized controlled oncology trials, labelling whether patients with localized and/or metastatic disease were eligible. Each trial was classified twice—using the abstract and full text—and outputs were compared with manually annotated ground-truth labels. Performance was assessed using accuracy, precision, recall, and F1 score, and statistical significance using the McNemar test. Results For identifying trials including patients with localized disease, GPT-5 achieved an accuracy of 86% (95% CI, 81%-91%; F1 = 0.90) using abstracts and 92% (95% CI, 88%-95%; F1 = 0.94) using full texts (P = .027). Performance for detecting trials, which include patients with metastatic disease, was comparably high (99% vs 98% accuracy; F1 = 1.00-0.99). Overall accuracy for assigning combined labels increased from 86% (95% CI, 81%-91%) using abstracts to 92% (95% CI, 88%-95%) using full texts (P = .027). Discussion and Conclusion Providing full-text articles to GPT-5 significantly improved the classification of oncology trials by eligibility criteria in this dataset. Full-text analysis appears particularly valuable for extracting eligibility criteria in oncology that are frequently omitted or not explicitly described within the abstract.