Interpretable Subgroup Discovery with Abstention in Small, Heterogeneous Clinical Trials: A Retrospective Multi-Dataset Study

Abstract

Small, heterogeneous clinical datasets pose a challenge for whole-cohort prediction because clinically meaningful treatment or response patterns may be diluted across biologically diverse patients. We describe and evaluate NetraAI, an interpretable dynamical-systems framework for selective subgroup discovery that uses finite-iteration contraction-inspired dynamics and long-range memory (LRM) to identify stable, outcome-linked Model-Derived Subgroups (MDS). This system can abstain by assigning No Call when a stable subgroup assignment is not supported. A large language model (LLM) Strategist is outlined only as a possible future extension; it is not evaluated here and contributes nothing to the results reported. Foundation and language models asked to perform subgroup discovery directly did not recover the structure the specialized discovery step recovered. We demonstrate this framework across three retrospective clinical trial datasets: Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) schizophrenia (olanzapine vs. perphenazine comparative treatment-preference benchmark), Canadian Biomarker Integration Network in Depression (CAN-BIND) depression (escitalopram response), and Comprehensive Molecular Characterization of Advanced Pancreatic Ductal Adenocarcinoma for Better Treatment Selection (COMPASS) pancreatic cancer (GnP vs. FOLFIRINOX observational regimen-associated response). The benchmark is not a contest between NetraAI and competing predictors: the same eight downstream methods are evaluated with and without what NetraAI discovered. Given the full feature sets and their own selection procedures, those methods were at or near chance on all three datasets, and blind de novo searches by an independent interaction model and by a pretrained tabular foundation model did not recover an equivalent signature or subpopulation. In internal downstream evaluation, given the discovered variables alone—the same patients, the same classifiers, the full cohort and no abstention of any kind—every one of the eight methods improved on every dataset, 24 of 24 method-dataset comparisons, moving from a raw-feature range of 0.46–0.62 AUC to 0.54–0.78. Restricting further to the subpopulation in which those variables hold improved all eight methods again in CAN-BIND and in COMPASS, 16 of 16 comparisons, reaching 0.66–0.83 and 0.96–1.00, respectively; in CATIE, where the called subgroups are the least outcome-homogeneous of the three, it improved only one of eight. Taking the framework as a whole, 23 of 24 method-dataset combinations improved over the raw-feature baseline. NetraAI abstains on patients without stable subgroup structure, calling 27.7% to 40.4% of each cohort. The contribution demonstrated is therefore subgroup discovery rather than downstream prediction, and its beneficiaries are the conventional methods themselves. In COMPASS, NetraAI identified a three-SNV signature associated with regimen-linked response ranking among called patients; because the cohort was observational and the permutation test was conditional on the selected signature, this finding is exploratory. The two mechanisms are complementary rather than competing: variable discovery establishes which features carry the structure, and abstention establishes in which patients it holds. Neither mechanism replaces conventional modeling. Variable discovery improved every method on every dataset, the pretrained tabular foundation model included; identifying the population in which those variables hold conferred further benefit in two of the three datasets and not in the third. These findings support NetraAI as an exploratory system for generating compact, inspectable subgroup hypotheses that may inform future enrichment strategies after external validation, and indicate that its value lies in what it contributes to other methods rather than in competing with them.