Preventing relapse in ulcerative colitis remission maintenance: machine learning prediction and reinforcement-learning herbal prescription optimization with external validation

Abstract

Maintaining remission in ulcerative colitis (UC) with traditional Chinese herbal medicine is challenging because fixed-intensity prescriptions may not accommodate changes in inflammatory burden. This study aimed to predict 52-week relapse and develop a data-driven policy for sequential adjustment of herbal treatment intensity. We analysed a development cohort of 1,181 patients receiving herbal maintenance therapy and an independent external validation cohort of 96 patients. Twenty-one baseline clinical, biomarker, herbal-dose and behavioral variables, expanded to 31 features after one-hot encoding, were used to train and compare seven machine-learning models through stratified five-fold cross-validation and external validation. Discrimination, calibration and decision-curve net benefit were evaluated. Remission maintenance was subsequently modelled as a sequential decision problem using 4,141 scheduled-visit transitions. A gradient-boosted reward model was used for model-based off-policy evaluation of a net-clinical-benefit policy that penalized unnecessary treatment intensification in clinically quiescent patients. Calibrated logistic regression provided the best overall performance, with an internal cross-validation area under the receiver operating characteristic curve (AUROC) of 0.783 (95% confidence interval [CI], 0.755–0.809) and an external AUROC of 0.710 (95% CI, 0.600–0.820). The corresponding Brier scores were 0.180 (95% CI, 0.169–0.191) and 0.215 (95% CI, 0.170–0.263), respectively. Medication adherence was the strongest predictor of relapse, followed by fecal calprotectin, C-reactive protein and Coptidis Rhizoma dose. The learned policy increased herbal intensity with increasing fecal calprotectin and achieved a higher estimated mean reward than observed clinician behavior (0.552 ± 0.018 versus 0.063 ± 0.021). Policy-concordant visits were associated with a higher next-visit remission rate than discordant visits (70.0% versus 59.6%). Combining interpretable relapse prediction with reinforcement learning offers a personalized decision-support framework for optimizing herbal treatment intensity during UC remission maintenance. Because the policy was derived and evaluated using observational data and model-based off-policy methods, prospective clinical evaluation is required before routine implementation.