A framework for assessing algorithmic discrimination risks in training data: a case of pediatric type 1 diabetes

Abstract

Abstract Objectives To develop a generalizable framework for identifying algorithmic discrimination risks arising from subgroup imbalances in machine learning training data, with relevance to medical informatics applications where heterogeneous real-world data can bias model behavior. Materials and Methods We introduce a discrimination risk assessment framework for training datasets, a 4-step methodology integrating: (1) controlled representation sampling, (2) ensemble-based model training, (3) multilevel subgroup disparity quantification, and (4) mitigation-oriented interpretation. The framework systematically perturbs training subgroup composition while holding evaluation sets fixed to isolate representation effects. Validation was performed across 7 publicly available pediatric type 1 diabetes datasets using sociodemographic variables and continuous glucose monitoring data to assess robustness under heterogeneous data sources. Results Representation balance alone does not guarantee stable or equitable model outputs. Ensemble analyses revealed subgroup-dependent volatility, with some groups consistently contributing to model generalization, while others inducing instability despite increased representation. These findings demonstrate structural sensitivity to data composition that is not captured by standard performance-only evaluations. Discussion The framework exposes disparities and instability patterns that remain hidden under single-model or balanced-data evaluations. By quantifying representation-driven behaviors and subgroup-specific training value, it offers a diagnostic tool for understanding data-induced risks prior to model deployment. Conclusion This work provides a methodology for assessing discrimination risks in training datasets used in medical informatics. By integrating data composition effects, prediction stability, and subgroup-level disparity analysis, the framework supports more reliable and transparent development of machine learning systems across diverse clinical and nonclinical contexts. Clinical trial registration This research did not involve any new clinical trial. All analyses were performed on data from previously registered clinical trials, as listed below: PEDAP dataset: The source of the data is the PEDAP Trial Study Group. (2024). PEDAP Public Dataset (Release 4). Retrieved from https://public.jaeb.org/dataset/599. The analyses content and conclusions presented herein are solely the responsibility of the authors and have not been reviewed or approved by PEDAP Trial Study Group. ClinicalTrials.gov Identifier: NCT04796779; registered March 15, 2021. IOBP2 RCT dataset: The source of the data is the Insulin Only Bionic Pancreas Pivotal Trial. Retrieved from https://public.jaeb.org/dataset/579. The analyses content and conclusions presented herein are solely the responsibility of the authors and have not been reviewed or approved by the Bionic Pancreas Research Group or Beta Bionics. ClinicalTrials.gov Identifier: NCT04200313; registered December 16, 2019. DCLP5 dataset: The source of the data is the University of Virginia. DCLP5 Dataset. Retrieved from http://public.jaeb.org/dataset/535. The analyses content and conclusions presented herein are solely the responsibility of the authors and have not been reviewed or approved by the University of Virginia. Associated ClinicalTrials.gov Identifier: NCT03844789; registered February 18th, 2019 DCLP3 dataset: The source of the data is the University of Virginia. The International Diabetes Closed Loop (iDCL) trial. DCLP3 Public Dataset (Release 3). Retrieved from http:///public.jaeb.org/dataset/573. The analyses content and conclusions presented herein are solely the responsibility of the authors and have not been reviewed or approved by the University of Virginia. ClinicalTrials.gov Identifier: NCT03591354; registered July 19, 2018. CITY dataset: The source of the data is the Jaeb Center of Health Research. CGM Intervention in Teens and Young Adults with T1D (CITY Public Dataset). Retrieved from http://public.jaeb.org/dataset/565. The analyses content and conclusions presented herein are solely the responsibility of the authors and have not been reviewed or approved by the Jaeb Center of Health Research. ClinicalTrials.gov Identifier: NCT03263494; registered August 28, 2017. SENCE public dataset: The source of the data is the Jaeb Center of Health Research. SENCE Public Dataset. Retrieved from http://public.jaeb.org/dataset/554. The analyses content and conclusions presented herein are solely the responsibility of the authors and have not been reviewed or approved by the Jaeb Center of Health Research. ClinicalTrials.gov Identifier: NCT02912728; registered September 23, 2016.