Exploratory benchmarking of AI-generated diet plans for inherited protein metabolism disorders: a simulation-based evaluation of nutritional accuracy and clinical safety

Abstract

Artificial intelligence (AI)-based large language models (LLMs) are increasingly used to support nutrition-related decision-making; however, their ability to generate clinically appropriate dietary plans for inherited protein metabolism disorders remains largely unexplored. This study aimed to perform an exploratory simulation-based benchmarking analysis of AI-generated dietary plans for phenylketonuria (PKU), maple syrup urine disease (MSUD), and propionic acidemia (PPA) using disease-specific metabolic nutrition guidelines. Standardized pediatric case scenarios were developed for PKU, MSUD, and PPA. Using identical English-language prompts, ChatGPT-5.3 Pro and Gemini 3 Pro Advanced each generated 3-day dietary plans. Nutrient composition was analyzed using the BeBiS Nutrition Information System and evaluated against Dietary Reference Intakes (DRIs). Disease-specific nutritional targets, including amino acid intake, protein distribution, and energy provision, were benchmarked against recommendations from Genetic Metabolic Dietitians International (GMDI). Nutritional characteristics of the dietary plans generated by the two AI models were compared using exploratory statistical analyses. Both LLMs generated structured dietary plans with generally acceptable overall nutritional characteristics; however, clinically relevant deviations from disease-specific nutritional targets were identified across all three disorders. In the PKU case, both models achieved the recommended phenylalanine range, but neither simultaneously met protein and tyrosine recommendations. In the MSUD case, differences were primarily related to energy provision and branched-chain amino acid targets, while in the PPA case neither model achieved the recommended balance between intact protein and total protein. These findings demonstrated that conventional measures of nutritional adequacy alone were insufficient to determine the clinical appropriateness of AI-generated dietary plans for inherited protein metabolism disorders. General-purpose LLMs can generate structured dietary plans for inherited protein metabolism disorders; however, disease-specific metabolic targets are not consistently achieved. Evaluation of AI-generated dietary plans should therefore extend beyond conventional nutritional assessment and incorporate disease-specific benchmarking against established metabolic nutrition guidelines. This study provides a disease-specific benchmarking framework for evaluating AI-generated dietary plans in inherited protein metabolism disorders.