From concepts to clinicals: A comparative study of artificial intelligence platforms for competency-based medical education and postgraduate pharmacology problem-solving

Abstract

Artificial intelligence (AI) platforms have increasingly integrated into medical education, offering innovative avenues for enhancing critical thinking, subjective learning, and clinical reasoning. Large language models such as ChatGPT, Gemini, and Meta AI simulate human-like responses and are increasingly being utilized as adjunctive educational tools. However, systematic evaluations comparing their performance within competency-based medical education (CBME) in pharmacology remain limited. A cross-sectional comparative study was conducted to evaluate ChatGPT, Gemini, and Meta AI in answering 55 higher-order pharmacology questions mapped to CBME undergraduate competencies and postgraduate standards. Responses were anonymized and independently scored by three blinded pharmacology faculty members for accuracy, completeness, and clarity. Statistical analysis included the Friedman test, Wilcoxon signed-rank test for pairwise comparisons, and repeated measures analysis of variance (ANOVA). Data visualization was performed using bar graphs and box-and-whisker plots. ChatGPT achieved the highest mean score (8.5 ± 1.0), followed by Gemini (7.9 ± 1.2) and Meta AI (6.4 ± 1.5). The Friedman test indicated a highly significant difference among the platforms ( χ ² = 96.69, P < 0.001). Post hoc Wilcoxon signed-rank tests showed that ChatGPT significantly outperformed both Gemini and Meta AI ( P < 0.001), while Gemini also outperformed Meta AI ( P < 0.001). Repeated measures ANOVA confirmed significant differences in mean scores across platforms ( F = 217.36, P < 0.001). Among the evaluated AI platforms, ChatGPT demonstrated superior performance in solving higher-order pharmacology questions aligned with CBME frameworks. These findings support the integration of AI tools, as adjunctive resources in pharmacology education.