Evaluating the Effectiveness of a Faculty Development Program in Enhancing the Quality of Multiple-Choice Questions: A Quantitative Study
TL;DR
Incorporation of multiple structured faculty development workshops lead to measurable improvements in MCQ quality, and these training sessions can enhance the reliability and validity of medical assessments, benefiting both educators and learners.
Abstract
Background: Multiple choice questions (MCQs) are widely used in medical education for their objectivity and efficiency. However, poorly constructed questions can compromise the validity and reliability of the assessments. Faculty development programs have emerged as a key strategy to improve the quality of MCQs. Objective: This study aims to evaluate the impact of structured FDPs on the MCQs construction using objective metrics and participants’ feedback. Methods: This observational study was conducted over three years period (2022 - 2024) at University College of Medicine & Dentistry. Three workshops on MCQs development were organized for the faculty. Feedback was collected through structured forms using 5- point Likert scale. Additionally, MCQs submitted before and after the workshops were analyzed for item flaws, difficulty & discrimination indices. The study also evaluated the impact of training using the Kirkpatrick Model. The data was computed and analyzed using SPSS version 26.0. The Shapiro Wilk test determined the normality of the data. P value <0.05 was considered as significant. Result: A significant reduction in test-wiseness flaws (logical cues from 12.9% to 1.3%, p<0.001) and irrelevant difficulty (extraneous details from 7.6% to 0.9%, p<0.001) was observed. The proportion of moderately difficult items increased from 56.9% to 62.2%, while the proportion of very good discriminating items increased from 2.2% to 6.7%; however, neither change reached statistical significance (p=0.062 and p=0.098, respectively). Feedback from the participants revealed increasing satisfaction across workshops, with the highest ratings in workshop 3 (2024). Instructor expertise remained consistently rated above 87.5%. Conclusion: Incorporation of multiple structured faculty development workshops lead to measurable improvements in MCQ quality. These training sessions can enhance the reliability and validity of medical assessments, benefiting both educators and learners.