Educators’ experiences transitioning from a traditional to an AI-aware assignment rubric: a qualitative study in a health and medicine course

Abstract

Assessment strongly shapes student learning and engagement. Traditional assessment rubrics in health and medicine education often fail to account for emerging challenges posed by artificial intelligence. The use of AI tools risks encouraging surface-level learning and undermining academic integrity if not addressed through purposeful assessment design. This study evaluates the impact of transitioning from a traditional to an AI-aware rubric in a Health and Medicine course, examining how this shift influenced educators’ grading practices, instructional strategies, and perceptions of student engagement. A qualitative research design employed semi-structured interviews with three educators who had taught the course using both original and revised rubrics. As the complete teaching team, their perspectives captured the experience of every educator who taught the course. Interview data were analysed using thematic analysis, with themes cross-referenced against the Twelve Tips for Designing AI-Resilient Assessments framework. Analysis revealed five overarching themes: perceived transformation in student learning and critical thinking; refinement of assessment practices and expectations; expansion of teaching strategies and curriculum design; implementation challenges and assessment authenticity; and institutional readiness and professional development. Educators perceived that the revised rubric supported deeper student engagement with AI, increased their grading confidence, and prompted shifts in instructional approaches. AI-aware rubric design can meaningfully reshape both teaching and assessment practices. However, sustainable implementation requires institutional support, structured professional development, and clearer policy guidance on acceptable AI use. The study also refines the Twelve Tips framework, positioning it as a flexible design heuristic rather than a prescriptive checklist. These findings are particularly relevant for undergraduate health professions programs navigating rapid educational and technological change.