When AI Gets It Wrong: Conscious Competence Development through Student Design Reflections
TL;DR
This research explores the use of a latent Dirichlet allocation model to automatically classify students' design reflections, thereby improving the efficacy of their reviews, and advocates for professional engineering licensure bodies to modernize policies to encourage thoughtful, rigorous evaluation of AI models before deployment.
Abstract
Reflections in engineering are a fruitful tool for encouraging lifelong learning and provide instructors with insight into how to adapt the learning environment to meet students' needs. However, reflections are impractical for large classes. This research explores the use of a latent Dirichlet allocation model to automatically classify students' design reflections, thereby improving the efficacy of their reviews. The model showed promising quantitative performance, but from a human reviewer's perspective, it lacked interpretability. The contradicting results encouraged a reflective discussion of the potential misuses and risks of artificial intelligence (AI), both within and outside the classroom, without a qualitative review. Based on the authors' experiences and lessons from this study, this paper advocates for professional engineering licensure bodies to modernize policies to encourage thoughtful, rigorous evaluation of AI models before deployment. This article was written to encourage engineering educators to utilize AI critically and responsibly.