A rubric to assess generative AI-based feedback on student writing assignments
TL;DR
A rubric is described to enable instructors to assess differences between feedback provided by GenAI models based on five key areas, to serve as a resource for other instructors interested in evaluating various GenAI models for use as scientific writing feedback supplements in their own classrooms.
Abstract
ABSTRACT Generative artificial intelligence (GenAI) tools are an increasingly common resource used in the classroom and writing process. The landscape of available GenAI tools is rapidly evolving, so having a systematic and straightforward way to evaluate new tools for incorporation into the classroom is key. For example, in science writing education, GenAI tools can be used as a supplement to instructor feedback on student writing, allowing an additional opportunity for critique and revision by the student. Here, we describe a rubric we developed to enable instructors to assess differences between feedback provided by GenAI models based on five key areas: (i) accuracy, (ii) constructiveness, (iii) clarity and readability, (iv) recognition of strengths, and (v) original text. We show data comparing the feedback provided by multiple GenAI models on student work based on course guidelines. This rubric can serve as a resource for other instructors interested in evaluating various GenAI models for use as scientific writing feedback supplements in their own classrooms.