‘A bit of chaos and madness’: the AI assessment scale and the work of assessment reform

Abstract

Generative artificial intelligence (GenAI) has intensified pressure on universities to redesign assessment while maintaining integrity, equity, and validity. Frameworks such as the Artificial Intelligence Assessment Scale (AIAS) offer one response, but evidence of how faculty experience their implementation remains limited. This qualitative study examines AIAS implementation as two parallel cases: a private international university in Vietnam, which adopted the first version institution-wide, and a public UK university, which adopted the revised version in a single school. Data from five focus groups with 30 faculty members were analysed using hybrid thematic analysis, with Critical AI Literacy as a sensitising concept. Six themes were developed: recognising and integrating AI, facilitating conditions, building capacity, pathways to adoption, ethics in practice, and reframing pedagogy. Faculty valued the AIAS as a shared language for legitimising GenAI use, clarifying boundaries, and prompting reflection on assessment design. However, they described implementation as shaped by governance, tool access, faculty confidence, workload, integrity concerns, disciplinary context, and alignment with learning outcomes. Their accounts suggest the AIAS could prompt authentic assessment design and student engagement, but may become a compliance layer when disconnected from those conditions. The study contributes evidence on the conditions through which GenAI frameworks reach pedagogical enactment.