SpartanAI: An AI Agent for Automated Course Review

Abstract

Quality assurance in online and blended course design remains labor-intensive, costly, and difficult to scale across institutions. Faculty often lack timely access to instructional design support aligned with established quality standards such as the Quality Matters (QM) rubric. While instructional designers bring deep expertise in online course design, faculty often lack equivalent familiarity with evidence-based design principles; SpartanAI provides these instructors with structured, rubricgrounded feedback that would otherwise require specialized expertise or a formal review process to obtain. While recent advances in large language models (LLMs) have enabled automated analysis of textual artifacts, most AI applications in education remain student-facing and do not provide systematic, rubricaligned course review services for instructors. In this paper we present SpartanAI, an LLM-enabled AI Framework that automates QM-aligned course review through structured learning management system (LMS) ingestion, module-level content extraction, semantic indexing, and rubric-by-rubric evaluation. Instructors upload exported course packages which are parsed into structured representations, indexed using vector embeddings, and evaluated rubric by rubric. For each rubric, the system retrieves relevant course artifacts, generates alignment scores, identifies supporting evidence, and provides actionable revision guidance. Initial evaluation against QM-certified reviewer feedback on a pilot set of courses shows promising alignment in rubric-level scoring and feedback relevance. These results suggest that structured LLM-based evaluation pipelines can augment institutional course quality assurance processes and provide scalable, instructor-centered AI support.