Findings are interpreted as evidence that course-aware style feedback is promising as a pre-submission revision aid, but that future versions should combine deterministic rule checks with LLM-generated explanations, rule citations, and stronger verification support.
Abstract
This innovative practice full paper presents CodeStylist, a web application that provides course-standard-aware code style feedback for early undergraduate programming courses. CodeStylist addresses a common instructional gap: students are expected to follow local conventions for naming, formatting, comments, organization, and readability, but feedback on these expectations is often delayed or inconsistent. Unlike generic linters or general-purpose LLM prompts, CodeStylist supports course-specific standards, multi-file submissions, and file- and line-localized explanations intended to guide revision rather than grade correctness. We report a formative expert review with 18 instructional staff from one early undergraduate programming course. Participants explored the prototype using self-selected code artifacts and completed a survey about response quality, anticipated student use, and redesign priorities. Ratings indicated modest perceived utility but limited trust: perceived correctness averaged 60.7%, response helpfulness averaged 3.50/5, response usefulness averaged 3.33/5, and anticipated student learning averaged 2.61/5. Despite these concerns, 17/18 respondents expected students to use the tool primarily for style checking, often at least weekly. Open-ended feedback showed that respondents valued CodeStylist for making implicit course standards more visible, but were concerned about unreliable output, overreliance, and latency or cost. We interpret these findings as evidence that course-aware style feedback is promising as a pre-submission revision aid, but that future versions should combine deterministic rule checks with LLM-generated explanations, rule citations, and stronger verification support.
Peer code reviews involve students conducting a code review of a classmate's submission to a programming assignment. While peer code reviews have an established history of being used and studied in computing education, they have primarily been documented in introductory computing courses. This experience report describes how we implemented peer code reviews in an upper-division software engineering course that focuses on making modifications to large, existing code bases (i.e., brownfield development). We discuss the perceived learning benefits, perceived challenges, and agreement between peer and course staff reviews. Overall, students enjoyed being able to see different approaches to the programming task they had just submitted, but expressed concerns about feeling qualified to make effective peer code reviews due to their limited software engineering experience and the difficulty of assessing code design. We also find that students are capable of evaluating the functional correctness of their peer's submission, but struggle to give accurate assessments of their peer's design and code style. We conclude with recommendations specifically for instructors who wish to use peer code reviews in upper-division software engineering courses, such as using a structured template to scaffold the peer code reviews and allowing multiple opportunities to provide code reviews to improve students' self-efficacy.
Anshul Shah, Thomas Rexin, Andrew Smithwick et al.· Annual Conference on Innovat...· 0 citations
Despite a wealth of documented programming misconceptions, collections remain difficult to use in practice: static lists with no priority guidance leave educators without support for deciding which misconceptions deserve attention. We present a web-based misconception repository for Java/Processing designed to change this, with each entry connecting misconceptions to origins, typical errors, and concrete teaching opportunities. To support navigation, we introduce two teaching-relevance criteria: Learner Discoverability (how likely learners are to notice a misconception themselves) and Learning Barrier (how strongly it hinders progress if unaddressed). These criteria were developed iteratively using LLM-based pilot ratings to validate and refine definitions before involving human raters, reducing an initial set of five dimensions to two. Three human instructors rated all misconceptions, yielding moderate inter-rater reliability (α = 0.57 and α = 0.60), with a majority vote raising consistency to α = 0.9 and α = 1.0. Combined with observed frequency data and topic/subtopic filtering, the criteria enable sorting and compound filters (such as Silent Blockers or Common Obstacles) that translate ratings into actionable entry points for lesson preparation and tutor training, going beyond existing collections that offer no priority guidance or instructional filtering.
Corinna List, Moritz Mueller, Michael Kipp· Proceedings of the 2026 Unit...· 0 citations
The rapid integration of large language models (LLMs) into undergraduate education presents an urgent challenge for engineering instructors. Despite widespread student adoption, there remains a critical lack of domain-specific empirical evidence to guide pedagogical policies and classroom interventions. This manuscript presents a descriptive study design and preliminary findings from an undergraduate engineering mechanics course conducted in Spring 2026. We detail a reproducible survey instrument used to capture student AI usage patterns, attitudes, and verification practices, which are subsequently linked to academic performance metrics. Additionally, we document a deployable sequence of nine structured, instructor-led AI demonstrations designed to model strategic LLM delegation and evaluation. While our preliminary data highlight shifting student behaviors and complex relationships between AI reliance and course outcomes, the primary contribution of this work is the provision of an open-access methodological framework. By making our complete study design, survey tools, and demonstration materials publicly available, we urge other engineering educators to collect and share similar empirical data. Navigating this unprecedented technological shift will require a collaborative, evidence-based approach to fully understand its long-term impacts on student learning.
S. Geng, Helen Lallos-Harrell, Jiya Ashar et al.· 0 citations
Addressing learner heterogeneity in programming education is challenging due to variations in student speed, prior knowledge, and motivation. While differentiated instruction, such as tiered sequences, allows students to engage at appropriate difficulty levels, manually creating these resources is labour-intensive. This paper introduces CodeOwl, an AI-driven tool that automates the generation of tiered Parsons problems. Starting from a sample task or specific programming concepts, CodeOwl produces tiered sequences of Parsons problems automatically. We evaluated CodeOwl with a mixed-method framework comprising complexity analysis, expert ratings, and user studies. Analysis of 297 tiered sequences (three tiers each) revealed that 98.7% achieved a positive complexity increase, successfully rising in difficulty from Tier 1 to Tier 3. Experts rated the generated problem statements as highly clear. While teachers praised the tool's utility, they identified a need for greater control over curriculum alignment. Similarly, students reported positively but requested enhanced feedback mechanisms and alternative interaction modes.
Luca Cisternino, Florian Obermuller, Gordon Fraser· 0 citations
Delivering high-quality formative feedback in large introductory programming courses remains challenging despite the widespread use of automated grading systems. Although large language models (LLMs) can generate natural-language explanations grounded in student code and test outcomes, their diagnostic reliability and pedagogical suitability require systematic evaluation. This study presents a comparative analysis of six LLMs for generating formative feedback on introductory Java programs containing predefined defects under controlled conditions. Model outputs were evaluated across key dimensions of feedback quality and performance. Results reveal substantial cross-model variation, particularly in multi-defect scenarios. More precise fault localization was associated with higher diagnostic accuracy and completeness, but also with more frequent instructional-constraint violations. No single model consistently outperformed others across all criteria, and response latency did not reliably correspond to diagnostic quality. These findings underscore the context-dependent nature of LLM-generated formative programming feedback.
Melina Najimi, Saba Yazdani, Marzieh Ahmadzadeh· Proceedings of the Canadian...· 0 citations
In programming courses, instructors may need to interpret whether a submission is consistent with a student's prior programming profile, especially when code similarity alone is inconclusive. Existing source-code authorship methods are often evaluated on programming-contest or open-source datasets, where reusable templates and local code patterns can produce strong author-related signal. Educational repositories present a different setting. Students solve shared assignments while their programming practices are still developing. This study uses task-aware evaluation to contrast these production contexts and tests whether repository-visible process features add information beyond final code in six matched educational comparisons. Contest data provide a high-signal contrast, with a Kick Start mean top-1 of 0.938. Educational datasets produce substantially lower attribution performance. Adding process features raises the educational mean from 0.094 to 0.233 and mean pairwise verification ROC-AUC from 0.556 to 0.752. The comparisons show that measured signal depends on production context and that process patterns can complement weak final-code signal in educational repositories. Such models are therefore appropriate only as instructor-mediated decision support, not as independent proof of authorship.