Navigating Large Programming Misconception Collections Using Teaching-Relevance Criteria

Abstract

Despite a wealth of documented programming misconceptions, collections remain difficult to use in practice: static lists with no priority guidance leave educators without support for deciding which misconceptions deserve attention. We present a web-based misconception repository for Java/Processing designed to change this, with each entry connecting misconceptions to origins, typical errors, and concrete teaching opportunities. To support navigation, we introduce two teaching-relevance criteria: Learner Discoverability (how likely learners are to notice a misconception themselves) and Learning Barrier (how strongly it hinders progress if unaddressed). These criteria were developed iteratively using LLM-based pilot ratings to validate and refine definitions before involving human raters, reducing an initial set of five dimensions to two. Three human instructors rated all misconceptions, yielding moderate inter-rater reliability (α = 0.57 and α = 0.60), with a majority vote raising consistency to α = 0.9 and α = 1.0. Combined with observed frequency data and topic/subtopic filtering, the criteria enable sorting and compound filters (such as Silent Blockers or Common Obstacles) that translate ratings into actionable entry points for lesson preparation and tutor training, going beyond existing collections that offer no priority guidance or instructional filtering.