HintLab: An Interactive Tool for Hint Generation, Analysis and Evaluation

Abstract

Large Language Models (LLMs) are increasingly used in educational contexts, yet their default answer-centric interaction paradigm conflicts with established principles of effective learning. Direct solution delivery can reduce productive struggle and encourage cognitive offloading rather than deep understanding. To address this tension, we present HintLab, a visualization-driven, human-in-the-loop platform for generating, analyzing, and refining LLM-generated pedagogical hints. The system operationalizes five cognitively grounded evaluation dimensions—Convergence, Relevance, Readability, Familiarity, and Leakage Avoidance—and exposes them through interactive visual analytics that support comparative inspection, manual refinement, and scaffolded sequencing of hints. Unlike standard chat interfaces, HintLab enables structured multi-model comparison, answer-space reduction analysis, and convergence-based scaffolding design without requiring domain modeling or programming expertise. A human evaluation involving 901 generated hints demonstrates favorable ratings across multiple pedagogically motivated dimensions, while a hint-assisted question answering study shows that the generated hints substantially improve users' ability to answer previously unsolved questions. The system is publicly available, with source code accessible on GitHub, and a demonstration video online.