Skip to content
Preprint

Auditable Release Control for Pedagogical Leakage in LLM Tutors

Aug 2026 · 0 citations · 48 references
Computer Science

TL;DR

These results establish an auditable release boundary and failure attribution under declared contracts, not universal semantic safety or learning gains, not universal semantic safety or learning gains.

Abstract

Large language model tutors can be correct and helpful yet disclose an answer or decisive reasoning before that disclosure is authorized. We formalize this state- and action-dependent failure as pedagogical leakage and introduce an authorization-aware complete-mediation boundary. A selector emits one of five disclosure contracts, trusted policy gates privileged modes, and a renderer proposes language. A single release function applies inspectable checks, optional cumulative verification, and action-specific fallback; replayable traces separate selection, generation, verification, and enforcement failures. Matched component attribution exposes a safety-utility frontier. On 599 fixed Gemini 3.5 proposals, strict mediation reduces blinded three-model panel-majority leakage flags from 181 to 0 (paired problem-cluster difference -30.22 points, 95% CI [-35.00,-25.72]), while replacing 581 responses and lowering helpfulness. Checker-triggered fallback alone yields 11 majority flags; adding the semantic verifier yields 14 and no reliable marginal gain. A global A1 scaffold yields 0 majority and 54 any-judge flags, outperforming fitted Q on automatic safety and utility. In an externally timestamped replication over 40 unseen problem clusters and 480 attack sequences, high-assurance release reduces majority flags from 42 to 8 (-7.08 points, 95% CI [-13.13,-2.29]); seven failures persist, one is introduced, and mean helpfulness falls by .192. These results establish an auditable release boundary and failure attribution under declared contracts, not universal semantic safety or learning gains.

View source

Similar papers

2026

Auditing the Reliability of Automated Risk Evaluators in Agentic Tutoring Systems Under Pedagogical Style Variations

Agentic tutoring systems increasingly rely on large language models (LLMs) to generate real-time feedback, yet risky outputs, such as answer leakage, contradiction, and hallucination, can directly affect learning. Answer leakage is a central concern in this setting. It exposes problem solutions to students and can be v...

Wen-Zhuo Song, Xuechen Wang, Shuang Liang et al. · 0 citations
Preprint Aug 2026

Prior Audit-Repair Context Shifts LLM Verifier Thresholds Toward Leniency

Measuring false alarms on human-verified-correct ProcessBench traces with the present task held byte-identical, it is found that a completed audit ->repair episode already in the model's context lowers false alarms in 15 of 15 model x wording combinations.

Parsa Mazaheri, Kasra Mazaheri · 0 citations
Jul 2026

Context-Masked Truncated Reasoning Audits for Answer-Key Dependence in LLM Tutors

Context masking is established as necessary for attributing early answer availability to an explanation rather than its hidden input when early-prefix evidence disappears after masking.

Bo-Nan Shen, Ding-Yan Shang, You Wang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.