WikiMonitor-onto: Ontology-aware Staleness Propagation for LLM-maintained Knowledge Bases
Abstract
. Large Language Models (LLMs) are increasingly used to maintain persistent, domain-specific knowledge bases—a paradigm in which assertions must remain accurate as the field evolves. Existing staleness detection treats each assertion independently, missing a structural reality: when a foundational concept becomes outdated, every dependent concept inherits some degree of that staleness through ontology relationships. We present WikiMonitor-Onto, a lightweight propagation layer built on WikiMonitor that models staleness as a signal flowing through a domain ontology graph. We extract a concept graph of 642 nodes and 487 edges from 61 AI lecture documents, define three typed propagation relations ( is-a , depends-on , related-to ), and propagate staleness via weighted Breadth-First Search (BFS) with exponential hop decay. On a human-annotated gold standard of 62 concepts (25 indirect-stale, 37 fresh), the independent baseline detects zero indirect-stale concepts by construction, while WikiMonitor-Onto achieves precision 0.824 and recall 0.560 at the optimally tuned configuration ( λ = 0.30). Grid search reveals that is-a and depends-on carry equal optimal propagation weight (both 0.90). A sensitivity analysis confirms that propagation is robust to the choice of seed-value distribution, with F1 varying by only 0.10 (0.571–0.667) across four tested distributions. Propagation saturates at hop depth 1 under conservative thresholds, suggesting that one-hop propagation suffices for high-precision deployment.