Human-in-the-Loop Generative AI for KOS Registry Publishing: Designing a Dual-Automation Workflow
TL;DR
A practical, quality-assured method for publishing KOS-related RFPs as context-bearing registry records is contributed by integrating a two-track pipeline—scripted page generation for structured fields and generative summarization for narrative RFP text—with HITL governance for expert verification and correction.
Abstract
Knowledge organization system (KOS) registries bring together diverse KOSs under a shared metadata framework, making it easier to gain a coherent overview and compare them across domains and types. However, the contextual documentation needed to properly interpret, differentiate, and interlink individual KOSs is not always readily available or verifiable at the point of registration. This paper treats KOS-related requests for proposals (RFPs) as context-bearing documentation and proposes a human-in-the-loop (HITL), generative artificial intelligence (AI)-assisted workflow for publishing RFP pages in a MediaWiki-based registry. The workflow separates two automations: (1) an Apps Script pipeline that generates MediaWiki-ready wikitext from structured fields in a Notion database through a fixed, rule-based mapping to produce a standardized page skeleton, and (2) slot-based structured summarization that converts narrative RFP text into publishable sections (Tasks, Methods, Deliverables). Accountability is strengthened through expert-led validation of both outputs—verifying the generated page structure and ensuring that slot-based summaries remain faithful to the source RFP text—followed by targeted manual revision where automation alone is insufficient. In an applied case, the structured pipeline was operationally stable, with medium-complexity cases dominated by recurring issues in classification notation formatting and cross-database value mapping. These issues were largely correctable but still required expert review and manual correction. The summarization pipeline produced largely publishable drafts. Revisions were concentrated in the Deliverables slot and in items requiring multiple regeneration runs, while unsupported additions were rare but consequential and were removed or corrected through expert review prior to publication. This study contributes a practical, quality-assured method for publishing KOS-related RFPs as context-bearing registry records by integrating a two-track pipeline—scripted page generation for structured fields and generative summarization for narrative RFP text—with HITL governance for expert verification and correction.