A Conversational Agent with Hybrid NLU and Dual-Corpus RAG for KPI Alert Management in Tourism Business Intelligence

Abstract

Monitoring operational Key Performance Indicators (KPIs) in Business-to-Business (B2B) tourism platforms demands continuous reconfiguration of alert systems, a task that conventional interfaces render inaccessible to non-technical stakeholders confronted with multi-screen forms and proprietary identifiers. This paper presents a conversational agent that lets such users create, modify, list, and explain KPI alerts through natural language, while guaranteeing the structural correctness of every configuration. Its core contribution is a schema-derived slot-completeness model of nine slot groups constraining a Large Language Model (LLM) tool-calling agent, paired with normalisation patterns derived at runtime from live database metadata and a bounded validation loop returning field-level errors to the model. A dual-corpus Retrieval-Augmented Generation module grounds the agent’s knowledge branch in schema documentation and a JSON-LD ontology, while configuration is grounded in live metadata; a human-in-the-loop checkpoint precedes every commit. The system is deployed as a prototype and evaluated in an automated pilot over a 50-utterance corpus, where it reaches 81.2% exact configuration match against 38.5% for the strongest unconstrained baseline (+42.7 percentage points, McNemar p<0.001) and emits no invalid schema identifier, against 15.8% for that baseline. A single-layer ablation locates the effect in the schema-aware tool layer. Corpus, annotations, prompts and evaluation scripts are released for replication.