FroLineR: Front-Line Response with Retrieval-Augmented Prompt-Engineered Reply Generation for IT Help Desks
Abstract
IT help desks at large organizations face a high volume of recurrent, well-documented user requests that nevertheless require human-written replies, creating a persistent staff workload that is repetitive in content but non-trivial in tone and procedural correctness. We present FroLineR, short for Front-Line Response, a system that drafts the initial staff reply to such tickets in the login and account-activation category and integrates into a human-in-the-loop ticketing workflow on a Romanian-language ticketing platform. The generator is an unmodified instruct model augmented with retrieval from a small set of hand-curated guide documents, using a Romanian system prompt refined over several rounds of staff review. To evaluate and refine the prompt without manual labeling, we cluster the first user message of every historical thread with both BERTopic and Semantic Signal Separation (S3), score configurations along coherence and lexical-diversity axes, and extract a 200-message evaluation set from the winning model. Prompt convergence was certified by several rounds of manual review by support staff. The production system is quantized to Q4_K_M GGUF, served through llama-cpp-python behind a small Flask API, and deployed with GPU offloading on the target server, reducing end-to-end per-answer latency from approximately 830 s on the server’s CPU to roughly 61 s once layers are offloaded to the GPU, with no observable degradation in answer quality.