Enterprise AI Prompt Screening System
Abstract
Abstract: Generative Artificial Intelligence (AI) tools are currently being widely used in business environments, but the prompts employed are unstructured text that can include personal data, financial information, credentials, or confidential information about an organization. There is also a risk involved in prompt injection, since it enables user instructions to circumvent the existing rules or reveal the system's configuration. In most situations, there is no technical method of controlling access between the user and the model. The present paper introduces an Enterprise AI Prompt Screening System which features a deterministic and explainable screening phase before a large language model that is run locally. The system makes use of Spring Boot, Spring Security, Spring AI, React, and MySQL; it identifies sensitive entities by means of Java recognisers based on rules, detects signals indicating that direct prompt injection is occurring, calculates a weighted Prompt Risk Score, and then carries out one of five policy actions: Allow, Mask, Confirm, Review, or Block. Prompts that are simply allowed or sanitized are sent on to an offline Ollama model. The system is to be evaluated using a synthetic labelled prompt dataset according to the predefined acceptance criteria
