Application of Large Language Models for Detecting Semantic Ambiguity in Industrial Instructions: Impact on Human–Machine Interaction and User Experience in Process Automation Systems of a Metallurgical Plant

Abstract

In the context of industrial digitalization and the widespread adoption of process automation systems, Knowledge Management Systems (KMS) play a key role in providing operational personnel with up-to-date instructions and regulations. However, the inherent ambiguity of natural language in technical documentation remains a serious obstacle, leading to incorrect operator actions, process deviations, and increased safety risks. This article investigates the integration of Large Language Models (LLMs) into KMS and its impact on user experience and human–machine interaction in industrial automation environments. A method called Semantic Latent Choice Detection is presented, designed to systematically identify interpretation ambiguities in process instructions and operator commands. Unlike existing approaches that require access to the internal model architecture (“white box”) or token-level logits, the proposed method is logit-free and operates with closed commercial LLMs (“black box”) via standard API interfaces. The method analyzes the semantic similarity of binary text blocks and polysemous terms within the context of a specific technological process. Using a metallurgical production case study, we demonstrate how the system detects hidden semantic collisions (e.g., the difference between “adding ferroalloys into the ladle” and “feeding ferroalloys onto the conveyor”) that are missed by traditional rule-based validation methods. Instead of arbitrarily selecting an interpretation, the system initiates a clarification request to the human operator, thereby reducing cognitive load, preventing erroneous automated decisions, and increasing trust in the KMS. An empirical evaluation conducted in a real-world industrial setting (unit control rooms and dispatch centers) shows a statistically significant reduction in errors related to misinterpretation of process regulations. The article contributes to the fields of automation engineering, knowledge management, and human-centered automation by proposing a novel method for validating operational instructions in high-risk industrial environments.