Beyond Threshold Alarms: A Multimodal On-Premises AI Framework for Intelligent Fault Detection and Guided Maintenance of Industrial Pump Set

Abstract

Abstract Equipment breakdowns in the oil and gas (O&G) industry are costly and disruptive, both financially and operationally. Predictive maintenance powered by artificial intelligence (AI) can help prevent these breakdowns, but most AI solutions rely on the cloud — which is often impractical at O&G sites because of data security requirements and unreliable connectivity. This paper presents an on-premises, multimodal AI framework for predictive maintenance that keeps the field engineer at the center of the process, rather than replacing them. The framework combines three AI capabilities that work together to support the people doing the actual repair work. A lightweight LSTM model continuously watches sensor data — vibration, temperature, pressure, and power — to catch early signs of trouble. A YOLO-based vision model watches live camera feeds to spot visible problems such as seal leakage, misalignment, and wear. When either model flags something unusual, the two models are combined in a fusion layer to produce a more confident diagnosis. That diagnosis is then handed to a retrieval-augmented generation (RAG) chatbot, running on a small language model hosted entirely on site. Rather than sending a static alert, the chatbot has a conversation with the field engineer — asking clarifying questions to narrow down the likely cause, as an experienced colleague would, and drawing its answers from historical maintenance records and OEM manuals rather than guessing. Once the cause is identified, an augmented reality (AR) interface walks the engineer through the repair step by step, right at the equipment. Because everything runs on-premises, the framework continues to work even when connectivity is unreliable and keeps sensitive operational data on site rather than sending it to the cloud. The human-in-the-loop design is central to this: AI does not make the call on its own — it gives the field engineer earlier warning, a second set of eyes on visible defects, and an expert-like assistant to think through the problem before acting. This paper describes the architecture, presents field observations from an initial deployment, and discusses how the approach reduces unplanned downtime while staying practical for the connectivity and security constraints typical of O&G field sites.