These findings endorse the viability of LLMs as a feasible alternative for data error detection; nonetheless, challenges such as hallucination and cost aspects remain, especially regarding the resources necessary for the effective implementation of these models in practical applications.
Modern Software-as-a-Service (SaaS) environments generate voluminous, heterogeneous logs that exhibit rapid schema drift and semantic sparsity, rendering traditional template-based detection methods ineffective. This paper presents LLM-EP, a unified framework that utilizes Large Language Models (LLMs) for proactive error prediction via semantic log analysis. Unlike existing heuristics, LLM-EP employs a drift resilient semantic parser and an embedding driven normalization engine to maintain accuracy amidst evolving log formats. We propose a hybrid offline-online architecture that balances deep semantic understanding with real-time inference requirements, incorporating a robust outof-vocabulary (OOV) mechanism for zero-shot generalization to unseen log patterns. Evaluation on production SaaS datasets indicates that LLM-EP achieves a 25% gain in parsing accuracy and a 12% improvement in F1-score over state-of-the-art baselines. Most notably, LLM-EP reduces Mean Time to Detect (MTTD) by up to 60%, demonstrating its efficacy as a scalable foundation for automated site reliability engineering.
The viability of integrating LLM‐assisted fault detection into specific industrial scenarios is investigated and an improvement in fault detection accuracy over conventional methods is demonstrated in this work.
Yihua Ye, Yin Zhu, Liming Che et al.· Safety Science and Technolog...· 0 citations
The results show that the studied configuration options significantly impact energy and performance, mainly driven by attention type and prefix caching, while chunked prefill has a limited effect under the default vLLM serving configuration and evaluated workloads.
Nada Zine, Tristan Coignion, V. Stoico et al.· 1 citation
The proposed spectral signature provides a meaningful proxy for broad performance trends across LLMs, enabling efficient organization, comparison, and analysis of large model collections.
Zhuoying Zhang, Ishanvi Prasad, Yuanzhe Hu et al.· Proceedings of the 32nd ACM...· 0 citations
LLM-Detector is proposed, a framework that utilizes the in-context learning capacity of LLMs for structured, prompt-conditioned scoring synthesis, enabling LLMs to derive anomaly detection logic from structured normal-state knowledge.
The first Poisoned Chalice of LLM Evaluation Competition is organized, which frames contamination detection as a white-box membership inference task on source code and provides participants with curated datasets, target models, baseline attacks, and a final evaluation on a held-out model and dataset.
J. Katzy, Ali Al-Kaswan, R. Popescu et al.· SIGSOFT FSE Companion· 1 citation· ⚡1