Jul 2026· International Conference on Artificial Intelligence Testing· pp. 68-75· 0 citations· 25 references
Abstract
Command injection vulnerabilities remain a significant security threat in dynamic languages such as Python, particularly in widely used open-source projects. Recent advances in large language models (LLMs) have shown strong potential in code-related tasks, motivating their application to vulnerability detection.In this paper, we evaluate LLM-based approaches for detecting command injection vulnerabilities and generating security tests. We apply our method to six widely used GitHub projects—Django, Flask, TensorFlow, Scikit-learn, PyTorch, and LangChain—covering over 50,000-star repositories with broad real-world usage. Our study analyzes the effectiveness of LLMs in terms of detection accuracy, efficiency, and practical applicability.We further compare multiple LLMs to assess their relative performance in vulnerability detection and test generation. The results demonstrate that LLM-based approaches can effectively identify vulnerabilities in both complete and fragmented code while providing executable security tests for validation. Our findings highlight the strengths and limitations of LLMs and offer practical insights for leveraging them to enhance automated software security testing.
The increasing deployment of Large Language Models (LLMs) in critical infrastructure has introduced a class of security risks that remain insufficiently characterised and poorly tooled in practice. Attack vectors including prompt injection, jailbreaking, code execution facilitation, covert data exfiltration, and traini...
M. Yamin· International Conference on...· 0 citations
The application of LLMs for detecting malicious PowerShell scripts and producing human-interpretable explanations for their classification decisions are investigated, showing that LLMs are capable of identifying and explaining malicious PowerShell scripts, although performance varies across different models.
Meng Wang, Emma Topolovec, B. Arana et al.· 0 citations
The proposed CodeSIFT, a threat-model-agnostic detection method that leverages influence functions to identify batches of prompts that induce anomalous model behavior, suggests that influence-function-based detection is a promising direction for identifying malicious code-generation prompts without requiring prior know...
Francesco Quinzan, Noor Munir, Yi-Shun Lu et al.· 1 citation
Findings show that insecure code generation is not merely a collection of independent defects, but a structured and prompt-conditioned phenomenon, motivating cluster-aware verification and prompt-level intervention for safer LLM-assisted programming.
VICBench enables robust evaluation of vulnerability detection approaches and shows that state-of-the-art algorithms V-SZZ and LLM4SZZ achieve only 33.3%-40.1% F1, confirming that using existing approaches still entails significant manual effort.
Jin Lu, Xuening Han, Yan Zhong et al.· 0 citations
Software vulnerability detection using Static Application Security Testing tools still produces a high number of false positives, which increases the burden of manual verification for developers. A high false positive rate can reduce development efficiency, divert attention from critical vulnerabilities, and indicate a...
Danu Tryas Pristowo, Dana Sulistiyo Kusumo· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
