LLM-Based Detection and Test Generation for Command Injection Vulnerabilities in Python
Command injection vulnerabilities remain a significant security threat in dynamic languages such as Python, particularly in widely used open-source projects. Recent advances in large language models (LLMs) have shown strong potential in code-related tasks, motivating their application to vulnerability detection.In this paper, we evaluate LLM-based approaches for detecting command injection vulnerabilities and generating security tests. We apply our method to six widely used GitHub projects—Django, Flask, TensorFlow, Scikit-learn, PyTorch, and LangChain—covering over 50,000-star repositories with broad real-world usage. Our study analyzes the effectiveness of LLMs in terms of detection accuracy, efficiency, and practical applicability.We further compare multiple LLMs to assess their relative performance in vulnerability detection and test generation. The results demonstrate that LLM-based approaches can effectively identify vulnerabilities in both complete and fragmented code while providing executable security tests for validation. Our findings highlight the strengths and limitations of LLMs and offer practical insights for leveraging them to enhance automated software security testing.