Jun 2026· International Joint Conference on Computer Science and Software Engineering· pp. 85-90· 0 citations· 18 references
Abstract
Penetration testing is still highly dependent on trained professionals (who are required to manually coordinate heterogeneous security tools, multi-phase findings, and actionable reports) that is both time-consuming, expertise-oriented and inadequate to the speed of current threats. The framework described in this paper, SHARKAPT, is an autonomous AI-based penetration testing framework that uses the Model Context Protocol and large language model orchestration to conduct end-to-end automated security testing of both network and web application-related systems. SHARKAPT uses a multi-agent architecture based on LangGraph with five specialised agents: reconnaissance, network penetration testing, web application testing, exploitation as well as report generation and a decision engine that dynamically chooses 60+ real-world security tools. Google Gemini 3.1 Pro evaluated empirically against Metasploitable2 and OWASP Juice Shop has coverage of 71.7% of the Open Web Application Security Project Top 10 2021 challenges (9/10 categories), 79.1% Metasploitable2 known-vulnerability coverage, a 99.1% step success rate on the web session (111 steps), and a 94.5% step success rate on the network session (73 steps). All finding results are rated using a deterministic CVSS v3.0 calculator and stored in organised HTML reports containing per-finding evidence.
MTGuard is proposed, a hybrid analysis-based defense framework designed to safeguard the use of MCP tools in LLM agents by leveraging lifecycle-aware static-dynamic co-analysis and effectively mitigates multiple categories of harmful tool use across different LLM agents while maintaining performance on benign user tasks.
Ping He, Yuexiang Xie, Yaliang Li et al.· 0 citations
Large language model agents driving security tool suites over the Model Context Protocol are increasingly common. Yet the factors that bound their capability remain poorly characterized: how much depends on the model versus the client that drives it, whether constraining the agent to the orchestrator's own tools helps, and where capability is limited by reasoning rather than by missing tools. Using HexStrikeAI, an open-source orchestrator that exposes 150+ tools, as a testbed, we follow a methodology that evaluates the system, diagnoses its failures, and applies targeted improvements. We run 86 picoCTF challenges across seven categories and three difficulty tiers, under three tool-access regimes and three model/client configurations (774 trials). We then apply corrections to existing tools, agent-behavior changes, and eleven new capability tools, and re-run the previously-unsuccessful trials. The diagnosis isolates the driving client as a first-order factor for a fixed model (a 2.1 * gap between two DeepSeek clients) and a monotonic difficulty gradient, with the largest gains in the mid tier. The overall solve rate rises from 55.4% to 72.0%, and every configuration improves significantly (paired McNemar p<0.001, non-overlapping 95% confidence intervals). The residual failures are reasoning- or environment-bound rather than missing-tool. A 60-run stability sub-study finds single-run verdicts reproducible (17/20 unanimous). We discuss what the results imply for how such orchestrators should be evaluated, and we are explicit about the limits: the study uses a single benchmark, the fixes were tuned on the same challenges they were evaluated on, and the client effect is demonstrated for one model only, so its generality to other models remains a hypothesis.
Romain Gerard, Assmaa Zeghaider, Yan Guo· 0 citations
Testing RESTful API is increasingly complicated but indispensable to quality assurance of cloud-native applications. This paper reports a multi-agent system called MASTEST that combines LLM-based intelligent agents and programmed agents to automate REST API testing. They form a complete tool chain covering the whole workflow of REST API test with API specification in the OpenAPI Swagger format as the input. It also incorporates human testers in the process to review and correct LLM generated test artefacts to control the quality of testing activities. MASTEST is evaluated on two LLMs, GPT-4o and DeepSeek V3.1 Reasoner with five public APIs. Its performances on various testing activities are measured by a wide range of metrics, including adequacy and coverage metrics, the syntax and data type correctness of generated test scripts, the usability of LLM generated test cases and scripts, as well as the bug detection ability. Experiment results demonstrated that both DeepSeek and GPT-4o achieved a high overall performance but had strengths and weaknesses on different testing activities. MASTEST generated test cases achieved 94% and 98% unit test coverage and 79% and 78% system test coverage for GPT-4o and DeepSeek respectively in comparison with human designed test cases. The generated test scripts maintained 100% syntax correctness and only required minimal manual edits for semantic correctness. The generated test scripts contain assertions on the expected status code as well as contents in the response messages. They are highly capable of detecting bugs in the REST APIs. Experiment data shows that the bug detection rates are between 2.13 to 4.50 per operation. These findings indicate that MASTEST is highly efficient and effective.
Xiaoke Han, Hong Zhu· International Conference on...· 1 citation· ⚡1
Large Language Models (LLMs) are increasingly used to generate executable software environments from repository artifacts. However, functional executability does not necessarily imply conformity with architectural, security, workflow, and production intent. This study investigates whether a frontier LLM can generate Dockerfiles and Docker Compose configurations for multi-service applications using repository contents without access to developer-authored deployment artifacts. Three heterogeneous repositories combining Python, Node.js, .NET, React, Rust, Java, Redis, PostgreSQL, and MySQL-compatible infrastructure were evaluated using deterministic end-to-end HTTP oracles and manual structural comparison. All three generated environments became functionally operational, although one required a Rust base-image update from version 1.85 to 1.88. The model correctly reconstructed service topology, application ports, infrastructure dependencies, service hostnames, a background worker, hidden proxy configurations, and a file-based Docker secrets mechanism. However, it consistently omitted network segmentation, multi-stage builds, dependency-layer caching, live-reload volumes, production frontend serving, restrictive backend-port policies, and cross-platform build logic. Based on these observations, the study formalizes the distinction between functional correctness and deployment-intent fidelity and analytically derives a minimal explicit deployment specification for information that cannot be reliably inferred from repository artifacts.
Oleg Grynets, Kyrylo Fursov, V. Lyashkevych et al.· 0 citations
: Integrating Large Language Models (LLMs) into production-grade business systems requires a clear boundary between deterministic application logic and non-deterministic AI behavior. This paper introduces a runtime architecture that extends the BAB (Business Application Builder) framework on Jakarta EE to support agent-augmented applications hosted alongside ontological-driven generated cores. The core of the solution, the AgentRuntimeManager , moves beyond simple API wrappers by providing a robust environment for agent lifecycles, asynchronous event dispatching, and error handling through configurable retry policies. To ensure system stability, we decouple LLM providers (such as OpenAI and Ollama) using an abstraction layer built on LangChain4j, allowing for runtime provider switching without redeploying the core logic. We demonstrate the practical utility of this architecture through a self-healing log-monitoring chain. In this setup, agents intercept high-severity log events and, through a structured analysis service, propose corrective actions. A specialized SqlErrorHealingAgent illustrates how LLM-proposed fixes can be filtered through layered safety constraints, such as the automatic blocking of destructive SQL operations like DROP or TRUNCATE, before any action is committed. The result is an architectural pattern that lets model-driven rigor and LLM agent flexibility coexist in enterprise environments without compromising either.
Borivoj Bogdanović, M. Segedinac, Z. Konjović et al.· SINTEZA· 0 citations
Manual penetration testing needs specialist skills that most student developers and small teams simply don't have time
to build, and the commercial DAST products that automate this work — Burp Suite Professional, Acunetix, Nessus — are priced
for enterprise budgets rather than a college project or a solo developer's side application. Mature open-source alternatives such
as OWASP ZAP close the cost gap but not the usability one: getting useful output still means learning how to configure a scan
policy, spider a target, and read through a long list of raw findings. This paper describes VulnScanner Pro, a self-hosted
assessment tool that tries to sit in the gap between those two extremes. It runs eight checks against a target URL — SQL
injection, cross-site scripting, HTTP security-header analysis, SSL/TLS configuration, CSRF protection, WHOIS, IP
geolocation, and Nmap-based port scanning — behind a FastAPI backend and a Next.js dashboard, storing every scan in
SQLite so past results can be revisited later. A rule-based scoring function turns the combined module output into one 0–100 risk
number and a CRITICAL/HIGH/MEDIUM/LOW label, and a matching recommendation engine looks up remediation text and
reference links for whatever was found Active scanning of a system that you don’t own can be a legal problem as much as a
technical one. So the API by default won’t start a scan of a system unless you explicitly pass in a consent flag and some text
justification for why you are scanning the system. In a case study on a web application owned by the second author, the authors
ran the full eight-module automated security assessment. It took about a minute to run the assessment and it reported a risk
score of 20 out of 100 or LOW. The six checks for missing HTTP security headers were the only issues found. The checks for
SSL/TLS, SQL injection, Cross-Site Scripting (XSS), and Cross-Site Request Forgery (CSRF) all reported that they found no
issues. Only the two expected web ports (80 and 443) were open on the web application. The risk breakdown for the web
application is shown below along with four recommendations for fixing the six missing HTTP security headers. Each of the
recommendations are linked to authoritative external references for additional information on how to fix each of the issues
found
Athili Laxmi Lavanya, Mycharla Madhavkumar, D. Gangadhar· International Journal for Re...· 0 citations