MTGuard is proposed, a hybrid analysis-based defense framework designed to safeguard the use of MCP tools in LLM agents by leveraging lifecycle-aware static-dynamic co-analysis and effectively mitigates multiple categories of harmful tool use across different LLM agents while maintaining performance on benign user tasks.
Abstract
The rapid development of large language model (LLM) agents has enabled their broad adoption across diverse real-world tasks. To standardize interactions between LLM agents and external environments, Model Context Protocol (MCP) tools have emerged as a de facto standard and have been widely integrated into these systems. However, the use of MCP tools also introduces new safety risks, as LLM agents can be induced to perform malicious or unauthorized actions. Although prior work has proposed defenses for securing tool use in LLM agents, most methods rely on static analysis, i.e., inspecting prompts and generated outputs, which limits the defense effectiveness and robustness. To address these limitations, we propose MTGuard, a hybrid analysis-based defense framework designed to safeguard the use of MCP tools in LLM agents by leveraging lifecycle-aware static-dynamic co-analysis. Extensive evaluation demonstrates that MTGuard effectively mitigates multiple categories of harmful tool use across different LLM agents while maintaining performance on benign user tasks.
The Model Context Protocol (MCP) is an open source JSON-RPC protocol that standardizes how large language models (LLMs) interact with external systems through programmatic functions known as tools. Attackers or malicious agents can exploit certain modalities of these MCP tools to degrade the overall quality of service of agent-based applications. For example, an agent may request an excessively large search radius or very long videos, overloading backend systems and potentially causing slowdowns or denial-of-service. Each modality including text, images, video, and location introduces distinct vectors for resource abuse, complicating the development of consistent mitigation strategies. Moreover, multimodal and crossdomain tools expose diverse request schemas and parameters, making it difficult to define policies that are both generalizable and precise enough to enforce meaningful resource constraints. In this paper, we present AEGIS, a policy enforcement component that enables administrators to define fine-grained safeguards against resource abuse across heterogeneous MCP tools and modalities. AEGIS leverages the reasoning capabilities of large language models to analyze, categorize, and normalize diverse tool invocations into a unified, policy-friendly representation accessible to security practitioners. Integrated with the Open Policy Agent and the ContextForge AI Gateway, AEGIS detects and mitigates abusive behaviors while preserving the flexibility of MCP-based agent ecosystems.
The rapid integration of Large Language Models into autonomous agentic systems has introduced unprecedented capabilities for task automation, reasoning, and tool utilization. However, equipping these agents with the ability to access external APIs, internal databases, and arbitrary web resources significantly broadens the attack surface for data exfiltration. Malicious actors can leverage prompt injection and indirect prompt injection techniques to hijack the agent control flow, forcing it to retrieve sensitive information and transmit it to attacker-controlled destinations. Existing security mechanisms, which predominantly rely on static prompt filtering or heuristic anomaly detection, fail to maintain context across complex, multi-step tool invocations. This paper introduces a comprehensive framework based on Tool-Flow Taint Analysis designed to mitigate data exfiltration in Large Language Model agents. By adapting classical dynamic taint analysis principles to the semantic and operational workflows of agent toolchains, the proposed architecture tracks sensitive data from predefined sources through intermediate natural language reasoning steps to potential external sinks. We establish robust semantic taint propagation rules capable of operating within the non-deterministic environment of Large Language Models. Extensive experimental evaluations demonstrate that the proposed system achieves high efficacy in intercepting exfiltration attempts while maintaining a low false-positive rate and negligible latency overhead. The findings provide a critical foundation for securing next-generation autonomous agents against sophisticated data-stealing attacks in enterprise environments.
Chun Tian, Hiu-Tung Li, Michelle Yu· Journal of innovative resear...· 0 citations
Large Language Models (LLMs) are now deployed at an unprecedented scale across many critical sectors, rapidly transitioning from experimental AI tools to embedded components of production software systems. This accelerated adoption, often enabled by low-code integrations, has lowered technical barriers while simultaneously expanding the attack surface of modern applications, particularly when deployments occur without sufficient domain-specific security expertise. In many cases, security maturity has not progressed at the same pace as capability expansion, creating systemic exposure across confidentiality, integrity, and availability dimensions. To provide structured clarity amid this rapid growth, this paper presents a comparative and standards-aligned analysis of LLM security risks and defense mechanisms grounded in the OWASP GenAI Top-10 (2025). We systematically examine each vulnerability class, map representative attack patterns to primary mitigation strategies, evaluate their security property impact, and analyze practical limitations and implementation trade-offs. In addition, we introduce a severity-based assessment to prioritize risks according to operational and systemic impact, offering a quantitative perspective on defensive readiness. Our findings indicate that current mitigation strategies are predominantly reactive, concentrated at inference time, and unevenly distributed across the LLM lifecycle. Controls addressing training pipelines, supplychain dependencies, and autonomous system behaviors remain comparatively less mature and less standardized. By integrating vulnerability classification, defense mapping, severity prioritization, and trade-off analysis within a unified framework, this study provides actionable guidance for strengthening secure, resilient, and standards-driven LLM deployment in high-stakes environments.
Md Abdul Barek, Md Bajlur Rashid, A. K. I. Riad et al.· Annual International Compute...· 0 citations
The model context protocol (MCP) has rapidly emerged as a standard framework for integrating large language models (LLMs) with external tools and resources. However, its metadata-driven and non-deterministic tool selection mechanism introduces a previously overlooked security threat. Leveraging this weakness, we uncover the confused deputy attack, where an adversarial server with subtly manipulated metadata covertly overshadows a benign one, intercepting tool invocations without exhibiting overtly malicious behavior. To systematically study this threat, we develop Puppet, the first automated security evaluation framework that: (i) enriches benign tool descriptions through selective requirement engineering to maximize semantic expressiveness, (ii) restructures them into LLM-preferred formats using description schema transformation, and (iii) applies name prioritization to introduce complementary lexical bias. Furthermore, Puppet synthesizes valid user queries to enable systematic attack evaluation. We comprehensively evaluate Puppet across 14 models from 6 providers on 2 MCP hosts, demonstrating tool selection hijacking rates up to 90.89% and end-to-end malicious payload execution rates up to 86.46%, while remaining undetectable by representative security scanners (MCP-Scan and McpSafetyScanner), which are architecturally incapable of detecting metadata-level manipulation attacks. Counterintuitively, we find that reasoning-enabled models are significantly more vulnerable than their non-reasoning counterparts. Our findings expose a critical design-level attack surface in the MCP ecosystem and highlight the urgent need for principled security safeguards.
Zhiyuan Li, Jingzheng Wu, Yuhao Peng et al.· ACM Transactions on Software...· 0 citations
Large language model (LLM) agents integrated with external tools are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies largely rely on manually implemented or reused environments, stochastic LLM-based tool simulation, and predefined injection locations, limiting scalable security research across broader domains. To bridge this gap, we propose **ToolHazard**, a scalable adversarial environment synthesis framework that reduces human engineering and supports expansion with additional seed domains and compute. Through an Environment Simulator, an Attacker Agent, and a User Simulator, ToolHazard synthesizes executable stateful environments, discovers viable injection points and generates environment-specific payloads, and constructs state-grounded long-horizon tasks. Based on ToolHazard, we build **ToolHazard-Bench** for stress-testing agents under complex workflows and diverse environmental attacks. Experiments reveal substantial agent vulnerabilities and show that injection timing and placement affect attack effectiveness. Moreover, ToolHazard-generated alignment data improves security on both ToolHazard-Bench and AgentDojo while preserving benign task utility.
Yutao Mou, Pengfei Yang, Zhenfei Yin et al.· 0 citations
The widespread adoption of Infrastructure-as-Code (IaC) has made cloud misconfiguration a critical security concern, while Large Language Models (LLMs) and Small Language Models (SLMs) have been redefining the programming process. We present an empirical benchmark evaluating whether LLMs and SLMs can generate security-compliant AWS Terraform configurations. Our automated pipeline integrates Checkov and Trivy into a GitLab CI/CD workflow across four Amazon S3 scenarios, evaluating six models under three prompt strategies of increasing security specificity. Security compliance improves consistently with prompt detail, though no model achieves full compliance in any configuration. Our findings suggest that prompt design is a critical factor, highlighting the need for a proper pipeline for developing and validating LLM-assisted secure IaC generation. All artifacts are publicly available.
Francis Luis Santos Vargas, R. Mansilha, Diego Kreutz· Anais do I Simpósio de Infra...· 0 citations