2026· International Journal of Scientific Research and Management· 1 citation
TL;DR
A previously unstated class of adversarial input called a clean attack - an input that is syntactically correct, semantically consistent with the declared task context, consistent with all observable policy constraints and still has the goal of misguiding the agent away from the original operator goal - is identified and formalized in this paper.
Abstract
AI agents are being used more in high-pressure situations like managing email, running code, engaging with financial APIs, and supervising multi-agent pipelines. However, current taxonomy of adversarial attacks was mostly proposed for classifiers and generative models alone and fails to adequately describe the testbed of an agent with persistent state, multiple tools, and delegated power. A previously unstated class of adversarial input called a clean attack - an input that is syntactically correct, semantically consistent with the declared task context, consistent with all observable policy constraints, similar to legitimate operator instructions and still has the goal of misguiding the agent away from the original operator goal - is identified and formalized in this paper. These attacks go around the exposed dots of the “traditional” agent security architecture that only filters on the surface. The paper has three main contributions. One, it brings in a formal definition of the clean attack as a four-tuple of input, intent vector, policy envelope and behavioral outcome. Second, it suggests two operationalizable metrics: semantic validity score (SVS) and behavioral drift index (BDI) for systematically measuring the severity of clean attack. Third, the paper these metrics and taxonomy are validated, both by a purpose-built benchmark, AegisBench, and by 300 attack scenarios in three agent classes and (twelve) commercial agent pipelines. The experimental results show that clean attacks are a safety threat of a different category: while the conventional adversarial tasks are practically impervious to these attacks (4.2% success rate of the strongest agents), they achieve a mean attack success of 61.4%.
A formal framework for security in agentic AI is developed, built around an ideal predicate ϕ that decides, for a given ( prompt, context, result ) triple, whether the agent’s behaviour is acceptable.
F. Durak, Tadayoshi Kohno, Franziska Roesner· 0 citations
It is argued that adversarial vulnerability stems from the absence of boundary verification, a security primitive that enforces explicit validation of data as it crosses inter-agent boundaries, including content, identity, execution intent, and state integrity.
Faisal Haque Bappy, Tahrim Hossain, T. S. Zaman et al.· 0 citations
RedAgentBench is introduced, an executable framework for autonomous red-teaming and faithful measurement that shows that executable evaluation can improve safety measurement and identify actionable intervention points.
Zixing Chen, Xingyuan Liu, Jie Zhu et al.· 3 citations
Tool-using AI agents can invoke internal APIs, retrieve documents, update records, and coordinate enterprise workflows. These capabilities create a runtime security problem: an agent may select an unauthorized tool, hallucinate an endpoint, follow malicious instructions embedded in retrieved context, rely on poisoned m...
Swapneswar Ray· International Journal of Glo...· 0 citations
A four-layer taxonomy mapping 13 vulnerability types across perception, brain, action, and interaction layers is contributed, and seven open problems centered on containment are identified.
Md Jafrin Hossain, Mohammad Arif Hossain, Nirwan Ansari· 0 citations
A defensive architecture for under unbounded attack techniques and unenumerable attack objectives, where the highest-value security investments are those that hold regardless of technique, and relocates residual reactivity to a single measurable point.
Ajayi Abisoye, N. Hussain, Abolaji Adebayo· International Journal of Eng...· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduJul 13, 2026
StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduOct 1, 2026
The “HardFlow” algorithm could help generative AI models produce high-quality outputs that obey strict requirements when “pretty close” doesn’t cut it.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.