AutoDojo is introduced, a generative benchmark built on AgentDojo and AgentDyn that generates IPI adaptively for a given agent and defense, and it is demonstrated that standard static benchmarks often significantly overestimate defense efficacy.
Xinhang Ma, Taoran Li, Chao-Wei Xiao et al.· 5 citations
ROPE is presented, which is anchored in a structural notion of trust, and holds attack success rate to 1.6--2.6 while retaining 82--100% of undefended clean utility, significantly exceeding state-of-the-art system-level defenses in utility while attaining comparable or better security.
Xinhang Ma, Chao-Wei Xiao, William Yeoh et al.· 3 citations· ⚡2
The proposed approach for malicious sentence classification that is both context- and query-aware and outperforms state-of-the-art IPI defense baselines under static attacks, while in the case of adaptive attacks, the AT variants provide significantly higher utility, lower attack success rate, and often both.