Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution
Modern web agents built on large vision-language models process webpages, select relevant UI elements, and translate model outputs into browser actions. Existing visual red-teaming approaches use adversarial visual content to manipulate this process. However, they primarily target model inference and do not explicitly...
Wan-Jing Han, Levi Taiji Li, Mu Zhang et al.
· 0 citations