Beyond the Verdict: Evidence-Aligned Evaluation of Visual Prompt-Injection Guardrails
Verdict-only evaluation does not reveal whether a vision-language model (VLM) used the visual evidence that should support its decision. We study this problem in web-agent guardrails, where a VLM judges whether on-screen text conflicts with a user instruction. We introduce Mind2Web-Injection, a benchmark of 9,954 instr...
Suyoung Lee, Myungsub Choi
· 0 citations