The Uncontrolled Variable: Vision–Language Model Refusal Responds to Image Presence in Ways Risk Cannot Explain ∗
Haoyu Zhang, Yi Feng, Shi-Bo Zheng et al.
· 0 citations
3 papers indexed here
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
The Uncontrolled Variable: Vision-Language Refusal Is Conditioned on the Image-Attachment Interface, and Not Robust to Irrelevant Image Properties
We show that aligned vision-language models also condition refusal on a property of a request's form: whether an image is attached, holding everything the request asks fixed. Attaching a blank canvas, an image that cannot be read, cannot relate to the request, and is byte-identical across every prompt in its condition,...
Haoyu Zhang, Yi Feng, Han-Wen Liu et al.
· 0 citations
Preprint
Jul 2026
Attack Ensembles Expose a Safety-Utility Trade-off in Black-Box Guard Defenses Against Encoded VLM Jailbreaks
Safety classifiers ("guards") are the dominant black-box defense for vision-language models, yet a guard judges an input's surface form, not its meaning: a harmful request re-encoded as set theory, formal logic, a classical language, code, or text rendered inside an image slips past a guard that would block it in plain...
Haoyu Zhang, Zhuo-Xiang Wang, Shi-Bo Zheng et al.
· 0 citations