Haoyu Zhang

Unread or Unenforced? Separating Representation from Enforcement Failure in Content Guards

When an encoded attack passes a content guard, the guard either never represented the payload's harmful content or represented it and failed to act. End-to-end attack success rate reports one number for both, yet the two have opposite remedies: one is a representational limit that more safety training cannot reach, the...

The Uncontrolled Variable: Vision–Language Model Refusal Responds to Image Presence in Ways Risk Cannot Explain ∗

Depth, Not Breadth: Best-of-N Jailbreaking Beyond Surface Noise

Best-of-N jailbreaking spends a query budget on surface variation, scrambling and recasing a request until one draw lands. We ask what a budget buys when its variance is moved into a structural channel instead, holding the search identical across both arms so the encoding is the only difference. Against SAGE, the stron...

Refusing Everything Looks Safe: Restoring the Benign Arm to Encoded-Prompt Evaluation

Encoded-prompt attacks are evaluated almost entirely on their harmful arm: a benchmark sends obfuscated harmful requests and reports how often the model complied, and shows that on two of the four models the loss is caused by the protocol rather than by the character transformation, and on a third by the characters.

The Uncontrolled Variable: Vision-Language Refusal Is Conditioned on the Image-Attachment Interface, and Not Robust to Irrelevant Image Properties

It is shown that aligned vision-language models also condition refusal on a property of a request's form: whether an image is attached, holding everything the request asks fixed, which shifts benign refusal by tens of points.

Decoy Images Amplify Caption-Mediated Defenses Against Encoded Jailbreaks

We report a counter-intuitive interaction between image inputs and existing black-box defenses on Vision--Language Models (VLMs): pairing an encoded jailbreak prompt with an unrelated decoy image can sharply lower attack success rate (ASR). The operative change is in the defense pipeline, not in the image. Across five...

Recover, Decode, Reguard: Guard-Agnostic Defense Amplification againstEncoded VLM Jailbreaks

An empirical safety– utility ceiling for the non-iterative recovery-based defenses the authors evaluate, recurring across every guard and both target VLMs, is exposed.

Attack Ensembles Expose a Safety-Utility Trade-off in Black-Box Guard Defenses Against Encoded VLM Jailbreaks

Safety classifiers ("guards") are the dominant black-box defense for vision-language models, yet a guard judges an input's surface form, not its meaning: a harmful request re-encoded as set theory, formal logic, a classical language, code, or text rendered inside an image slips past a guard that would block it in plain...

Can LLMs Resolve Dependencies? A Benchmark for Semantic-Versioning Constraint Reasoning and Dependency Resolution

A coding agent that installs packages and untangles version conflicts is implicitly reasoning about semantic-versioning constraints and dependency resolution. Whether current language models can actually do this has not been measured, and that is the gap we address. DepResolve-Bench is a programmatically generated benc...