Skip to content
Book Open access

SAGE: Typographic Adversarial Defense for Large Vision-Language Models via Scene-Aware Guard Engine

Nov 2026 · 0 citations

Abstract

Recently, multimodal typographic adversarial attacks have raised critical safety concerns about Large Vision-Language Models (LVLMs). By embedding semantically coherent but misleading text into images, these attacks reveal severe vulnerabilities of LVLMs. Existing defenses primarily rely on text filtering or input purification, which fail to capture cross-modal inconsistencies and often disrupt visual integrity. To this end, we propose SAGE (Scene-Aware Guard Engine), a novel defense framework that fundamentally shifts from adversarial content removal to reasoning-level semantic control. Specifically, we introduce a scene-aware semantic consistency detector that identifies adversarial text that contradicts visual context under the query. Subsequently, an explicit semantic separation mechanism is employed to isolate misleading textual signals from visual reasoning. Furthermore, we develop a self-reflective optimization module that adaptively refines defense strategies through failure-driven feedback. Extensive experiments across diverse attack methods, models, and benchmarks demonstrate that SAGE significantly improves defense success rates while maintaining competitive performance on benign samples, exhibiting outstanding robustness and generalizability.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.