Skip to content

Adversarial Attack and Detection in Token‐Pruned Large Vision‐Language Models

Sep 2026 · Concurrency and Computation · Vol 38 · 0 citations · 37 references

TL;DR

An attention‐flip attack that manipulates pruning‐related attention redistribution by suppressing high‐importance visual tokens while promoting low‐importance regions, thereby increasing the likelihood that discriminative visual evidence is discarded during inference is proposed.

Abstract

Large vision‐language models (LVLMs) incur high computational costs, and visual token pruning is commonly used to remove redundant tokens and enhance efficiency. However, from a security perspective, such acceleration mechanisms may have a non‐monotonic impact on adversarial robustness: They may either improve robustness by removing perturbation‐related evidence or reduce robustness by discarding semantically important visual tokens. In this paper, we first analyze the setting‐dependent robustness effects of dynamically pruned LVLMs. Building on this analysis, we propose an attention‐flip attack that manipulates pruning‐related attention redistribution by suppressing high‐importance visual tokens while promoting low‐importance regions, thereby increasing the likelihood that discriminative visual evidence is discarded during inference. To counter this threat, we further propose a micro‐feature‐based adversarial detector. Rather than relying on global attention patterns, the detector captures local distortions around the pruning boundary using a multidimensional feature representation, including threshold‐neighborhood density, truncated residual energy ratio, mask spatial dispersion, and statistical descriptors. Experimental results show that, under the same perturbation budget and iterative setting, the proposed attack consistently achieves higher attack success rates (ASRs) than standard first‐order attack baselines on pruned Qwen2‐VL and LLaVA‐OneVision models. For adversarial examples generated by the proposed Attention‐Flip attack, the detector achieves accuracies of 94.79% on Qwen2‐VL and 93.58% on LLaVA‐OneVision. It further reaches 90.37% on standard PGD examples generated from an unpruned Qwen2‐VL model.

View source

Similar papers

Oct 2026

Flexible Targeted Adversarial Alignment with Frequency-based Visual Focus for Attacking Large Vision-Language Models.

Although Large Vision-Language Models (LVLMs) have shown great capabilities across various downstream tasks, they are proven to be vulnerable to adversarial examples, thus resulting in critical safety issues. Existing adversarial attacks against LVLMs predominantly rely on uniform, image-wise perturbations, while large...

Dai-Zong Liu, Tian-Yao Luo, Junhao Dong et al. · 0 citations
Conference Open access Sep 2026

Understanding and Exploiting Phase Sensitivity for Attacking Large Vision–Language Models

This paper proposes a novel LVLM attack method, called BadPhase with further backdoor designs, to implant adversarial phase as triggers into any image inputs via data poisoning so as to control the LVLMs’ predictions and finds that LVLMs are sensitive to the phase-aware image structure.

Dai-Zong Liu, Junhao Dong, Xiang Fang et al. · 0 citations
Open access Sep 2026

EGP-Defense: Enhancing Adversarial Robustness of LVLMs via Training-Free Edge-Guided Prompting

A novel, training-free adversarial defense method called Edge-Guided Prompt Defense (EGP-Defense), which performs adversarial defense during the model inference stage, and significantly improves the robustness of LVLMs against three types of adversarial attacks in both image classification and image caption tasks.

Bo-Yu Wang, Zi-Wen He, Xin-Jue Hu et al. · 0 citations
Open access Aug 2026

Sparse Adversarial Patch Attack and Robustness Evaluation Algorithm for Vision-Language Models

A novel sparse adversarial patch attack algorithm (Sparse Patch Attack, SPA), which successfully misleads the text generation results of VLMs by generating highly dispersed discrete pixel perturbations in non-salient regions of the image.

T.-Y. Chen, X.-Y. Hu, J.-F. Wang et al. · 0 citations
Sep 2026

Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models

This work proposes Feature-Aware Token Attack (FATA), which couples attention suppression with cosine-based feature preservation on a fixed set of salient clean-image tokens and has the lowest measured detection rate among four attacks across three evaluated detectors at a 5% false-positive rate.

Shilinlu Yan, Bo-Wen Chen, Yuechen Zhang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.