This work establishes a high-probability generalization bound for ViTs in classification tasks under adversarial settings, and elucidates the roles of several factors in mitigating perturbation effects, norm regularization of weight matrices and depth-wise propagation constraints on layer-wise norms.
This work proposes a white-box spectral-subspace-guided attack (SSGRA) that aligns intermediate representations with the subspace spanned by the bottom right singular vectors, offering a spectral interpretation of adversarial vulnerability in VLMs, providing insights for improving their robustness.
Chethan Krishnamurthy Ramanaik, Tobias Callies, Michael Hecht et al.· 0 citations
Vision Transformers (ViTs) increasingly rely on input-adaptive inference, such as token pruning and early halting, to meet energy and latency budgets. This survey examines a recent class of adversarial efficiency degradation attacks that target these mechanisms to increase computation without necessarily degrading accuracy. We unify and compare two representative attacks, SlowFormer (a universal adversarial patch) and DeSparsify (per-image perturbations), across three popular token-pruning frameworks: A-ViT, ATS, and AdaViT. We standardize reporting using GFLOPs, accuracy loss, and an Attack Success (AS) metric that measures how much of the model's compute savings the attack takes away. Understanding these attacks is crucial for designing countermeasures that not only mitigate risk but also remain lightweight, since deployment often occurs in low-power settings such as mobile or embedded devices. To organize our analysis, we focus on three questions: how input-adaptive optimizations (e.g., token pruning and early halting) create attack surfaces for efficiency degradation; how such attacks operate in practice and which optimizations are most vulnerable; and which defenses exist today and whether they meaningfully restore efficiency under attack.
Anadi Goyal, Nandish Chattopadhyay, Anupam Chattopadhyay et al.· 0 citations
This study aims to systematically evaluate the adversarial robustness of Convolutional Neural Network (CNN), VGG19, and Vision Transformer (ViT) architectures for chest X-ray classification. Specifically,this study aims to determine the relative influence of model architectures and defense strategies under identical experimental conditions The research method: All three architectures were evaluated using a chest X-ray dataset under white-box attacks using the Fast Gradient Sign Method (FGSM) and the Iterative Fast Gradient Sign Method (IFGSM), with perturbation levels ranging from ε = 0.00 to 0.30. Five defense strategies-Adversarial Training, Adversarial Distillation, TRADES, MART, and Adversarial Weight Perturbation (AWP)-were compared to a baseline without defenses within a unified experimental framework. Differences between defense methods were assessed using the Friedman test. The results showed that without defenses, all models experienced substantial performance degradation, with IFGSM causing a more severe degradation than FGSM. Under FGSM attacks, VGG19 demonstrated the most consistent resilience across defense strategies. Under IFGSM attacks, ViT combined with Adversarial Training achieved the strongest resilience. Defense effectiveness varied across architectures, and no single defense consistently performed best. Friedman’s test identified significant differences between defense methods for FGSM and IFGSM attacks across all architectures (p < 0.001). The study concluded that adversarial resilience is largely determined by the interaction between model architecture and defense strategy. Defense selection has a greater impact on resilience than architecture complexity alone, highlighting the need for architecture-specific defense selection when developing secure medical image classification systems.
Y. Heryadi, I. Sonata, Bambang Krismono Triwijoyo· Matrik· 0 citations
Deep Neural Networks (DNNs) have found successful deployment in numerous vision perception systems. However, their susceptibility to adversarial attacks has prompted concerns regarding their practical applications, specifically in the context of autonomous driving. Existing defenses often suffer from cost inefficiency, rendering their deployment impractical for resource-constrained applications. In this work, we propose an efficient and effective adversarial attack detection scheme leveraging the multi-task perception within a complex vision system. Adversarial perturbations are detected by the inconsistencies between the inference outputs of multiple vision tasks, e.g., object detection and instance segmentation. To this end, we developed a consistency score metric to measure the inconsistency between vision tasks. Next, we designed an approach to select the best model pairs for detecting inconsistencies effectively. Finally, we evaluated our defense against PGD attacks across multiple vision models on the BDD100k validation dataset. The experimental results demonstrated that our defense achieved a ROC-AUC performance of 99.9% detection within the considered attacker model.
Cong Chen, J. Monteuuis, Jonathan Petit· 0 citations
Transfer-based black-box adversarial attacks provide a practical means to evaluate the robustness of deep neural networks under restricted access to target models. However, existing approaches suffer from severe performance degradation in cross-architecture scenarios, particularly when adversarial examples crafted on Vision Transformers (ViTs) are transferred to convolutional neural networks (CNNs). We argue that this limitation cannot be fully attributed to optimization strategies alone, but may also be related to the mismatch between perturbation structure and model-specific inductive biases. To address this issue, we propose a structure-aware adversarial perturbation refinement framework that explicitly enforces spatial coherence during forward propagation. The proposed method consists of three components. First, spatial autocorrelation analysis is used to guide perturbation allocation toward structurally discriminative regions. Second, spatially connected perturbation patterns are introduced to help preserve perturbation effects under convolutional smoothing and pooling operations. Third, the perturbation strength is adaptively adjusted across network depth to balance structural disruption and semantic preservation. Extensive experiments on the ImageNet benchmark show that the proposed approach achieves competitive and often stronger performance than the compared transfer-based attacks across diverse ViT and CNN architectures, with particularly notable gains in the challenging ViT-to-CNN transfer setting. These results provide empirical support for the utility of structure-aware perturbation refinement in improving black-box adversarial transferability across heterogeneous visual architectures.
Qi-Rui Lu, Liansong Zong, Fu-Ran Liu et al.· Neural Networks· 0 citations
FDT-PC (Frequency Domain Transformation with Perceptual Constraints), a novel method that enhances adversarial transferability across different model architectures, is proposed, which achieves superior black-box attack performance on both CNNs and Vision Transformers, outperforming existing state-of-the-art input transformation methods.
Bo Li, Li Tang, Xin Jin et al.· ACM Transactions on Multimed...· 0 citations