Season, a spectrum-aware orthogonal gradient refinement framework for L-infinity transfer attacks against black-box target models on ImageNet, using a white-box surrogate to improve transfer success rate.
Abstract
Transfer-based adversarial attacks often transfer poorly across heterogeneous architectures because CNNs favor local textures while Vision Transformers (ViTs) rely on global shapes. We propose Season, a spectrum-aware orthogonal gradient refinement framework for L-infinity transfer attacks against black-box target models on ImageNet, using a white-box surrogate. Season decomposes each update into a low-frequency branch capturing structural cues and a high-frequency branch capturing textures. A low-saliency guidance scheme reallocates high-frequency energy to background regions, preserving foreground structures that ViTs depend on. An orthogonal projection then forces the textural update to lie in the orthogonal complement of the structural direction, mitigating feature interference. As a training-free plug-and-play wrapper, Season enhances eight gradient-stabilization and input-enhancement attacks without modifying their cores. Across eight CNN, ViT, and MLP targets, Season improves transfer success rate by 6.6 percentage points on average and up to 16.0 points over strong baselines under a unified protocol.
Transfer-based black-box adversarial attacks provide a practical means to evaluate the robustness of deep neural networks under restricted access to target models. However, existing approaches suffer from severe performance degradation in cross-architecture scenarios, particularly when adversarial examples crafted on Vision Transformers (ViTs) are transferred to convolutional neural networks (CNNs). We argue that this limitation cannot be fully attributed to optimization strategies alone, but may also be related to the mismatch between perturbation structure and model-specific inductive biases. To address this issue, we propose a structure-aware adversarial perturbation refinement framework that explicitly enforces spatial coherence during forward propagation. The proposed method consists of three components. First, spatial autocorrelation analysis is used to guide perturbation allocation toward structurally discriminative regions. Second, spatially connected perturbation patterns are introduced to help preserve perturbation effects under convolutional smoothing and pooling operations. Third, the perturbation strength is adaptively adjusted across network depth to balance structural disruption and semantic preservation. Extensive experiments on the ImageNet benchmark show that the proposed approach achieves competitive and often stronger performance than the compared transfer-based attacks across diverse ViT and CNN architectures, with particularly notable gains in the challenging ViT-to-CNN transfer setting. These results provide empirical support for the utility of structure-aware perturbation refinement in improving black-box adversarial transferability across heterogeneous visual architectures.
Qi-Rui Lu, Liansong Zong, Fu-Ran Liu et al.· Neural Networks· 0 citations
FDT-PC (Frequency Domain Transformation with Perceptual Constraints), a novel method that enhances adversarial transferability across different model architectures, is proposed, which achieves superior black-box attack performance on both CNNs and Vision Transformers, outperforming existing state-of-the-art input transformation methods.
Bo Li, Li Tang, Xin Jin et al.· ACM Transactions on Multimed...· 0 citations
Transfer-based black-box attacks are an important tool for evaluating deployed vision models, yet adversarial examples generated from Vision Transformer (ViT) surrogates often exhibit limited cross-architecture transferability. Existing momentum-based attacks are effective for convolutional neural network (CNN) surrogates, but they can accumulate stale directions and overfit the surrogate when the source model is a ViT. This paper presents Ada-MGNS, a ViToriented transferable attack that combines adaptive momentum with deep attention guidance. The adaptive component measures the directional discrepancy between the current guided gradient and the accumulated trajectory, and then attenuates stale momentum when the search direction becomes unstable. The guidance component fuses the classification gradient with an auxiliary gradient extracted from the last transformer block’s attention responses, encouraging perturbations to disturb both output decisions and semantic aggregation. Experiments on ImageNet with four ViT surrogates, thirteen standard black-box targets, and five defense models show that Ada-MGNS consistently improves attack success rates over representative ViT-specific baselines, remains compatible with DI/TI transformations and effective against adversarially trained and purification-based defenses.
Lei Lu, Run-Han Yao, Qinghe Du et al.· 2026 International Conferenc...· 0 citations
Vision Transformers (ViTs) increasingly rely on input-adaptive inference, such as token pruning and early halting, to meet energy and latency budgets. This survey examines a recent class of adversarial efficiency degradation attacks that target these mechanisms to increase computation without necessarily degrading accuracy. We unify and compare two representative attacks, SlowFormer (a universal adversarial patch) and DeSparsify (per-image perturbations), across three popular token-pruning frameworks: A-ViT, ATS, and AdaViT. We standardize reporting using GFLOPs, accuracy loss, and an Attack Success (AS) metric that measures how much of the model's compute savings the attack takes away. Understanding these attacks is crucial for designing countermeasures that not only mitigate risk but also remain lightweight, since deployment often occurs in low-power settings such as mobile or embedded devices. To organize our analysis, we focus on three questions: how input-adaptive optimizations (e.g., token pruning and early halting) create attack surfaces for efficiency degradation; how such attacks operate in practice and which optimizations are most vulnerable; and which defenses exist today and whether they meaningfully restore efficiency under attack.
Anadi Goyal, Nandish Chattopadhyay, Anupam Chattopadhyay et al.· 0 citations
Deep learning has boosted remote sensing (RS) scene classification, but adversarial examples can still cause high-confidence misclassification with imperceptible perturbations. Adversarial purification (AP) offers a practical test-time defense without retraining the classifier. However, most existing methods are confined to pixel-space restoration, which may leave residual adversarial effects that persist and amplify through feature extraction, ultimately biasing the prediction. To address these issues, a dual-domain AP (DDAP) framework is proposed to mitigate adversarial effects at both the pixel and feature levels in a unified pipeline. In the pixel domain, a pixel-domain frequency-aware diffusion purification (PFDP) module performs diffusion-based restoration through a frequency-aware dual-stream U-Net (FD-UNet). By integrating adaptive spectral filtering with multidomain consistency constraints, PFDP reduces adversarial-perturbation-dominated high-frequency responses while preserving structural details and semantic information in RS imagery. In the feature domain, an adversarial vulnerable channel dropout (AVCD) strategy models unshifted shallow-feature statistics with a Gaussian mixture model (GMM) and adaptively assigns channelwise dropout probabilities based on a samplewise shift score and channel vulnerability, thereby suppressing residual adversarial influence before downstream classification. Extensive experiments on UC Merced (UCM) and aerial image dataset (AID) across multiple backbones and attack types demonstrate that DDAP consistently improves robustness while maintaining a favorable clean–robust balance compared with representative baselines.
Yuru Su, Shaohui Mei, Mingyang Ma et al.· IEEE Transactions on Geoscie...· 0 citations
Point cloud perception is important in autonomous driving, robotics, and other security-critical 3D systems, yet learned point cloud classifiers remain vulnerable to transferable adversarial perturbations. A central difficulty in transfer-based black-box attacks is surrogate overfitting: an update that is highly effective on an accessible source model may not generalize to an unknown target architecture. We introduce SpecEOT, a source-agnostic and graph-spectral expectation-over-transformation attack. A fixed graph Fourier transform (GFT) basis is constructed from each clean point cloud. At every optimization iteration, each non-identity view independently samples a frequency band and a perturbation sign from uniform distributions; the resulting view gradients are averaged with equal weights and used to update the adversarial point cloud through projected Adam ascent. We evaluate the stochastic method over repeated seeds, extend the ablation to two source architectures, and analyze the interaction between band count and randomization strength while reporting computational cost and assessing robustness to Gaussian jitter and point dropout. SpecEOT achieves strong transferability on ModelNet40 and ShapeNet.
Yang Gao, Jingyi Liu, Hongjia Liu et al.· Applied Sciences· 0 citations