The proposed DiffAttack framework significantly outperforms existing adversarial techniques, achieving a high average attack success rate of 84.86% across multiple face recognition models (e.g., FaceNet).
Abstract
Facial biometric identification relies on the distinctiveness of user attributes within a high-dimensional embedding space. However, the decision boundaries of deep face recognition (FR) systems are often sufficiently narrow that they can be conflated, rendering the models vulnerable to adversarial attacks. In such scenarios, the FR system fails to distinguish between an authentic source and a meticulously crafted adversarial face. Existing adversarial methods targeting facial biometrics are limited in both performance and their ability to generate high-quality images that are imperceptible to humans. Moreover, these methods often fail when the source and target images belong to different demographic groups or genders. To address these limitations, we present a novel approach for adversarial face generation via latent-space optimization. We leverage latent diffusion models directly to guide generation toward target identity embeddings, as measured by a face recognition model. Our proposed \textbf{DiffAttack} framework has been evaluated on standard benchmarks, such as the FFHQ and CelebA-HQ datasets. DiffAttack significantly outperforms existing adversarial techniques, achieving a high average attack success rate of 84.86% across multiple face recognition models (e.g., FaceNet). Notably, DiffAttack demonstrates superior transferability, surpassing traditional noise-based methods by over 15.28% and semantic-based approaches by approximately 5.21% on benchmark datasets like FFHQ and CelebA-HQ.
Facial recognition (FR) models are vulnerable to adversarial attacks, in which attackers manipulate facial images to expose system vulnerabilities, underscoring the urgent need to improve the transferability of adversarial attacks. However, existing methods fail to fully leverage diverse initialization strategies for extending surrogate models, thereby limiting the transferability of adversarial samples. To address this, we propose the Multi-Initialization Enhanced Aggregation (MEA) attack method. This approach improves transferability by combining different parameter initializations to generate a diversified set of surrogate models. MEA consists of two stages: Multi-Initialization Adversarial Enhancement (MIAE) and Enhanced Adversarial Aggregation (EAA). In the MIAE stage, we enhance model diversity through checkpoint saving driven by diversity metrics and multi-layer initialization. In the EAA stage, we further enhance transferability by adding perturbations to high-level features. Additionally, we integrate an adversarial makeup technique that generates adversarial disguises from reference images, thereby further boosting the attack's effectiveness. Experimental results show that MEA outperforms the second-best input transformation attack by 20.35% and achieves a 9.22% improvement over existing facial adversarial attacks.
Physical-domain adversarial attacks have been extensively studied in face recognition and object detection, yet the field of palmprint recognition remains largely unexplored. Existing methods generate grayscale patches constrained by the single-channel input of most palmprint models. When deployed on skin, these patches contrast sharply with the surrounding tissue and are readily noticeable to human observers, undermining the covertness required in practical attacks. To address this limitation, we propose a Color Adversarial Patch (CAP) generation algorithm that leverages style transfer principles to produce visually natural color patches while maintaining high attack success rates. The method initiates the patch with a style prior using a pre-trained Contrastive Arbitrary Style Transfer (CAST) model and jointly optimizes adversarial loss, style loss, and smoothness loss within a unified framework. A three-channel averaging strategy is adopted to ensure compatibility with single-channel recognition models during gradient backpropagation. Experiments on the Tongji palmprint dataset show that the generated color patches achieve average cosine similarity values above the decision threshold in physical-domain tests, with peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) values significantly higher than those for their grayscale counterparts. Ablation studies validate the indispensable role of each loss component. CAP offers a practical balance between attack effectiveness and visual camouflage, demonstrating the feasibility of concealed physical-domain attacks on palmprint recognition systems.
Yue Liu, Qi Xiong, Lu Leng et al.· Electronics· 0 citations
Generative diffusion models have revolutionized facial image synthesis, yet robust identity preservation in high resolution outputs remains a critical challenge. This issue is especially vital for security systems, biometric authentication, and privacy sensitive applications, where any drift in identity integrity can undermine trust and functionality. We introduce Diff-ID, a diffusion based framework that enforces identity consistency while delivering photorealistic quality. Central to our approach is a custom 210K image dataset synthesized from CelebA-HQ, FFHQ, and LAION-Face and captioned via a fine tuned BLIP model to bolster identity awareness during training. Diff-ID integrates ArcFace and CLIP embeddings through a dual cross attention adapter within a fine tuned Stable Diffusion UNet. To further reinforce identity fidelity, we propose a pseudo discriminator loss based on ArcFace cosine similarity with exponential timestep weighting. Experiments on held out and unseen faces show that Diff-ID does not exceed InstantID in raw ArcFace Face Similarity, but achieves substantially lower FID and the strongest FIQ based identity--realism trade off among the evaluated methods. We also present a unified DDIM based morphing pipeline that enables qualitative facial interpolation without per identity fine tuning. We further argue that identity preservation and photorealism should be evaluated jointly rather than in isolation, as high identity similarity alone does not guarantee realistic outputs. To make this trade off explicit, we report Face Image Quality (FIQ) as a complementary ratio based score that combines identity similarity and perceptual realism while keeping FS and FID as the primary metrics.
T. Rizwan, Sara Atito, Muhammad Awais et al.· 0 citations
This work proposes SRAP, which combines per-channel truncated SVD refinement with an identity-importance mask at every optimization step, and demonstrates that SRAP substantially improves protected-image fidelity across all reported metrics while maintaining competitive identity-disruption performance.
In computer vision and pattern recognition tasks, deep learning models are widely used, especially in face recognition systems. Even with their excellent performance, these models are still susceptible to a variety of adversarial manipulations, such as blur, additive noise, translation, flipping, scaling, rotation, and changes in illumination. Furthermore, some architectures might experience optimization problems like vanishing gradients, which would further impair the stability of the model. In order to provide robust face verification under adversarial attack, Optimized Deep Learning–based Adversarial Defense Mechanism (ODL-ADM) is proposed in this work. It projects adversarial samples into an immune feature space. A Learnable Convolutional Principal Component Network (LCPCN) is incorporated into the framework to create a representation space that is both discriminative and resistant to perturbations. Adversarially corrupted facial images are suppressed and reconstructed using a Stacked Attention-based Residual Generative Adversarial Network (SARGAN). Accurate identity recognition is achieved by an Improved Cross-Triple MobileNetV1 architecture after perturbation removal. Enhanced Fire Hawk Optimization (EFHO) is used for performance maximization and parameter tuning to further improve recognition performance. Following image reconstruction and adversarial perturbation removal, the suggested model achieves a 98% face recognition accuracy.
Sheilla Ann Bangoy Pacheco, Mahesh Goyani, Jayzel P. Bangoy et al.· ITEGAM- Journal of Engineeri...· 0 citations
Self-supervised learning (SSL) models are vulnerable to backdoor attacks. However, the systemic risks they pose in face representation have received little attention. The entanglement of identity features in self-supervised face learning presents unique challenges for attack stealthiness. To address this gap, we propose FIDA (Feature Instability-Driven Attack), a novel backdoor attack framework. FIDA uses subtle semantic triggers for injection, but its key innovation is a novel objective called Feature Instability Loss. It trains the encoder to increase the sensitivity of triggered features along perturbation directions sampled during attack optimization . By preventing the backdoor from exhibiting the rigid feature patterns typical of previous attacks, FIDA effectively evades the evaluated perturbation-based defenses. Experiments show that FIDA achieves a high attack success rate and generally preserves benign utility across the evaluated settings , posing a significant threat to real-world multimedia applications relying on facial analysis.
Zhi-Yang Chen, Changchun Yin, Hui-Qin Yang et al.· 0 citations