Deep Supervised Adversarial Robust Hashing for Retrieval.

Abstract

Deep hashing has emerged as an effective and widely adopted framework for similarity retrieval, owing to its computational efficiency and the powerful feature extraction capabilities of deep learning (DL). Despite their strong performance in multi-modal and high-dimensional data retrieval tasks, recent studies have exposed the susceptibility of DL models to adversarial attacks, including in retrieval scenarios. This underscores the critical need for enhancing robustness in DL models to ensure reliable inference. However, most adversarial robustness studies focus on classification tasks, where explicit labels facilitate supervised adversarial training. In contrast, retrieval tasks typically rely on similarity matrices rather than explicit labels, making direct adversarial optimization challenging and limiting its application in large-scale retrieval settings. To address this gap, we propose Deep Supervised Adversarial Robust Hashing (DSARH), an end-to-end framework that leverages similarity matrices and learnable hash codes to construct gradient-based worst-case perturbations, enabling efficient adversarial training and robust feature learning for retrieval. Extensive experiments on cross-modal and image retrieval tasks demonstrate that existing deep hashing models are highly vulnerable to adversarial perturbations, whereas DSARH achieves superior robust generalization across a wide range of adversarial scenarios. Moreover, the robust visual features learned by DSARH help mitigate modality heterogeneity, resulting in consistent improvements in both standard and adversarial performance across multiple image-text retrieval benchmarks compared to state-of-the-art baselines. These results highlight the critical role of adversarial robustness in developing reliable and effective multi-modal retrieval systems.