CRAF, a cross-view residual-aware fusion framework that uses ALLM-guided cross-view attention to enrich SSL representations and adopts ALLM as a high-level reference to separate ALLM-explainable information from complementary SSL residual information, demonstrates robustness to unseen spoofing attacks.
Abstract
Recent advances in speech synthesis and voice conversion have made deepfake speech increasingly realistic, making generalization to unseen spoofing attacks a critical challenge. Pretrained speech and audio models offer a promising direction for improving robustness to such unseen attacks. Self-supervised learning (SSL) models capture fine-grained, low-level acoustic characteristics, whereas Auditory Large Language Models (ALLMs) provide higher-level contextual representations. These complementary views can provide useful cues for improving generalization to unseen attacks. However, direct fusion does not explicitly disentangle information shared across the two views from view-specific complementary information, limiting effective cross-view integration. To address this, we propose CRAF, a cross-view residual-aware fusion framework that uses ALLM-guided cross-view attention to enrich SSL representations and adopts ALLM as a high-level reference to separate ALLM-explainable information from complementary SSL residual information. The residual is selectively refined through adaptive gating and integrated through SSL-primary fusion. Experiments on ASVspoof 5 show that CRAF with Kimi-Audio achieves an EER of 5.96% and a minDCF of 0.1192, demonstrating robustness to unseen spoofing attacks.
A novel E-Branchformer-based architecture that effectively leverages self-supervised speech representations for audio deepfake detection and employs parallel branches to simultaneously capture global contextual dependencies through multi-head self-attention and local temporal patterns through convolutional processing.
Phuong Dat, Học Thủ, T. Nguyễn et al.· 0 citations
This paper proposes the Global-Local Adaptive Detector (GLAD), a Hierarchical Global-Local backbone that explicitly bridges the granularity gap by fusing global linguistic and acoustic features with fine-grained local signal details and introduces SaniBoost, a composite data augmentation strategy for robust signal stan...
Ze-Lin Zhao, Guan-Jie Huang, D. H. K. Tsang et al.· arXiv.org· 0 citations
The transition from vocoder-based to neural-codec speech synthesis makes generalization more difficult for speech deepfake detectors, particularly those relying on speech-oriented representations. It remains unclear which acoustic representations retain discriminative information when the generation mechanism changes....
This work proposes a novel domain-adaptive dual-gating MoE framework for SDD under unseen attack types and acoustic conditions that significantly outperforms the baseline and demonstrates superior generalization capabilities and efficient design.
Si-Qing Qin, Zhe Li, K. Lee et al.· Interspeech· 1 citation
Unified all-type audio deepfake detection aims to determine whether an input clip is real or fake when its audio type may be speech, environmental sound, singing voice, or music. Existing speech-centric or type-dependent solutions are insufficient for this setting because the test-time audio type is unknown, while the...
Cun-Hang Fan, Jun-Qin Cao, Tian Gao et al.· 1 citation
EMBNet is proposed, a task-oriented deepfake speech detection framework that integrates efficient channel attention (ECA) with a multi-scale bottleneck to enhance the representation of subtle acoustic anomalies and may support forensic audio authenticity assessment by improving the discrimination between genuine and ma...
Haitao Yang, Fen Li, Xin Cai et al.· Journal of Forensic Sciences· 0 citations