Aug 2026· ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)· 0 citations· 14 references
TL;DR
This paper proposes an end-to-end Transformer-based framework, termed Progressively Explicit Query Network (PEQNet), for multi-face forgery detection and localization, and introduces triple contrastive learning to model the mutual exclusivity among real, fake, and background regions.
Abstract
Deepfake technology has facilitated the spread of malicious content, creating an urgent need for efficient and accurate face forgery detection techniques. Traditional detection methods usually follow a two-stage paradigm, where faces are first detected and then classified as real or fake. However, such approaches can not simultaneously handle multiple faces, and errors in the first stage directly affect the second stage, leading to error accumulation. In this paper, we propose an end-to-end Transformer-based framework, termed Progressively Explicit Query Network (PEQNet), for multi-face forgery detection and localization. Our method extracts instance-aware activation queries from multi-scale, cross-domain features, and jointly processes them with image features through a progressively explicit query-driven decoding network to predict query-dependent masks and classes. During training, we introduce triple contrastive learning to model the mutual exclusivity among real, fake, and background regions. Additionally, we define specific real-or-fake queries to capture global class-level information, which are iteratively refined by the predicted class-aware masks and optimized by aligning them with the ground-truth. Extensive experiments demonstrate that PEQNet achieves superior performance compared with state-of-the-art methods, confirming its effectiveness for multi-face forgery detection in realistic scenarios.
The rapid increase of deepfakes has raised significant concerns due to their spread on social media. Traditional multi-face forgery detectors crop and verify each face independently, ignoring background context and inter-face relationships, which often yields suboptimal performance. To overcome these limitations, we leverage instruction-based Large Vision-Language Models (LVLMs), which can interpret entire images and follow complex textual instructions. We propose a simple yet effective single-stage multi-face forgery detector, called IMFD (Instruction-based Multi-face Forgery Detector), which is trained end-to-end to jointly localize faces and predict per-face forgery labels. Rather than treating face box prediction only as a joint objective, IMFD explicitly integrates predicted face bounding boxes into the instruction as visual cues that enhance instruction grounding and forgery detection. To support the training and evaluation of IMFD, we convert existing multi-face forgery datasets into an instruction-based format. Experimental results and analyses show that IMFD improves multi-face forgery detection by integrating face bounding boxes into the instruction, and consistently outperforms various state-of-the-art methods.
Dasom Choi, Sang-Jun Moon, Hyeongchan Im et al.· 0 citations
This work explores an approach that integrates wavelet-based frequency analysis with deep learning to enhance deepfake detection, and suggests that wavelet sub-bands expose manipulation cues that are useful for detecting unseen fake classes, but they should not be interpreted as a uniform robustness improvement.
Video forgery has become increasingly prevalent with advanced editing tools, posing serious threats to digital media authenticity. However, existing approaches used for detecting manipulations often faced challenges in handling intra-frame forgeries, decreased generalization, sensitivity to noise, and inadequate selection of key frames, thus leading to a decrease in the overall detection performance. To tackle these issues and to design an effective intra-frame forgery detection system, this research proposes an Optimized Multi-level Mixed Attention-enabled Hybrid Learning-based Bidirectional Gradient (OM2AHL-BiG) Model. The framework leverages the hybrid learning technique to enhance representation of features, Bidirectional Long Short-Term Memory (BiLSTM) for understanding long-range temporal dependencies, and Gradient Boosting Machine (GBM) for precise classification. Additionally, the mixed attention mechanisms allow for refining relevant features, and the Cooperative Search Hunter optimization (CoSH) is designed for hyperparameter tuning in order to improve the detection accuracy. Overall, the OM2AHL-BiG model results in enhanced detection, adaptability to various forgery types, and improved interpretability, making it an effective solution for detecting deepfake and intra-frame video forgeries. The experimental results obtained on the FaceForensics++ dataset showcased an improved performance under 90% of training percentage by yielding an accuracy of 98.62%, precision of 99.2%, sensitivity of 98.04%, specificity of 98.91%, and 98.62% F1 score, surpassing conventional approaches effectively.
A generative detector that localizes tampering by estimating the local restoration cost required to align a query image with authentic visual-text statistics, rather than by learning forgery-specific decision boundaries is proposed, and Sparse-Constraint Rectified Flow is introduced, a detector-oriented adaptation of Flow Matching for spatially sparse anomaly localization.
Jiangling Zhang, Shuxuan Gao, Zeyu Chen et al.· 0 citations
As generated videos become increasingly realistic, reliable video forgery detection is increasingly important. Existing studies typically optimize and use video forgery detectors as black boxes, while the latent forgery-discriminative knowledge inside them remains largely unexplored. Instead of continuing to rely on resource-intensive full-model retraining to steadily improve detection performance, we ask whether video forgery detection can also be achieved by uncovering and activating sparse forensic knowledge within the detector. We find that forgery-discriminative knowledge is not uniformly distributed across the full representation space, but is concentrated in a sparse set of functionally specialized neurons. Based on this insight, we propose a video forgery-intrinsic neuron discovery (V-FIND) framework. V-FIND first localizes critical layers that exhibit pronounced discrepancies between real and forged videos, and then identifies latent anchor neurons that consistently carry forgery-discriminative signals, organizing them into a compact forensic subspace. With the original backbone frozen and only a lightweight linear classifier trained, this subspace still delivers strong detection performance across multiple external benchmarks for generated videos. Further neuron intervention experiments provide direct evidence for the functional specificity of the discovered neurons. Overall, these results suggest that video forgery detectors contain sparse, extractable, and reusable forgery-discriminative knowledge, offering a new perspective on understanding and exploiting their intrinsic forensic capability.
Shichao Kan, Chengpeng Hong, Jingtong Dou et al.· 0 citations
Generalizable face forgery detection has become a critical problem in multimedia forensics as modern face manipulation techniques can generate increasingly realistic facial content. Vision foundation models provide a promising basis for this problem, but existing detectors usually rely on the last-layer visual feature, implicitly assuming that the most transferable forgery evidence is concentrated in the final representation. In a vision transformer, different layers preserve different visual properties, and face manipulation traces may appear as local texture defects, region-level structural conflicts, or high-level semantic inconsistencies. We revisit CLIP from a hierarchical perspective and propose HIAF, a Hierarchical Interaction and Adaptive Fusion framework for face forgery detection. HIAF extracts representations from multiple transformer depths and learns to use them through two dedicated modules. Cross-Layer Feature Interaction (CFI) performs a bottleneck self-attention along the layer dimension, enabling each layer to absorb complementary evidence from other depths and exposing cross-level inconsistencies that are informative for forgery detection. Layer-aware Adaptive Fusion (LAF) then assigns dynamic weights to different layers, adaptively promoting the layers whose forensic cues are most reliable while suppressing less discriminative ones. To further stabilize the layer-wise feature spaces, we introduce a multi-view contrastive objective that treats each layer as an independent view and regularizes real samples into compact manifolds. Extensive experiments show that HIAF achieves an average AUC of 88.93% in cross-dataset evaluation and 96.54% in cross-manipulation evaluation, outperforming recent state-of-the-art methods.
Yi-Meng Zhao, Shuo Zhu, Jia-Lang Liu et al.· 2026 12th International Conf...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
