LocTFL: Self-Supervised Coarse-to-Fine Audio-Visual Temporal Localization of Local Deepfakes

Abstract

Local audio-visual deepfakes modify only a short word or phrase within an otherwise authentic video. Because the manipulated evidence occupies a small temporal region, global aggregation weakens detection cues, while accurate start/end estimation becomes as important as video-level classification. We present LocTFL, a three-stage framework for local deepfake detection and temporal localization. Stage 1 pretrains a dual-stream masked autoencoder [13] on real talking-face videos to learn natural audio-visual correspondence. A proxy-and-register temporal bottleneck aggregates local evidence and exchanges long-range context without dense attention over all spatiotemporal patches. Stage 2 adapts the pretrained encoders through bidirectional cross-modal fusion and frame-wise classification to produce a coarse forgery probability map. Stage 3 applies a gradient-detached temporal super-resolution branch that upsamples the fused proxy sequence with a lightweight 1D CNN and regresses continuous boundaries using one-dimensional Distance-IoU [38]. On Local-Deepfake-CN [33], LocTFL achieves 0.965 AUC and 0.977 accuracy for video-level detection, together with 0.630 AP@0.5 and 0.355 AP@0.75 for temporal localization. Boundary refinement raises AP@0.5 from 0.462 to 0.630 and AP@0.75 from 0.235 to 0.355. After benchmark-specific fine-tuning, LocTFL obtains 0.730 AP@0.5 and 0.620 AP@0.75 on LAV-DF [4], indicating that the design remains effective beyond the Chinese benchmark. Overall, the results show that learning normal audio-visual dynamics before supervised adaptation and decoupling semantic localization from high-resolution boundary regression are effective for subtle local forgery forensics.