Skip to content
Log in
Preprint

Diff-ID: Identity Consistent Facial Image Generation and Morphing via Diffusion Models

T. Rizwan Sara Atito Muhammad Awais Zhenhua Feng Josef Kittler
Jul 2026 · 0 citations · 38 references
Computer Science

Abstract

Generative diffusion models have revolutionized facial image synthesis, yet robust identity preservation in high resolution outputs remains a critical challenge. This issue is especially vital for security systems, biometric authentication, and privacy sensitive applications, where any drift in identity integrity can undermine trust and functionality. We introduce Diff-ID, a diffusion based framework that enforces identity consistency while delivering photorealistic quality. Central to our approach is a custom 210K image dataset synthesized from CelebA-HQ, FFHQ, and LAION-Face and captioned via a fine tuned BLIP model to bolster identity awareness during training. Diff-ID integrates ArcFace and CLIP embeddings through a dual cross attention adapter within a fine tuned Stable Diffusion UNet. To further reinforce identity fidelity, we propose a pseudo discriminator loss based on ArcFace cosine similarity with exponential timestep weighting. Experiments on held out and unseen faces show that Diff-ID does not exceed InstantID in raw ArcFace Face Similarity, but achieves substantially lower FID and the strongest FIQ based identity--realism trade off among the evaluated methods. We also present a unified DDIM based morphing pipeline that enables qualitative facial interpolation without per identity fine tuning. We further argue that identity preservation and photorealism should be evaluated jointly rather than in isolation, as high identity similarity alone does not guarantee realistic outputs. To make this trade off explicit, we report Face Image Quality (FIQ) as a complementary ratio based score that combines identity similarity and perceptual realism while keeping FS and FID as the primary metrics.

arXiv Semantic Scholar
View source

Similar papers

Preprint Jul 2026

DiffAttack: Evasion Attacks Against Face Recognition via Latent Diffusion Models

The proposed DiffAttack framework significantly outperforms existing adversarial techniques, achieving a high average attack success rate of 84.86% across multiple face recognition models (e.g., FaceNet).

Omid Ahmadieh, Nima Karimian · 0 citations
Preprint Jul 2026

MorphUNet: Alpha-Controlled Biometric Transport for Diffusion-Based Face Morphing Attacks

MorphUNet is the first diffusion-based morphing framework using trainable parent-separated dual cross-attention inside the denoising U-Net: a Biometric Transport Layer carrying parent-specific identity evidence through denoising, attending to each parent separately before combining residuals via the morphing parameter alpha.

T. Rizwan, Sara Atito, Zhenhua Feng et al. · 0 citations
Preprint Aug 2026

When Diffusion Models Forget Who You Are: Identity Preservation in Face Inpainting under Large Occlusions

Face inpainting with diffusion models has recently achieved impressive visual quality, yet preserving identity fidelity under significant occlusion and conflicting text guidance remains a major challenge. To address this issue, we present Reference Semantic Inpainting for Face (ReSem-Face), a cascaded diffusion framework that introduces an explicit identity-conditioned semantic prior for multi-reference face inpainting. Our approach distills representative identity features from multiple references to reconstruct missing semantic regions, which then guide the diffusion process through a multi-stream conditioning architecture. This design provides strong semantic constraints when pixels are absent and stabilizes identity reconstruction while remaining compatible with prompt-driven edits. Experiments on CelebAHQ-IDI-5 and VGGFace2 demonstrate that ReSem-Face yields more reliable identity-preserving completion under severe semantic masks and improves text-controlled editing quality compared with representative baselines.

Feng Ding, Shuhuai Xie, Yue Zhou et al. · 0 citations
Jul 2026

Sketch-text-driven rectified flow for identity-preserving 4D face generation

Sketch-text-driven rectified flow (STDRF), a conditional rectified-flow framework for identity-preserving and semantically controllable 4D face generation, and a sketch encoder enhanced by Geometric Contour and Texture Detail preprocessing and MixStyle domain adaptation are proposed.

Baodong Wang, Fang Liu, Wei Cao et al. · 0 citations
Preprint Jul 2026

Emergent Region-Level Facial Correspondence in Frozen Vision Foundation Models

These results establish frozen DINOv3 as a strong zero-shot representation for region-level facial correspondence and identify intermediate self-supervised features as the most useful layer for dense face analysis.

Izaldein Al-Zyoud, A. E. Saddik · 0 citations