3D face reconstruction from single 2D human image using deep learning method

Abstract

Three-dimensional (3D) face reconstruction from a single two-dimensional (2D) image enables the creation of personalized 3D avatars that preserve facial identity for immersive applications in the metaverse. This thesis aims to develop a realistic 3D human avatar generation framework for the metaverse platform MANGOs (Metaverse of Academic Nexus for Global Opportunities). The framework employs the Faces Learned with an Articulated Model and Expressions (FLAME) model as the underlying parametric 3D representation of face to represent facial shape, pose, and expression. Inspired by Detailed Expression Capture and Animation (DECA), which reconstructs 3D facial geometry from a single 2D image without requiring explicit 3D ground-truth supervision, this thesis comprehensively investigates the contributions of all the loss functions to 3D face reconstruction through quantitative and qualitative analyses. This work is evaluated on the NOW benchmark using both non-metrical and metrical reconstruction protocols to assess the robustness and accuracy of the reconstruction framework. In addition, this thesis introduces a landmark mirroring strategy to improve the recovery of facial contours in occluded regions. To enhance visual realism of the reconstructed 3D face model, the Basel Face Model (BFM) UV texture map, a mean texture, and a texture projection technique are employed to generate facial skin textures. The reconstructed 3D faces are then integrated with the Skinned Multi-Person Linear (SMPL) body model to create complete 3D human avatars, followed by the attachment of an armature to enable skeletal animation. These avatars are subsequently deployed on the MANGOS platform, demonstrating the real-world applicability of this research.