Realistic Facial Attribute Control via Variational Autoencoding

Abstract

1 Abstract —This paper presents the design and implementation of a generative system based on a variational autoencoder (VAE) for reconstructing and manipulating facial features in images. Positioned within the field of artificial intelligence, the work explores the latent space of a convolutional VAE trained on the CelebA dataset, which includes over 200,000 images labeled with 40 binary facial attributes. The pipeline involves preprocessing, geometric alignment using Dlib and OpenCV, encoder-decoder design, and GPU-accelerated training. A key innovation is the composite loss function, combining Binary Cross-Entropy, perceptual loss from VGG19 activations, and Kullback-Leibler divergence. This improves reconstruction fidelity by preserving local detail, global structure, and latent distribution regularity. To further enhance control, the paper introduces a selective orthogonalization of latent vectors using the Gram-Schmidt process, informed by correlation matrix analysis. This reduces attribute entanglement and enables clearer, more independent manipulation of facial features. The results show high-quality reconstructions and consistent semantic edits, highlighting the model’s effectiveness in deepfake-inspired applications. The system provides a foundation for further research in visual content generation, while also addressing controllability and interpretability within generative models.