Text-to-image generation: From GANs and Diffusion Models to LLM-Augmented Paradigms

Abstract

. As one of the core cutting-edge fields of AI-generated content, text-to-image generation has brought about dramatic changes to traditional content creation fields. This paper divides the development of text-to-image generation into three stages: Generative Adversarial Networks (GAN), Diffusion Models and the exploration of Large Language Model (LLM) Augmented Paradigms. Based on the current representative technological evolution paths, LLM-enhanced text-to-image generation methods are further summarized into three main paradigms: LLM-based Text Encoder Enhancement, LLM-based Caption & Prompt Refinement, LLM-grounded Layout Planning. Although current LLM enhancement methods still have certain limitations, they have shown great potential in semantic understanding and generative control and are expected to further promote the development of text-to-image technology. This research dissects the progression of text to image generation and outlines three stages for LLM enhancement. It not only offers a clear theoretical framework and practical direction for further research in the domain, but also enriches the understanding of the challenges and opportunities related to the semantic understanding and controllability of generated content, possessing significant academic value and application significance.