This survey deeply explains the basic principles of representation learning, and introduces its practical application cases in various fields, and points out the main limitations of current models and prospects the future research directions.
Abstract
Representation learning has become a cornerstone of artificial intelligence, designed to automatically extract low‐dimensional, meaningful features from high‐dimensional, sparse raw data. By drastically reducing the reliance on manual feature engineering, representation learning enhances model performance across a wide range of tasks. The field has evolved significantly over the past decades, transitioning from early linear methods, such as Principal Component Analysis (PCA), to modern deep learning paradigms powered by neural networks, generative adversarial networks (GANs), and pre‐trained models. Although the rapid development of representation learning has significantly promoted the progress of natural language processing (NLP), computer vision, and recommender systems, the general practitioners still have a poor understanding of its historical background, core principles, and wide range of applications. To some extent, this limits the full development of its potential. To this end, this survey aims to provide a comprehensive and easily understandable overview for a wider audience. This survey conducts a systematic literature review to tease out the evolution of representation learning and analyse its core drivers. At the same time, this survey deeply explains the basic principles of representation learning, and introduces its practical application cases in various fields. This survey also points out the main limitations of current models and prospects the future research directions.
An in-depth and up- to-date overview of the GANs environment, principally highlighting the progress made over 2020 and beyond and proposing the idea of hybrid generative systems in the future while emphasizing the oppositional approach's extraordinary and enduring features.
Zahraa Salah Dhaif, Hind Jumaa Serteep· International Journal of Adv...· 0 citations
Image processing has become one of the most consequential frontiers of artificial intelligence (AI). The convergence of deep learning [1], large-scale data, and accelerated computing has transformed how machines perceive, interpret, and reconstruct visual information, enabling capabilities that were unattainable with classical, hand-engineered pipelines. Convolutional neural networks first demonstrated that hierarchical feature learning could surpass manually designed descriptors on large-scale recognition tasks [2], deep residual learning subsequently enabled the stable optimization of substantially deeper networks [3], and the emergence of attention-based and transformer architectures has further reshaped the field by modeling long-range spatial dependencies that earlier models struggled to capture [4,5]. In parallel, generative modeling—propelled by generative adversarial networks (GANs) [6] and, more recently, diffusion-based frameworks [7]—has redefined the boundaries of image synthesis, restoration, and cross-modal translation. Despite this rapid progress, translating AI-driven models into dependable, deployable systems remains challenging. Real-world imagery is rarely clean: it is degraded by noise, rain, blur, low illumination, and domain shift; annotated data are often scarce, particularly in medical and remote-sensing settings, despite the success of label-efficient architectures such as U-Net [8] and the recent emergence of promptable segmentation foundation models [9]; and high-stakes applications increasingly demand not only accuracy but also robustness, calibrated uncertainty, computational efficiency, and interpretability
S. Easwaramoorthy· Computer Modeling in Enginee...· 0 citations
In the last decade, we have witnessed the immense development and impact of the computer vision domain and how it has affected various aspects of life. The trigger for this evolution came in September 2012, when AlexNet, a neural network architecture, achieved unprecedented results on the ImageNet Large Scale Visual Recognition Challenge, marking a turning point for deep learning–based visual recognition. This led to significant progress in deep learning over the next few decades, spurring advances in vision tasks including image classification, object detection, segmentation, and generative modeling. This systematic literature review provides a chronological analysis of the developments that have shaped modern computer vision. It reviews the early days of computer vision and its advances in model architectures from convolutional networks to residual, attention, transformer, and hybrid architectures, and it closely analyzes important design patterns about connectivity and efficiency. It also considers the spectrum of learning strategies, ranging from supervised to self-supervised, weakly supervised, and open-vocabulary learning, as well as transfer and multi-task methodologies. The review further highlights optimization and scheduling methods that facilitate the training of large-scale models, and analyzes how performance commonly goes beyond accuracy to include localization quality and efficiency metrics. In short, this is a critical review of computer vision, illustrating how architectural design, learning paradigms, and evaluation practices have co-evolved over time to facilitate more flexible and scalable systems, and outlining new research directions.
The rapid growth of artificial intelligence and deep learning has revolutionized diverse domains, including computer vision where transfer learning through pre-trained models has become a fundamental technique for enhancing performance of models and reducing computational costs. Although numerous pre-trained deep learning models have been widely adopted, their effectiveness varies significantly across application domains and datasets. The lack of comprehensive comparative studies makes it challenging for researchers to identify the most suitable models for achieving optimal performance in specific tasks. This study systematically compares seven pre-trained feature extractors across three architectural families, convolutional neural networks (CNNs), Vision Transformers (ViTs), and self-supervised models to provide practical guidance on model selection for downstream deep learning tasks. These models were evaluated on five benchmark datasets. Features were extracted with frozen weights and evaluated using linear probing, k-nearest neighbor retrieval, and 5-shot classification. ConvNeXt-B achieved the highest mean linear probe accuracy (90.40%), while DINOv2-S produced the best feature geometry for retrieval tasks (87.64%). CLIP-ViT-B/32 demonstrated the strongest cross-domain transfer, leading on texture recognition and satellite imagery few-shot classification. Older CNN architectures lagged significantly behind modern models by approximately 7%. Overall, this study simultaneously evaluates multiple feature extractors across multiple visual domains and evaluation protocols.
Rafeek Sibrikhan, M. Mufassirin· Sri Lankan Journal of Techno...· 0 citations
In the deep learning era, computer vision has advanced quickly, radically changing how machines interpret and evaluate visual data. Data-driven deep neural networks that can immediately learn complicated representations from images have essentially replaced traditional computer vision algorithms that depended on manually created feature extraction. With an emphasis on important architectures including Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and new hybrid models, this review article offers a thorough overview of contemporary deep learning-based computer vision. It also looks at popular benchmark datasets, such as ImageNet, COCO, and CIFAR, emphasizing how important they are for model building and assessment. The study also examines a variety of practical uses, from autonomous systems and facial recognition technology to object identification and medical picture analysis. Despite impressive advancements, a number of issues still exist, including as high processing demands, reliance on data, interpretability of models, and ethical issues with prejudice and privacy. The report concludes by outlining future research possibilities, with a focus on real-time deployment, multimodal learning, and efficient model construction. The purpose of this review is to offer an organized and perceptive overview of computer vision's present situation and promise in the deep learning age.
Experimental findings show that pretrained models outperform those trained from scratch in terms of accuracy, convergence speed, and robustness, and a unified framework is proposed to integrate both processes in a deep learning pipeline.
Kwame Nkosi· International Journal of App...· 0 citations