Jul 2026· Machine Learning and Knowledge Extraction· Vol 8, pp. 199· 0 citations· 81 references
TL;DR
A systematic analysis of the machine learning and deep learning models underpinning vehicle autonomy, spanning classical convolutional neural networks for object detection and semantic segmentation to recurrent and Transformer-based architectures for trajectory prediction and motion planning is presented.
Abstract
Autonomous driving systems rely on a sophisticated pipeline of artificial intelligence models to perceive, predict, and plan in dynamic environments. This review presents a systematic analysis of the machine learning and deep learning models underpinning vehicle autonomy, spanning classical convolutional neural networks (CNNs) for object detection and semantic segmentation to recurrent and Transformer-based architectures for trajectory prediction and motion planning. It also provides a critical examination of the autonomous vehicle sensor stack, including cameras, LiDAR, radar, ultrasonics, and GNSS/IMU as data acquisition systems, highlighting modality-specific AI challenges such as monocular depth estimation, 3D point cloud processing, and radar Doppler interpretation. The evolution of perception and decision-making pipelines is reviewed, contrasting modular architectures with end-to-end learning paradigms that directly map raw sensor data to control commands, and discussing their trade-offs in interpretability, safety assurance, and robustness to rare edge cases. We further survey specialized hardware accelerators and heterogeneous automotive SoCs designed to meet stringent real-time and power constraints. Industrial strategies are compared, including multi-modal sensor fusion and vision-centric approaches based on large-scale imitation learning. Finally, we identify open challenges related to robustness under adverse conditions, domain shift, causal ambiguity, and the need for interpretable and certifiable AI in safety-critical autonomous driving systems.
Sensor fusion plays a critical role in enabling reliable perception for autonomous driving systems by integrating heterogeneous data from multiple sensors such as cameras, LiDAR, radar, and inertial units. However, inconsistencies in spatial alignment, temporal synchronisation, and data representation present significant challenges to achieving a unified understanding of the driving environment. This paper proposes a conceptual framework that interprets sensor fusion as a reconstructive process, transforming diverse sensory inputs into a coherent environmental model. The study systematically analyses three levels of fusion: data-level, feature-level, and decision-level, and examines how artificial intelligence enhances each stage through learned alignment, cross-modal feature representation, and uncertainty-aware decision making. Furthermore, the paper connects fusion strategies to key autonomous driving tasks, including object detection, tracking, localisation, and planning, highlighting the relationship between task requirements and fusion architecture design. Finally, major challenges such as domain shift, long-tail scenarios, sensor failure, and interpretability are discussed. This work provides a structured perspective on AI-driven sensor fusion and its role in building robust and adaptive intelligent driving systems.
This paper presents a comprehensive survey on deep learning based multi-modal perception fusion frameworks for autonomous driving systems. The traditional single-modal solutions have intrinsic limitations in robustness against harsh weather and dynamic environments, and the integration of complementary sensors, such as cameras, LiDAR, and millimeter-wave radar, provides a critical pathway to high-level automation. We provide a systematic review of the basic fusion paradigms, classifying them into early, late and deep (feature-level) architectures, and evaluating their trade-offs in terms of computational overhead, architectural modularity and joint feature learning. Special focus is given to state-of-the-art architectures, comparing the high representational power of the Tensor Fusion Networks (TFN) to the computationally-efficient Multi-modal Circulant Fusion (MCF) and context-aware Transformer frameworks. This study places real-world deployment challenges in the context of algorithmic paradigms, investigating the friction between “black-box” deep learning models and stringent functions. Besides algorithmic paradigms, this work puts real-world deployment challenges into context and discusses the friction between ``black-box'' deep learning models and rigorous functional safety standards (e.g., ISO 26262), and energy constraints of new energy vehicles. Finally, we explore the paradigm shift to "vehicle-road synergy" (V2X) infrastructure as a critical mechanism for providing the safety redundancy and edge-computing capabilities needed for fully reliable, next-generation autonomous driving.
Xichen Huang· Applied and Computational En...· 0 citations
Autonomous driving has become a transformative technology poised to reshape modern transportation systems. This paper explores multimodal fusion techniques that integrate various sensors with machine vision for autonomous driving. We examine the integration of various sensor modalities, including cameras, LiDAR, and millimeter-wave radar, alongside advanced machine vision algorithms such as YOLO, Faster R-CNN, Point Pillars, and MVX-Net for environment perception. This work addresses major challenges in sensor fusion, including data synchronization, coordinate transformation, real-time computation and conflict resolution of heterogeneous sensor data. We systematically analyze three typical fusion architectures: data-level, feature-level and decision-level fusion, and compare their performance in information retention, computational efficiency and system robustness. Through representative application cases in object detection and classification, high-precision localization and mapping, and decision-making and path planning, we demonstrate how multimodal fusion significantly enhances the robustness, accuracy, and reliability of autonomous vehicle perception systems. The paper further discusses current limitations including computational overhead, adverse-weather robustness, and lack of standardized evaluation, and outlines future directions such as end-to-end learning, 4D radar integration, and V2X-enabled cooperative perception. The results prove that reliable multimodal fusion is a core prerequisite for realizing safe and stable autonomous driving.
Yu-He Lu· Applied and Computational En...· 0 citations
Interaction with the physical world differentiates physical AI from other forms of AI. Autonomous driving exemplifies this; vehicles must perceive and respond to dynamic environments with human-like or better perception-reaction times. This survey addresses the fundamental challenge of deploying high-performance models for multimodal fusion in resource-constrained automotive environments. We organise state-of-the-art deep learning approaches into five paradigms—CNN-based, transformer-based, dense BEV-based, sparse-based, and hybrid—revealing trade-offs in accuracy, latency, and efficiency, as well as strengths and limitations in robustness under adverse operational design domains. The hardware-aware perspective is a differentiating contribution, presenting strategies for deployment on automotive platforms, reducing inference latency by up to 50% and improving robustness in adverse conditions by up to 20%. By synthesising sensor fusion, deep learning, compute platforms, and hardware-awareness, this work equips researchers and practitioners with actionable insights and strategies for perception systems, bridging theoretical advances and production-grade autonomous driving requirements.
Ken Power, M. Halton, Ciarán Eising· IEEE Open Journal of Vehicul...· 0 citations
For this reason, autonomous transportation systems have been established as a progressive technology that leverages Artificial Intelligence (AI), Internet of Things (IoT), computer vision, and advanced sensing technologies to improve road safety, operational efficiency, and smart mobility. Deep learning models have shown great strength in detecting objects, recognizing lanes and pedestrians, avoiding obstacles on the road, as well as analyzing real-time traffic situation. Although very accurate, these models are typically black-boxes which have limited transparency and confidence in safety-critical transportation applications. Interpretability can help with accident investigation, regulatory compliance, ethical decision-making, and public acceptance of autonomous vehicles where their lack presents major problems. We present an Explainable Deep Learning Framework for Autonomous Transportation Safety by combining convolutional neural networks with the Understandable Artificial Intelligence (XAI) techniques to ensure transparency and soulfulness in autonomous transportation. The proposed framework comprises sensor fusion, image processing, feature extraction, deep neural network inference and explainability mechanisms such as Gradient-weighted Class Activation Mapping (Grad-CAM), Local Interpretable Model-Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). It pre-evaluates the prediction interpretation, and creates visualisations and feature-level explanations that give stakeholders insight into how models can have high accuracy on detection outcomes. Additionally, the architecture promotes accountability, compliance with regulations, safer autonomous driving and public trust in intelligent transportation systems. Explainability is shown to be an important building block for designing robust autonomous transport platforms usable in the future.
Johan Håstad's mentor Arne Andersson, Börje Langefors· International Journal of Eme...· 0 citations
Autonomous robots play a crucial role in industrial manufacturing, healthcare, transportation, logistics, agriculture, disaster response, planetary exploration, and service robotics. Reliable visual perception is essential for enabling robots to recognize objects, understand scenes, localize themselves, and navigate safely in dynamic environments. Although CNN-based vision models have significantly improved perception accuracy, they often struggle to capture long-range dependencies and generalize to complex or unseen environments. Recent advances in Transformer-based vision models address these limitations by employing self-attention mechanisms to learn both local visual features and global contextual relationships. Architectures such as Vision Transformer (ViT), Swin Transformer, DETR, SAM, and Mask2Former have achieved remarkable performance in object detection, semantic segmentation, SLAM, localization, obstacle avoidance, and autonomous navigation. This paper presents a comprehensive review and proposes the Transformer-Based Visual Perception Models for Autonomous Robots (TBVPM-AR) framework. The framework integrates RGB cameras, depth sensors, LiDAR, IMUs, multimodal sensor fusion, transformer-based feature extraction, contextual reasoning, and edge-cloud computing to achieve robust perception in dynamic environments. Mathematical formulations for self-attention, positional encoding, and feature embedding provide the theoretical foundation of the architecture. Experimental evaluations demonstrate that the proposed framework outperforms CNN-based and hybrid approaches on standard robotic perception benchmarks, achieving over 98% visual perception accuracy with improved scene understanding, localization, obstacle detection, navigation, and computational efficiency. The proposed architecture offers a scalable, explainable, and adaptable solution for future Industry 5.0, collaborative robotics, autonomous vehicles, and smart cyber-physical systems.
Mahabala H.N· International Journal of Int...· 0 citations