TL;DR
A lightweight multi-task learning framework for safety-critical autonomous driving perception that strikes a balance between accuracy and real-time performance in multi-task perception, thereby supporting safety-critical perception by reducing perception latency and improving perception reliability.
Understanding the real-time behaviour of pedestrians and their surrounding context is essential for road safety. Unfortunately, current perception systems struggle to capture how multiple cues jointly influence behavior in diverse traffic scenarios. Most existing models in Intelligent Transportation Systems (ITS) and Autonomous driving system (ADS) predict only a single attribute, such as intent or trajectory, while overlooking the interaction between behavioral and environmental factors. We propose a unified multi-attribute framework based on a Vision Transformer, which is enhanced with lightweight, parameter-efficient adapters. This innovative modular design enables independent learning of eight complementary attributes, including pedestrian behavior (action, attention, crossing, occlusion) and scene context (weather, time of day, road presence, density). Our proposed framework updates fewer than 2% of the model parameters. We ran our proposed framework using the JAAD dataset, and the framework achieves over 90% accuracy on most attributes and exhibits consistent behavior–context relationships and demonstrates robustness under varied environmental conditions. To evaluate real-time suitability, we implemented an adaptive strategy that activates only the most relevant adapters based on scene conditions. This approach reduces computational cost by 28.5% and increases throughput by 40.8% without loss of accuracy. This research distinctly highlights the practical potential of modular, PEFT-enhanced transformers for scalable multi-attribute perception as foundational vision models continue to advance. The best-performing adapters and inference code are available on Github Repo.
With the large-scale implementation of scenarios such as logistics warehousing and park inspections, the demand for autonomous environmental perception by mobile robots continues to grow. Low-cost pure vision semantic perception has become a core technology for ensuring autonomous safe navigation of these robots. Traditional manual feature algorithms lack stability in complex road conditions and varying lighting conditions, while deep learning relies on autonomous hierarchical feature learning to break through the original performance limit. This article focuses on real-time semantic perception of embedded mobile cars, systematically sorting out three mainstream deep learning solutions: CNN, lightweight network, and visual Transformer, and comparing the adaptation differences of each architecture in segmentation accuracy, inference speed, and complex working conditions. Summarize the existing technological bottlenecks from three aspects: imbalanced real-time accuracy, weak robustness in complex environments, and difficulty in recognizing distant and occluded targets. Corresponding optimization ideas are proposed, including dynamic lightweight inference, cross-domain adaptive enhancement, convolutional attention hybrid architecture, and end-to-end collaborative deployment. The article comprehensively outlines the development trajectory of visual semantic perception technology for mobile vehicles, providing a systematic theoretical reference for the design and embedded deployment optimization of visual algorithms for mobile robots in warehousing and inspection tasks.
This study puts forward a resilient cross-task modeling architecture to overcome the precision-versus-latency dilemma in comprehensive traffic scene understanding. The backbone employs a C2f module to enhance gradient flow and small-object detection, while structural re-parameterization via repVGG blocks enables multi-branch training and fast single-path inference. In the segmentation decoder, the CARAFE content-aware upsampling operator replaces nearest-neighbor interpolation to preserve fine edge details, significantly improving lane line continuity. When assessed on the BDD100K corpus, our model's empirical results indicate it surpasses YOLOP as well as HybridNets in detecting objects, segmenting drivable zones, and delineating lane markings. Removal experiments verify the synergistic contributions of each module. Operating at 186 frames per second and utilizing 16.89 million parameters, the network achieves an advantageous tradeoff between precision and computational efficiency for perception tasks in autonomous vehicles.
Autonomous vehicles depend on fast and reliable perception systems to detect surrounding vehicles, pedestrians, cyclists, traffic signs, and other road objects in real time. This paper presents a comprehensive survey and analysis of one-stage object detectors for autonomous driving rather than an implementation of a new detection system. The survey reviews the evolution of major one-stage detectors, including YOLOv1, SSD, RetinaNet, EfficientDet, anchor-free detectors such as FCOS and CenterNet, and recent real-time models such as YOLOv10. It compares these architectures through their design choices, feature-fusion strategies, loss functions, deployment trade-offs, and reported benchmark performance. The paper also summarizes commonly used autonomous-driving datasets, evaluation metrics, open challenges, and future research directions. Overall, this survey highlights how one-stage detectors balance speed, accuracy, efficiency, and robustness, while also emphasizing the remaining gap between benchmark results and dependable real-world autonomous-driving performance.
Current road safety systems face challenges in detecting curved roads and vehicles under varying lighting and weather conditions, leading to lane departure and collision risks. To address this, we propose CurvLaneNet-YOLO, a deep learning framework based on YOLOv8 that simultaneously detects road curvature and vehicles in lanes. The system integrates a parallel polynomial lane detection head into the YOLOv8 architecture, enabling real-time curvature estimation alongside vehicle detection. This research develops an AI-enabled real-time monitoring system to substantially improve road safety. The system makes use of the enhanced capabilities and features of the YOLOv8 model, which involves data preparation, model training, extensive testing, and data augmentation to guarantee model precision. Main goals are estimation of road curvature in real-time, high-performance processing and better assist for the driver. To assess the research, the Cars sample from KITTI dataset on Kaggle, which includes 7,481 images of 640 × 640 resolution, were used. The research achieves an inference latency of 15.4 ± 0.3 ms/image on the test hardware, with mAP@0.5 = 0.9373 and mAP@0.5:0.95 = 0.9217, indicating significant improvements over existing work. This research demonstrates the feasibility of using deep learning techniques for vehicle detection and road curvature estimation in real-time. The results presented in this study show promising potential to integrate into a driver-assistance system, although the data used here is limited to a proof-of-concept validation on the KITTI dataset. The suggested solution is competitive in terms of accuracy and inference time on embedded hardware, providing a potential roadmap for its application in real-world scenarios within the context of intelligent transportation systems.
Highlights What are the main findings? A latent-space LM-JEPA framework enables resource-efficient multi-modal object detection and scene perception for connected and autonomous vehicles, achieving higher perception accuracy with lower inference latency compared to conventional LLM and VLM-based methods. Context-aware and adaptive sensor fusion, selective latent transmission, and lightweight edge-assisted reasoning improve cooperative scene understanding, yielding up to 25% better scene understanding, 20% higher intersection success rates, and a 15% reduction in transmitted model parameters. What are the implications of the main findings? Latent representation learning provides a practical alternative to token-based LLM and VLM inference, making real-time multi-modal perception feasible on resource-constrained edge devices in connected and autonomous vehicles. The proposed framework demonstrates that adaptive latent communication and collaborative reasoning can enhance the scalability, energy efficiency, and safety of future intelligent transportation and cooperative autonomous driving systems. Abstract This paper presents the latent model-joint embedding predictive architecture (LM-JEPA), a resource-efficient collaborative perception framework for connected and autonomous vehicles that integrates latent predictive representation learning with lightweight multi-modal reasoning. Autonomous driving in urban and highway environments requires accurate scene understanding under strict latency, energy, and communication constraints, limiting the practicality of large language model (LLM) and vision–language model (VLM)-based approaches in edge deployments. To address this, LM-JEPA encodes heterogeneous inputs including camera, LiDAR, radar, and map data into a unified latent space using a joint embedding predictive architecture, enabling efficient perception and reasoning without token-level inference. Unlike existing latent-space learning approaches that primarily learn predictive visual embeddings for single-modal perception, the proposed framework integrates multi-modal latent reasoning and adaptive sensor fusion to support collaborative perception under resource-constrained vehicular edge environments. The collaborative perception framework introduces a context-adaptive multi-modal fusion mechanism that dynamically weights sensor and model contributions, along with selective latent transmission and adaptive decoding for resource-aware operation. A lightweight VLM is integrated with an edge-assisted vehicular pipeline to support real-time on-vehicle inference with adaptive offloading based on latency and energy constraints, while a latent-space reasoning module enables cooperative decision-making. Experiments on BDD100K and nuScenes-QA show that LM-JEPA improves perception accuracy by 5% and reduces latency by approximately 7% over LLM and VLM baselines, while achieving up to 25% improvement in scene understanding, 20% higher intersection success rates, improved highway merging, and approximately 15% reduction in the transmitted model parameters.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
