Cross-Domain Outdoor Scene UnderstandingUsing PC-GAN and TL-SZKSE-DenseNet inAdverse Weather Conditions
TL;DR
A dual framework bringing together a Panoptic Cycle-Generative Adversarial Network (PC-GAN) with a Transfer Learning-based Squeeze-and-ZeroKnowledge Spatial Excitation DenseNet (TL-SZKSE-DenseNet) achieves realistic crossweather image translation with semantically aligned representations in clear and degraded scenes.
Abstract
In autonomous driving, Outdoor Scene (OS) understanding in unfavorable weather conditions is pivotal forenabling safe navigation. Conventional studies failed to capture the intricate pairwise object relationship within scenesin OS classification. To mitigate these challenges, this research introduces a dual framework bringing together aPanoptic Cycle-Generative Adversarial Network (PC-GAN) with a Transfer Learning-based Squeeze-and-ZeroKnowledge Spatial Excitation DenseNet (TL-SZKSE-DenseNet). The suggested PC-GAN achieves realistic crossweather image translation with semantically aligned representations in clear and degraded scenes. The TL-SZKSEDenseNet improves feature learning with the addition of a zero-knowledge spatial excitation mechanism that selectivelyenhances weather-independent and context-dependent features for enhanced discrimination in crowded scenes. Thisgenerative–discriminative hybrid method effectively overcomes structured and unstructured driving domain gaps, asdemonstrated on benchmark datasets such as BDD and IndiaScene365. Primarily, the OS images are generalized toadverse conditions using Panoptic Cycle-Generative Adversarial Network (PC-GAN). After that, using saliencymapping, the most significant regions in the generalized output are highlighted. In the meantime, the significant featuresare extracted utilizing the Orangutan Optimization Algorithm (OOA). In addition, from the detected objects, the ObjectFrequency and Relationship (OFR) is analyzed. Later, using TL-SZKSE-DenseNet, the OS are classified based on theOFR and the selected features. The performance of our proposed approach was tested on our newly developedIndiaScene365 dataset as well as various benchmark datasets such as BDD100k. Our newly developed dataset,IndiaScene365, includes 3000 images that entail three different weather conditions, which are foggy weather, rainyweather, and poor light conditions along with clear day weather. The proposed approach records 94.87% accuracy,93.24%, and 94.18% sensitivity on our dataset.