Reward Design for Vision-Assisted Autonomous Driving in MetaDrive: From Scalar Rewards to Event-Triggered Visual Guidance

Abstract

Visual feedback can characterize traffic interactions, yet a plausible visual score may not promote task completion. This paper examines this mismatch in a MetaDrive intersection task governed by state-based Proximal Policy Optimization (PPO). With matched 1.8-million-step training, a distilled scalar visual reward raises return but reduces success from 0.388 to 0.287. Event-Triggered Visual Guidance (ETVG) instead uses a frozen ResNet18 to process one overlaid top-down RGB frame every five environment steps and determine whether the ego vehicle should yield. At the first qualifying event, ETVG preserves PPO steering and caps only the longitudinal action for five steps. Offline labels derive from path conflicts and relative arrival times; deployment uses only current RGB. The frozen visual layer is attached to three independently trained PPO policies, each evaluated on 300 matched scenarios. Mean success increases from 0.338 to 0.410, with a significant paired gain for every policy seed (p<=0.0201); collision and road-departure rates also decrease. On an episode-disjoint test, the detector reaches 0.9849 ROC AUC and 0.8337 average precision. Exact-reset, state-trigger, raw-image, traffic-density, joint-training, and local-VLM ablations indicate that the gain arises from binding a single visual event to a restricted action rather than accumulating a general visual score.