MemGeoSeg: Location-Aware Semantic Segmentation with Spatially Indexed Memory for Repetitive Driving Scenarios

Abstract

Current visual perception techniques for self-driving vehicles mainly focus on the generalization across diverse scenes, and they often overlook the valuable spatial consistency present in the repetitive driving routes such as public transit lines, delivery and shuttle services. In this paper, we introduce MemGeoSeg, which is a novel multi-modal framework that enhances semantic segmentation by exploiting scene repetitions through GPS-guided spatial priors and historical memory. Our approach introduces a hierarchical GPS embedding module, which is a spatially indexed memory bank that accumulates location-specific visual knowledge and a cross-modal fusion mechanism with contrastive learning. To validate the idea of improving visual perception with repetitive driving scenarios, a new dataset, RMTD-AD, is constructed for evaluation. It contains over 13,000 annotated images across various weather and lighting conditions on repeated routes. Extensive experiments conducted on the dataset have demonstrated that MemGeoSeg significantly outperforms the state-of-the-art baseline, achieving an mIoU of 76.5% compared to SegFormer’s 71.8% (a 4.7 percentage-point improvement), with particularly strong gains in challenging scenarios like low-light and adverse weather conditions. The result shows that there are substantial benefits to incorporating geographical contexts and historical memory for location-aware perception in intelligent vehicles.