Cross-attention multimodal transformer: adaptive segmentation of urban functional areas by fusing street view and satellite imagery

Abstract

Urban functional area division helps to improve urban planning, new-area construction and public service distribution. To address the deficiency of single-source imagery, this paper proposes a cross-attention multimodal Transformer that integrates street-view cues, such as building facades and business types, with satellite features, including roof shape, block texture and road structure. Quality scoring, geometric alignment, cross-modal attention and uncertainty gating are used to adaptively segment the six functional areas of an urban area, reaching an mIoU of 88.6% and an F1 score of 90.1%, and surpassing SegFormer-B0. The way is still relatively robust in low light and under occlusion or location perturbation to provide some visual evidence for urban planning.