From understanding to rendering: multimodal LLM-guided green retrofitting of urban street views

Abstract

Urban street retrofitting is increasingly used to improve greenery, walkability, and perceived safety, yet planners and communities often lack intuitive visualizations of how a street may look after such interventions. This paper presents a perception-aware framework for green streetscape redesign that transforms a real street-view image and a high-level design goal into a realistic visualization of a retrofitted street. The proposed framework integrates an MLLM-guided planner for structured redesign operations, a rule-based compiler for semantic mask editing, a ControlNet-guided diffusion renderer for candidate generation, and a perception-aware selector for choosing the final design. Experiments on Cityscapes and Mapillary Vistas, show that the method achieves a favorable balance among environmental improvement, image realism, and structural preservation. Additional comparisons further demonstrate the value of multimodal planning and perception-aware ranking. These results suggest that controllable generative models can provide practical support for urban street retrofit visualization and assessment.