Generative AI-Assisted Visualization Prototyping for Cultural Heritage: A Computational Framework from 2D Planes to 3D Immersive Scenes

Abstract

Immersive visualization can support interpretation of architectural heritage in historical paintings, yet translating 2D pictorial evidence into navigable 3D scenes remains challenging. Conventional workflows rely on physical survey data, while direct generative AI (GenAI) may produce structural hallucinations and lack historical constraints. This study proposes a human-in-the-loop GenAI-assisted framework for producing immersive 3D visualization prototypes rather than historically verified reconstructions. It integrates multi-view image generation, knowledge-informed review, single-image-to-3D generation, topology inspection, and perceptual calibration. Four fragments from the Northern Song Dynasty painting Along the River During the Qingming Festival were examined as a single-case proof of concept. Across three tested model pairs, raw AI assets were generated in approximately 3–4 min and were suitable for distant-background use; close-up visualization required 1–2 h of refinement, while basic structural editability required 4–5 h of post-processing, reducing the initial time advantage. A mixed-methods study with nine domain experts and 30 non-expert participants used the UES-SF, an adapted VisAWI, and semi-structured interviews analyzed through inductive thematic analysis. All eight subscale scores exceeded their neutral midpoints after Bonferroni correction (all adjusted p<0.001), indicating favorable perceptions of the guided experience. Interviews suggested potential for spatial exploration, museum interpretation, and education. However, geometric discontinuities, detail loss, color deviation, and historical-semantic errors remained, requiring expert review and manual correction. Transferability beyond this artwork and architectural tradition remains untested.