3D Point Cloud–Based Digital Twin Reconstruction and VR Immersive Visualization of Borobudur’s Hidden Foot Reliefs

Abstract

Abstract. The Borobudur Temple’s Hidden Foot (Karmawibhangga layer), comprising 160 bas-relief panels sealed behind protective stone cladding since the early 20th century, presents a compelling challenge for digital cultural heritage practice. This paper presents a digital twin reconstruction and immersive virtual reality (VR) system that provides interactive access to this physically inaccessible archaeological space. The system integrates three pre-existing datasets: UAV photogrammetric point clouds of the outer temple structure, deep-learning-based 3D relief reconstructions derived from archival photographs (Pan et al., 2022), and semantic segmentation results of relief surfaces (Ji et al., 2023). Within a Unity 2022.3.22f1-based VR environment optimised for the HTC VIVE Pro Eye head-mounted display, these datasets are unified into a navigable Hidden Foot passage. A novel lighting-driven Level of Detail (LoD) strategy—where rendering complexity is governed by the illumination radius of a virtual torch—achieves 65–70 FPS on the HMD while preserving visual fidelity within the illuminated zone. Eye-gaze-triggered textual annotations support contextual iconographic interpretation, while interactive toggling between photorealistic and semantically labelled views enables both casual exploration and scholarly analysis. A user study (n = 10) employing a System Usability Scale-adapted questionnaire yielded mean satisfaction ratings exceeding 4.5/5 across all evaluation dimensions. The system demonstrates a scalable methodology for transforming inaccessible heritage layers into immersive, annotated digital environments.