Jul 2026· International Conference on Computer Graphics and Interactive Techniques· pp. 1-12· 0 citations· 59 references
Computer Science
TL;DR
StudioRecon is proposed, a pipeline that reconstructs 4D human scenes from sparse, low-overlap cameras by decoupling background and humans by synthesizing hundreds of camera-controlled novel views with a video diffusion model and achieves state-of-the-art novel view synthesis across four real-world datasets.
Abstract
Existing volumetric capture of dynamic human performance achieves high fidelity with dense camera arrays. However, in real-world scenarios, only a handful of low-overlap cameras are available, which degrades the output quality and leaves large areas unobserved. Recent 4D reconstruction methods have focused on low-overlap settings, yet they still produce noticeable artifacts in under-observed regions. Video diffusion models have emerged as another option, but they show geometrically inconsistent results for humans. To address these limitations, we propose StudioRecon, a pipeline that reconstructs 4D human scenes from sparse, low-overlap cameras by decoupling background and humans. We densify background supervision by synthesizing hundreds of camera-controlled novel views with a video diffusion model. We also robustly initialize deformable Gaussian humans with cross-view identity association and triangulated multi-view keypoint fitting. Finally, our recursive enhancement module with motion-adaptive consistency injection harmonizes the composed output, thereby further avoiding remaining artifacts. We achieve state-of-the-art novel view synthesis across four real-world datasets and demonstrate applications such as novel trajectory rendering and human replacement. Project page: https://sisyphm.github.io/studiorecon-page/.
Many consumer smartphones, stereo cameras, and light field cameras record multiple synchronized viewpoints in a single exposure event. However, novel view synthesis pipelines commonly use only a monocular stream and rely on camera motion or learned priors to obtain angular coverage. In this paper, we ask: why do we use only one viewpoint? We analyze sensor-limited multi-view, where one sensor trades off spatial and angular resolution, and exposure-limited multi-view, where multiple sensors on one commodity device observe each event simultaneously. We introduce a new dataset incorporating three types of commodity multi-view cameras, and evaluate sparse-view 3DGS and 4DGS baselines measuring reconstruction quality as a function of number of exposures and angle between extreme views. Our results demonstrate that using multiple cameras, even with a low baseline, significantly improves reconstruction quality in single-shot, few-shot, and casual video settings. In addition, under a fixed sensor budget, angular sampling improves reconstruction when exposures are scarce despite lower spatial resolution. The gains are most pronounced for single-shot and dynamic scenes, where a stationary monocular camera lacks the angular diversity to recover scene geometry and motion.
Shamus Li, Ruiming Cao, Laura Waller et al.· 0 citations
Recent advances in 3D reconstruction using mobile cameras have expanded their potential applications beyond traditional domains such as surveying and virtual reality, extending to a wide range of industries. In particular, 3D Gaussian Splatting (3DGS), which can generate photorealistic novel view synthesis from video captured with off-the-shelf RGB cameras, shows promise for industrial use cases involving mobile camera systems that collect images in real time. However, pipelines based on offline Structure-from-Motion (SfM), e.g., COLMAP, are computationally expensive and thus limit practical deployment. While neural network–based acceleration methods have emerged, they are typically limited to processing a small number of input frames, constraining both reconstruction accuracy and spatial coverage. This paper proposes a framework for efficient incremental optimization of 3DGS models. Our framework enables fast and effective fine-tuning by adaptively adjusting the camera poses for additional image frames based on the relationship between L1 and SSIM rendering losses used for 3DGS optimization. Applied to rapidly initialized 3DGS models, our approach achieves a 24% relative improvement in SSIM with just 120 seconds of additional optimization on the Mip-NeRF360 dataset.
Single-image 3D human reconstruction often suffers from over-smoothed textures and geometric inconsistencies. While diffusion models improve generative quality, their reliance on multi-view synthesis prior to 3D reconstruction is computationally expensive and prone to view inconsistency. We propose DiGS-Avatar, which reformulates this task as an efficient, diffusion-based UV-latent completion task, ensuring 3D consistency by design. To capture accurate spatial structure, we introduce a teacher-student framework where a multi-view teacher provides geometrically aligned pseudo-ground-truth latents to supervise a single-view diffusion student. Treating this inferred latent as a robust structural skeleton, our method injects high-level semantic features to accurately recover fine textural details without disrupting spatial integrity. The refined representation is then decoded into 3D Gaussian primitives. Extensive experiments demonstrate that DiGS-Avatar achieves state-of-the-art or highly competitive visual fidelity and zero-shot generalization, while reconstructing a fully animatable 3D avatar in just 0.71 seconds. Code is available at https://github.com/KLMAV-CUC/DiGS-Avatar.
Recent advances in 3D human reconstruction have improved overall performance, yet current models still fail in the most challenging real-world scenarios. They often produce unstable geometry, inaccurate limb articulation and unreliable predictions under depth ambiguity or self-occlusion. A key reason is that existing datasets still lack the combination of high-resolution images, high-precision annotations and diverse whole-body motions required to support robust reconstruction. To address this gap, we present Human4K, a large-scale 4K multi-view whole-body human reconstruction dataset with mocap-accurate SMPL-X annotations. Human4K contains over six million 4K images captured by an eight-view high-resolution camera system synchronized with a professional Vicon motion capture setup, covering 11 subjects performing complex, highly articulated and strongly self-occluded full-body motions. All sequences are processed by a Motion-Retargeting and Refinement Module (MRRM) to ensure precise alignment for the full body and extremities. Experimental results show that training with Human4K consistently improves whole-body reconstruction on standard benchmarks, with particularly large gains for hands, feet and depth-ambiguous limb configurations.
Tianshun Han, Ziyu Shi, Lijiang Liu et al.· 0 citations
The task of reconstructing dynamic neural scenes from sparse multi-view observations represents a significant challenge in computer vision and computer graphics. Traditional neural rendering techniques require dense view sampling to synthesize high-quality novel views, which is highly impractical for real-world dynamic environments where deploying numerous synchronized cameras is prohibitively expensive and logistically complex. When constrained to sparse views, existing dynamic reconstruction frameworks typically suffer from severe overfitting, resulting in pronounced geometric distortions, floaters, and temporal inconsistencies. This paper proposes a novel framework designed to achieve high-fidelity dynamic neural scene reconstruction from highly sparse viewpoints by integrating geometry-aware depth priors and robust multi-view correspondence constraints. By leveraging monocular depth estimation aligned with sparse structural cues, the system enforces a strict geometric foundation that prevents the neural field from degenerating in unobserved regions. Furthermore, we introduce a cross-view feature correspondence mechanism that penalizes photometric and geometric divergences across temporal and spatial domains, ensuring consistency in the deformation fields used to model scene dynamics. Comprehensive evaluations demonstrate that our methodology significantly suppresses artifacts and achieves superior novel view synthesis quality compared to baseline methods. The integration of these complementary constraints effectively bridges the information gap inherent in sparse observations, offering a scalable solution for dynamic scene capture.
Camila Wilson, Diego J. Ross· Journal of innovative resear...· 0 citations
This paper tackles the challenge of novel view synthesis in complex scenes with under-constrained motion, as captured in monocular videos. Existing methods mainly focus on handling motion restricted within a bounded 3D volume, relying on spatio-temporal information to drive dynamic Gaussian deformations. However, due to the inherent motion ambiguities in monocular dynamic 3D representations and the limited observations, these methods face challenges in handling such scenes, often leading to incomplete geometry and boundary artifacts. To mitigate these issues, we propose a semantics-guided scene decoupling module that separates Gaussian primitives into static and dynamic components based on motion vectors. Further more, to enhance the capability in modeling non-rigid motions, we introduce a motion-aware densification module for motion compensation, which alleviates the incomplete rendering of dynamic objects caused by insufficient spatio-temporal information. Experimental results on real-world datasets demonstrate that our approach outperforms state-of-the-art methods in preserving both the integrity and detailed appearance of moving objects in dynamic scenes.
Chulin Zhao, Xue Wang, Guoqing Zhou et al.· IEEE Transactions on Visuali...· 0 citations