OpenAsset: A Pipeline for Open-World Asset Integration into Indoor Scene Synthesis

Abstract

Text-driven 3D indoor scene synthesis has witnessed significant progress through Large Language Model (LLM)-based frameworks like ReSpace. However, current paradigms heavily rely on retrieving objects from pre-defined, static 3D asset libraries, which fundamentally constrains the diversity and personalization of generated scenes due to the closed-set nature of existing databases. Conversely, recent breakthroughs in promptable segmentation (SAM 3) and single-image 3D reconstruction (SAM 3D) have empowered the extraction of high-fidelity 3D geometry and texture from in-the-wild images. In this paper, we bridge the gap between text-driven scene layout generation and single-view object reconstruction by proposing OpenAsset. This novel framework converts single images of real-world objects into reusable, canonicalized 3D assets that seamlessly integrate into the scene synthesis workflow. Specifically, given a user concept prompt or target region, OpenAsset leverages SAM3 for precise instance isolation and SAM3D for geometry and texture recovery. To ensure compatibility with structured scene representations (SSR), we introduce an automated canonicalization module that normalizes the scale, orientation, and coordinate systems of the reconstructed meshes. By transforming “wild” visual percepts into standardized assets, OpenAsset effectively expands the controllable vocabulary of indoor scene synthesis beyond curated datasets, offering a practical pathway toward user-sourced open 3D scene generation that supports custom objects outside fixed predefined asset libraries. It is worth noting that the satisfactory performance of OpenAsset critically depends on effective segmentation and reconstruction results, which serve as essential prerequisites for our method.