SceneHGN: Hierarchical Graph Networks for 3D indoor scene generation with fine-grained geometry

Gao, Lin, Sun, Jia-Mu, Mo, Kaichun, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680, Guibas, Leonidas J. and Yang, Jie 2023. SceneHGN: Hierarchical Graph Networks for 3D indoor scene generation with fine-grained geometry. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (7) , pp. 8902-8919. 10.1109/TPAMI.2023.3237577

Preview

PDF - Accepted Post-Print Version Download (62MB) | Preview

Abstract

3D indoor scenes are widely used in computer graphics, with applications ranging from interior design to gaming to virtual and augmented reality. They also contain rich information, including room layout, as well as furniture type, geometry, and placement. High-quality 3D indoor scenes are highly demanded while it requires expertise and is time-consuming to design high-quality 3D indoor scenes manually. Existing research only addresses partial problems: some works learn to generate room layout, and other works focus on generating detailed structure and geometry of individual furniture objects. However, these partial steps are related and should be addressed together for optimal synthesis. We propose Scene HGN, a hierarchical graph network for 3D indoor scenes that takes into account the full hierarchy from the room level to the object level, then finally to the object part level. Therefore for the first time, our method is able to directly generate plausible 3D room content, including furniture objects with fine-grained geometry, and their layout. To address the challenge, we introduce functional regions as intermediate proxies between the room and object levels to make learning more manageable. To ensure plausibility, our graph-based representation incorporates both vertical edges connecting child nodes with parent nodes from different levels, and horizontal edges encoding relationships between nodes at the same level. Our generation network is a conditional recursive neural network (RvNN) based variational autoencoder (VAE) that learns to generate detailed content with fine-grained geometry for a room, given the room boundary as the condition. Extensive experiments demonstrate that our method produces superior generation results, even when comparing results of partial steps with alternative methods that can only achieve these. We also demonstrate that our method is effective for various applications such as part-level room editing, room interpolation, and room ge...

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Institute of Electrical and Electronics Engineers
ISSN:	0162-8828
Date of First Compliant Deposit:	13 February 2023
Date of Acceptance:	29 December 2022
Last Modified:	13 Nov 2024 06:00
URI:	https://orca.cardiff.ac.uk/id/eprint/156938

Actions (repository staff only)

Edit Item