DSG-Net: Learning disentangled structure and geometry for 3D shape generation

Yang, Jie, Mo, Kaichun, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680, Guibas, Leonidas J. and Gao, Lin 2023. DSG-Net: Learning disentangled structure and geometry for 3D shape generation. ACM Transactions on Graphics 42 (1) , 1. 10.1145/3526212

Preview

PDF - Accepted Post-Print Version Download (4MB) | Preview

Abstract

3D shape generation is a fundamental operation in computer graphics. While significant progress has been made, especially with recent deep generative models, it remains a challenge to synthesize high-quality shapes with rich geometric details and complex structures, in a controllable manner. To tackle this, we introduce DSG-Net, a deep neural network that learns a disentangled structured & geometric mesh representation for 3D shapes, where two key aspects of shapes, geometry and structure, are encoded in a synergistic manner to ensure plausibility of the generated shapes, while also being disentangled as much as possible. This supports a range of novel shape generation applications with disentangled control, such as interpolation of structure (geometry) while keeping geometry (structure) unchanged. To achieve this, we simultaneously learn structure and geometry through variational autoencoders (VAEs) in a hierarchical manner for both, with bijective mappings at each level. In this manner, we effectively encode geometry and structure in separate latent spaces, while ensuring their compatibility: the structure is used to guide the geometry and vice versa. At the leaf level, the part geometry is represented using a conditional part VAE, to encode high-quality geometric details, guided by the structure context as the condition. Our method not only supports controllable generation applications, but also produces high-quality synthesized shapes, outperforming state-of-the-art methods.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Association for Computing Machinery (ACM)
ISSN:	0730-0301
Funders:	The Royal Society
Date of First Compliant Deposit:	12 April 2022
Date of Acceptance:	12 March 2022
Last Modified:	06 Nov 2023 19:48
URI:	https://orca.cardiff.ac.uk/id/eprint/149165

Citation Data

Cited 12 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item