3DCascade-GAN: Shape completion from single-view depth images

Alhamazani, Fahd, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680 and Rosin, Paul L. ORCID: https://orcid.org/0000-0002-4965-3884 2023. 3DCascade-GAN: Shape completion from single-view depth images. Computers and Graphics 115 , pp. 412-422. 10.1016/j.cag.2023.07.033

Preview

PDF - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract

Depth images can be easily acquired using depth cameras. However, these images only contain partial information about the shape due to unavoidable self-occlusion. Thanks to the availability of large datasets of shapes, it is possible to use a learning-based approach to produce complete shapes from single depth images. State-of-the-art generative adversarial network (GAN) architectures can produce reasonable results. However, the use of relatively local convolutions restricts GAN architectures from producing globally plausible shapes. In this study, we develop a novel dynamic latent code selection mechanism in which the model learns to select only important codes from the latent space. Furthermore, a novel 3D self-attention (3DSA) layer is introduced that is able to capture non-local relationships across the 3D space. We further design a GAN architecture that uses a multistage encoder–decoder to recover the shape, where our 3DSA layer is introduced to the discriminator to help attend to global features, which stabilizes the model learning and encourages shape refinement, making our reconstruction more structurally plausible. Through extensive experiments, we demonstrate that our method outperforms other state-of-the-art methods for single depth image 3D reconstruction.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Elsevier
ISSN:	0097-8493
Date of First Compliant Deposit:	31 August 2023
Date of Acceptance:	13 July 2023
Last Modified:	06 Sep 2023 07:20
URI:	https://orca.cardiff.ac.uk/id/eprint/162119

Actions (repository staff only)

Edit Item