Stylistic scene enhancement GAN: Mixed stylistic enhancement generation for 3D indoor scenes

Zhang, Suiyun, Han, Zhizhong, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680, Zwicker, Matthias and Zhang, Hui 2019. Stylistic scene enhancement GAN: Mixed stylistic enhancement generation for 3D indoor scenes. Visual Computer 35 (6) , pp. 1157-1169. 10.1007/s00371-019-01691-w

Preview

PDF - Accepted Post-Print Version Download (34MB) | Preview

Abstract

In this paper, we present stylistic scene enhancement GAN, SSE-GAN, a conditional Wasserstein GAN-based approach to automatic generation of mixed stylistic enhancements for 3D indoor scenes. An enhancement indicates factors that can influence the style of an indoor scene such as furniture colors and occurrence of small objects. To facilitate network training, we propose a novel enhancement feature encoding method, which represents an enhancement by a multi-one-hot vector, and effectively accommodates different enhancement factors. A Gumbel-Softmax module is introduced in the generator network to enable the generation of high fidelity enhancement features that can better confuse the discriminator. Experiments show that our approach is superior to the other baseline methods and successfully models the relationship between the style distribution and scene enhancements. Thus, although only trained with a dataset of room images in single styles, the trained generator can generate mixed stylistic enhancements by specifying multiple styles as the condition. Our approach is the first to apply a Gumbel-Softmax module in conditional Wasserstein GANs, as well as the first to explore the application of GAN-based models in the scene enhancement field.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Springer Verlag
ISSN:	0178-2789
Date of First Compliant Deposit:	5 May 2019
Date of Acceptance:	21 March 2019
Last Modified:	19 Nov 2024 07:00
URI:	https://orca.cardiff.ac.uk/id/eprint/122167

Citation Data

Cited 16 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item