Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Feature proliferation — the "cancer" in StyleGAN and its treatments

Song, Shuang, Liang, Yuanbang, Wu, Jing ORCID: https://orcid.org/0000-0001-5123-9861, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680 and Qin, Yipeng ORCID: https://orcid.org/0000-0002-1551-9126 2023. Feature proliferation — the "cancer" in StyleGAN and its treatments. Presented at: International Conference on Computer Vision (ICCV) 2023, Paris, France, October 1 - 6, 2023. Proceedings of IEEE/CVF International Conference on Computer Vision. IEEE, pp. 2360-2370. 10.1109/ICCV51070.2023.00224

[thumbnail of 09471.pdf]
Preview
PDF - Accepted Post-Print Version
Download (8MB) | Preview

Abstract

Despite the success of StyleGAN in image synthesis, the images it synthesizes are not always perfect and the well-known truncation trick has become a standard post-processing technique for StyleGAN to synthesize high quality images. Although effective, it has long been noted that the truncation trick tends to reduce the diversity of synthesized images and unnecessarily sacrifices many distinct image features. To address this issue, in this paper, we first delve into the StyleGAN image synthesis mechanism and discover an important phenomenon, namely Feature Proliferation, which demonstrates how specific features reproduce with forward propagation. Then, we show how the occurrence of Feature Proliferation results in StyleGAN image artifacts. As an analogy, we refer to it as the "cancer" in StyleGAN from its proliferating and malignant nature. Finally, we propose a novel feature rescaling method that identifies and modulates risky features to mitigate feature proliferation. Thanks to our discovery of Feature Proliferation, the proposed feature rescaling method is less destructive and retains more useful image features than the truncation trick, as it is more fine-grained and works in a lower-level feature space rather than a high-level latent space. Experimental results justify the validity of our claims and the effectiveness of the proposed feature rescaling method. Our code is available at https://github.com/songc42/Feature-proliferation.

Item Type: Conference or Workshop Item (Paper)
Date Type: Publication
Status: Published
Schools: Advanced Research Computing @ Cardiff (ARCCA)
Computer Science & Informatics
Publisher: IEEE
ISBN: 9798350307191
Date of First Compliant Deposit: 25 September 2023
Date of Acceptance: 14 July 2023
Last Modified: 12 Jun 2024 16:36
URI: https://orca.cardiff.ac.uk/id/eprint/161951

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics