Feature proliferation — the "cancer" in StyleGAN and its treatments

Song, Shuang, Liang, Yuanbang ORCID: https://orcid.org/0009-0000-8370-6655, Wu, Jing ORCID: https://orcid.org/0000-0001-5123-9861, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680 and Qin, Yipeng ORCID: https://orcid.org/0000-0002-1551-9126 2023. Feature proliferation — the "cancer" in StyleGAN and its treatments. Presented at: International Conference on Computer Vision (ICCV) 2023, Paris, France, October 1 - 6, 2023. Proceedings of IEEE/CVF International Conference on Computer Vision. IEEE, pp. 2360-2370. 10.1109/ICCV51070.2023.00224

Preview

PDF - Accepted Post-Print Version Download (8MB) | Preview

Abstract

Despite the success of StyleGAN in image synthesis, the images it synthesizes are not always perfect and the well-known truncation trick has become a standard post-processing technique for StyleGAN to synthesize high quality images. Although effective, it has long been noted that the truncation trick tends to reduce the diversity of synthesized images and unnecessarily sacrifices many distinct image features. To address this issue, in this paper, we first delve into the StyleGAN image synthesis mechanism and discover an important phenomenon, namely Feature Proliferation, which demonstrates how specific features reproduce with forward propagation. Then, we show how the occurrence of Feature Proliferation results in StyleGAN image artifacts. As an analogy, we refer to it as the "cancer" in StyleGAN from its proliferating and malignant nature. Finally, we propose a novel feature rescaling method that identifies and modulates risky features to mitigate feature proliferation. Thanks to our discovery of Feature Proliferation, the proposed feature rescaling method is less destructive and retains more useful image features than the truncation trick, as it is more fine-grained and works in a lower-level feature space rather than a high-level latent space. Experimental results justify the validity of our claims and the effectiveness of the proposed feature rescaling method. Our code is available at https://github.com/songc42/Feature-proliferation.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Publication
Status:	Published
Schools:	Advanced Research Computing @ Cardiff (ARCCA) Computer Science & Informatics
Publisher:	IEEE
ISBN:	9798350307191
Date of First Compliant Deposit:	25 September 2023
Date of Acceptance:	14 July 2023
Last Modified:	09 Nov 2024 23:45
URI:	https://orca.cardiff.ac.uk/id/eprint/161951

Actions (repository staff only)

Edit Item