Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

StyleGAN-∞: extending StyleGAN to arbitrary-ratio translation with StyleBook

Dai, Yihua, Xiang, Tianyi, Deng, Bailin ORCID: https://orcid.org/0000-0002-0158-7670, Du, Yong, Cai, Hongmin, Qin, Jing and He, Shengfeng 2024. StyleGAN-∞: extending StyleGAN to arbitrary-ratio translation with StyleBook. IEEE Transactions on Visualization and Computer Graphics 10.1109/tvcg.2024.3522565

[thumbnail of main.pdf]
Preview
PDF - Accepted Post-Print Version
Download (60MB) | Preview
[thumbnail of supp.pdf]
Preview
PDF - Supplemental Material
Download (58MB) | Preview

Abstract

Although pre-trained large-scale generative models StyleGAN series have proven to be effective in various editing and translation tasks, they are limited to pre-defined fixed aspect ratio. To overcome this limitation, we propose StyleGAN-∞, a model that enables pre-trained StyleGAN to perform arbitrary-ratio conditional synthesis. Our key insight is to distill the expressive StyleGAN features into a StyleBook, such that an arbitrary-ratio condition can be translated to other forms by properly assembling pre-defined StyleBook vectors. To learn and leverage the StyleBook, we employ a network with three distinct stages, each corresponding to StyleBook extraction, StyleBook correspondence learning, and arbitrary-ratio synthesis. Extensive experiments on various conditional synthesis tasks, like super-resolution, sketch synthesis, and semantic synthesis, demonstrate superior performances over state-of-the-art image-to-image translation methods. Moreover, our model can easily generate megapixel images in diverse modalities by taking advantage of different pre-trained StyleGAN models.

Item Type: Article
Date Type: Published Online
Status: In Press
Schools: Computer Science & Informatics
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Q Science > QA Mathematics > QA76 Computer software
Publisher: Institute of Electrical and Electronics Engineers
ISSN: 1077-2626
Date of First Compliant Deposit: 24 December 2024
Date of Acceptance: 11 December 2024
Last Modified: 16 Jan 2025 16:30
URI: https://orca.cardiff.ac.uk/id/eprint/174909

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics