Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

WCGAN: Robust portrait watercolorization with adaptive hierarchical localized constraints

Lyu, Hongjin, Rosin, Paul L. ORCID: and Lai, YuKun ORCID: 2023. WCGAN: Robust portrait watercolorization with adaptive hierarchical localized constraints. Displays 80 , 102530. 10.1016/j.displa.2023.102530

[thumbnail of WCGAN_Displays.pdf]
PDF - Published Version
Available under License Creative Commons Attribution.

Download (3MB) | Preview


Deep learning has enabled image style transfer to make great strides forward. However, unlike many other styles, transferring the watercolor style to portraits is significantly challenging in image synthesis and style transfer. Pixel-correlation-based methods do not produce satisfactory watercolors. This is because portrait watercolors exhibit the sophisticated fusion of various painting techniques in local areas, which poses a problem for convolutional neural networks to accurately handle fine-grained features. Moreover, the common but problematic way of coping with multiple scales greatly impedes the performance of existing style transfer methods with fixed receptive fields. Although it is possible to develop an image processing pipeline mimicking various watercolor effects, such algorithms are slow and fragile, especially for inputs of different scales. As a remedy, this paper proposes WCGAN, a generative adversarial network (GAN) architecture dedicated to watercolorization of portraits. Specifically, a novel localized style loss suitable for watercolorization is proposed to deal with local details. To handle portraits of different scales and improve robustness, a novel discriminator architecture with three parallel branches of varying sizes of receptive fields is introduced. In addition, the application of WCGAN is expanded to video style transfer where a novel kind of video training data based on random crops is developed to efficiently capture temporal consistency. Extensive experimental results from qualitative and quantitative analyses demonstrate that WCGAN generates state-of-the-art, high quality watercolors from portraits.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Advanced Research Computing @ Cardiff (ARCCA)
Computer Science & Informatics
Publisher: Elsevier
ISSN: 0141-9382
Date of First Compliant Deposit: 20 September 2023
Date of Acceptance: 28 August 2023
Last Modified: 11 Jun 2024 12:35

Actions (repository staff only)

Edit Item Edit Item


Downloads per month over past year

View more statistics