WCGAN: Robust portrait watercolorization with adaptive hierarchical localized constraints

Lyu, Hongjin, Rosin, Paul L. ORCID: https://orcid.org/0000-0002-4965-3884 and Lai, YuKun ORCID: https://orcid.org/0000-0002-2094-5680 2023. WCGAN: Robust portrait watercolorization with adaptive hierarchical localized constraints. Displays 80 , 102530. 10.1016/j.displa.2023.102530

Preview

PDF - Published Version Available under License Creative Commons Attribution. Download (3MB) | Preview

Abstract

Deep learning has enabled image style transfer to make great strides forward. However, unlike many other styles, transferring the watercolor style to portraits is significantly challenging in image synthesis and style transfer. Pixel-correlation-based methods do not produce satisfactory watercolors. This is because portrait watercolors exhibit the sophisticated fusion of various painting techniques in local areas, which poses a problem for convolutional neural networks to accurately handle fine-grained features. Moreover, the common but problematic way of coping with multiple scales greatly impedes the performance of existing style transfer methods with fixed receptive fields. Although it is possible to develop an image processing pipeline mimicking various watercolor effects, such algorithms are slow and fragile, especially for inputs of different scales. As a remedy, this paper proposes WCGAN, a generative adversarial network (GAN) architecture dedicated to watercolorization of portraits. Specifically, a novel localized style loss suitable for watercolorization is proposed to deal with local details. To handle portraits of different scales and improve robustness, a novel discriminator architecture with three parallel branches of varying sizes of receptive fields is introduced. In addition, the application of WCGAN is expanded to video style transfer where a novel kind of video training data based on random crops is developed to efficiently capture temporal consistency. Extensive experimental results from qualitative and quantitative analyses demonstrate that WCGAN generates state-of-the-art, high quality watercolors from portraits.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Advanced Research Computing @ Cardiff (ARCCA) Computer Science & Informatics
Publisher:	Elsevier
ISSN:	0141-9382
Date of First Compliant Deposit:	20 September 2023
Date of Acceptance:	28 August 2023
Last Modified:	11 Jun 2024 12:35
URI:	https://orca.cardiff.ac.uk/id/eprint/162641

Actions (repository staff only)

Edit Item