How does Lipschitz regularization influence GAN training?

Qin, Yipeng ORCID: https://orcid.org/0000-0002-1551-9126, Mitra, Niloy and Wonka, Peter 2020. How does Lipschitz regularization influence GAN training? Presented at: 16th European Conference on Computer Vision (ECCV 2020), Glasgow, Scotland, 23-28 August 2020. Published in: Vevaldi, Andrea, Bischof, Horst, Brox, Thomas and Frahm, Jan-Michael eds. Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI. Lecture Notes in Computer Science. Lecture Notes in Computer Science Springer, pp. 310-326. 10.1007/978-3-030-58517-4_19

Preview	PDF - Accepted Post-Print Version Download (1MB) | Preview
Preview	PDF - Supplemental Material Download (62MB) | Preview

Abstract

Despite the success of Lipschitz regularization in stabilizingGAN training, the exact reason of its effectiveness remains poorly un-derstood. The direct effect ofK-Lipschitz regularization is to restrict theL2-norm of the neural network gradient to be smaller than a thresholdK(e.g.,K= 1) such that‖∇f‖≤K. In this work, we uncover an evenmore important effect of Lipschitz regularization by examining its im-pact on the loss function:It degenerates GAN loss functions to almostlinear ones by restricting their domain and interval of attainable gradi-ent values. Our analysis shows that loss functions are only successful ifthey are degenerated to almost linear ones. We also show that loss func-tions perform poorly if they are not degenerated and that a wide rangeof functions can be used as loss function as long as they are sufficientlydegenerated by regularization. Basically, Lipschitz regularization ensuresthat all loss functionseffectively work in the same way.Empirically, weverify our proposition on the MNIST, CIFAR10 and CelebA datasets.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Springer
ISBN:	9783030585167
ISSN:	0302-9743
Related URLs:	https://www.ecva.net/papers/eccv_2020/pa...
Date of First Compliant Deposit:	28 July 2020
Date of Acceptance:	2 July 2020
Last Modified:	07 Nov 2022 10:51
URI:	https://orca.cardiff.ac.uk/id/eprint/133740

Citation Data

Cited 6 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item