Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

How does Lipschitz regularization influence GAN training?

Qin, Yipeng, Mitra, Niloy and Wonka, Peter 2020. How does Lipschitz regularization influence GAN training? Presented at: 16th European Conference on Computer Vision (ECCV 2020), Glasgow, Scotland, 23-28 August 2020. Published in: Vevaldi, Andrea, Bischof, Horst, Brox, Thomas and Frahm, Jan-Michael eds. Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI. Lecture Notes in Computer Science Springer, pp. 310-326. 10.1007/978-3-030-58517-4_19

[img]
Preview
PDF - Accepted Post-Print Version
Download (1MB) | Preview
[img]
Preview
PDF - Supplemental Material
Download (62MB) | Preview

Abstract

Despite the success of Lipschitz regularization in stabilizingGAN training, the exact reason of its effectiveness remains poorly un-derstood. The direct effect ofK-Lipschitz regularization is to restrict theL2-norm of the neural network gradient to be smaller than a thresholdK(e.g.,K= 1) such that‖∇f‖≤K. In this work, we uncover an evenmore important effect of Lipschitz regularization by examining its im-pact on the loss function:It degenerates GAN loss functions to almostlinear ones by restricting their domain and interval of attainable gradi-ent values. Our analysis shows that loss functions are only successful ifthey are degenerated to almost linear ones. We also show that loss func-tions perform poorly if they are not degenerated and that a wide rangeof functions can be used as loss function as long as they are sufficientlydegenerated by regularization. Basically, Lipschitz regularization ensuresthat all loss functionseffectively work in the same way.Empirically, weverify our proposition on the MNIST, CIFAR10 and CelebA datasets.

Item Type: Conference or Workshop Item (Paper)
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Publisher: Springer
ISBN: 9783030585167
ISSN: 0302-9743
Related URLs:
Date of First Compliant Deposit: 28 July 2020
Date of Acceptance: 2 July 2020
Last Modified: 15 Feb 2021 17:30
URI: http://orca.cardiff.ac.uk/id/eprint/133740

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics