Two-Sample Tests, Integral Probability Metrics, and GAN Objective - Dougal J. Sutherland

DALI 2017 Workshop - Theory of Generative Adversarial Networks

http://dalimeeting.org/dali2017/gener...

Title:
Two-Sample Tests, Integral Probability Metrics, and GAN Objective

Speaker:
Dougal J. Sutherland (Gatsby unit, UCL), http://www.gatsby.ucl.ac.uk/~dougals/

Abstract:
One of the major failure patterns of typical GAN models is when the generator collapses to a single point considered highly realistic by the current discriminator, after which the learning problem becomes stuck. To help avoid this issue, we can replace the discriminator with a function that looks at an entire sample set at a time, so that no single point becomes attractive to the generator. Doing so brings us into the well-studied realm of two-sample testing. This talk will discuss several different techniques for two-sample testing and their application in GAN settings, including classifier-based two sample tests which correspond to the traditional GAN, the maximum mean discrepancy, and Wasserstein distances. We will also discuss the use of these types of distances as tools to diagnose convergence of generative models and discover ways in which their samples differ from the reference distribution.