Wednesday, September 7 at 10:00 a.m. (sharp)
PhD Candidate: Tristan Milne
Supervisor: Adrian Nachman
Thesis title: Optimal Transport, Congested Transport, and Wasserstein Generative
Adversarial Networks
***
Generative Adversarial Networks (GANs) are a method for producing a distribution $\mu$ that one can sample which approximates a distribution $\nu$ of real data. Wasserstein GANs with Gradient Penalty (WGAN-GP) [GAA+17] were designed to update $\mu$ by computing and then minimizing the Wasserstein 1 distance between $\mu$ and $\nu$. In the first part of this thesis we show that in fact, WGAN-GP do not compute this distance. Instead, they compute the minimum of a different optimal transport problem, the so-called congested transport [CJS08]. We then use this result to offer explanations of the observed performance of WGAN-GP. Our discovery also elucidates the role of the gradient penalty sampling strategy in WGAN-GP, and we show that by modifying this distribution one can ameliorate a transient form of mode collapse in the optimal mass flows.
The second part of this thesis presents new algorithms for generative modelling based on insights from optimal transport theory. The basic idea is to transform one distribution into another via iterated descent with an adaptive step size on learned Kantorovich potentials computed with WGAN-GP. We provide an initial convergence theory for this technique, as well as guarantees of convergence for an extension of this procedure when the target distribution is supported on a submanifold of Euclidean space with codimension at least two. As a proof of concept, we demonstrate via experiments that this provides a flexible and effective approach for several generative modelling problems, including image generation, translation, and denoising.
Further analysis of this algorithm reveals that it is connected to image restoration techniques via learned regularizers, which generalize the classical total variation denoising technique of Rudin-Osher-Fatemi (ROF) [ROF92]. We provide analogues of the results of [Mey01] on ROF to the learned regularizer setting. Leveraging this connection, we provide optimal transport versions of the iterated denoising [AXR+15] and multiscale image decompositions [TNV04] associated with ROF.
***
A copy of the thesis can be found here: thesis_July26th
no comment as of now