Generative Adversarial Network

Input: x, z(distribution)

output: y(complex distribution)

Discriminator

Adversarial (Natural selection)

Generator vs Discriminator

[Fix G, update D] or [Fix D, update G]

Interpolation

Divergence
$G^* = arg \min\limits_G Div(P_G, P_{data})$ Sampling
Training:
$D^* = arg \max\limits_D V(G, D)$
$V(G, D) = E_{y~P_data}[logD(y)] + E_{y~P_G}[\log(1 - D(y))]$
- Wasserstein distance
  Earth mover
  Too many possible moving plans → chose the shortest distance
  $\max\limits_{D\in1-Lipschitz} {E_{y~P_{data}}[D(x)] - E_{y~P_G}[D(x)]}$
  $1-Lipschitz$ → smooth enough
  If not 1-Lipschitz
  $E_{y~P_{data}}[D(x)]$ : Expected value of Discriminator from data ( $\mathbb{E} = \sum\limits_i{y_i p_i}$ )

Evaluation

Image classifier

Collapse
- Mode collapse
  Too restrict
- Mode dropping
  No diversity

Diversity
- Low diversity
- High diversity
Fréchet inception distance
Calculate differences before softmax

Conditional generator

Unpaired data

Unsupervised learning

domain to domain

Cycle GAN