GANs & Variational Autoencoders — AI Tutorials

Generative Adversarial Networks

A GAN (Goodfellow et al., 2014) consists of two networks in a minimax game:

Generator $G$ : maps noise $z \sim p(z)$ → fake image $G(z)$
Discriminator $D$ : classifies images as real (1) or fake (0)

Minimax objective:

$\min_G \max_D \; \mathbb{E}_{x \sim p_{data}}[\log D(x)] + \mathbb{E}_{z \sim p_z}[\log(1 - D(G(z)))]$

In practice, $G$ maximises $\mathbb{E}[\log D(G(z))]$ (non-saturating loss) to avoid vanishing gradients early in training.

Common Training Challenges

Problem	Symptom	Fix
Mode collapse	Generator produces only a few modes	Minibatch discrimination, spectral norm
Vanishing gradients	Generator loss saturates	LSGAN or Wasserstein loss
Training instability	Oscillating losses	Gradient penalty (WGAN-GP), LR balance
Checkerboard artifacts	High-freq patterns in images	Resize + conv instead of transposed conv

Variational Autoencoders

A VAE (Kingma & Welling, 2013) learns a generative model $p_\theta(x|z)p(z)$ with encoder $q_\phi(z|x)$ .

ELBO (Evidence Lower Bound): $\mathcal{L} = \mathbb{E}_{q_\phi(z|x)}[\log p_\theta(x|z)] - D_{KL}(q_\phi(z|x) \| p(z))$

Reparameterisation trick: Write $z = \mu + \sigma \odot \epsilon$ , $\epsilon \sim \mathcal{N}(0, I)$ to allow backprop through the sampling step.

KL divergence (closed form for Gaussian): $D_{KL} = -\frac{1}{2} \sum_j (1 + \log \sigma_j^2 - \mu_j^2 - \sigma_j^2)$

StyleGAN2 Innovations

Mapping network: 8-layer MLP maps $z \to w \in \mathcal{W}$ (disentangled intermediate space)
AdaIN: Style vector $w$ modulates feature statistics at each layer
Path length regularisation: Smooth latent space for controllable editing

Evaluation Metrics

Fréchet Inception Distance (FID): $\text{FID} = \|\mu_r - \mu_g\|^2 + \text{Tr}(\Sigma_r + \Sigma_g - 2(\Sigma_r \Sigma_g)^{1/2})$

Lower FID = better quality + diversity. FID < 10 is considered high quality for faces.

DCGAN implementationpython

import torch
import torch.nn as nn

NZ, NGF, NDF, NC = 100, 64, 64, 3

class Generator(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            nn.ConvTranspose2d(NZ,   NGF*8, 4, 1, 0, bias=False), nn.BatchNorm2d(NGF*8), nn.ReLU(True),
            nn.ConvTranspose2d(NGF*8,NGF*4, 4, 2, 1, bias=False), nn.BatchNorm2d(NGF*4), nn.ReLU(True),
            nn.ConvTranspose2d(NGF*4,NGF*2, 4, 2, 1, bias=False), nn.BatchNorm2d(NGF*2), nn.ReLU(True),
            nn.ConvTranspose2d(NGF*2,NGF,   4, 2, 1, bias=False), nn.BatchNorm2d(NGF),   nn.ReLU(True),
            nn.ConvTranspose2d(NGF,  NC,    4, 2, 1, bias=False), nn.Tanh(),
        )
    def forward(self, z): return self.net(z)

class Discriminator(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            nn.Conv2d(NC,    NDF,   4, 2, 1, bias=False), nn.LeakyReLU(0.2, True),
            nn.Conv2d(NDF,   NDF*2, 4, 2, 1, bias=False), nn.BatchNorm2d(NDF*2), nn.LeakyReLU(0.2, True),
            nn.Conv2d(NDF*2, NDF*4, 4, 2, 1, bias=False), nn.BatchNorm2d(NDF*4), nn.LeakyReLU(0.2, True),
            nn.Conv2d(NDF*4, NDF*8, 4, 2, 1, bias=False), nn.BatchNorm2d(NDF*8), nn.LeakyReLU(0.2, True),
            nn.Conv2d(NDF*8, 1,     4, 1, 0, bias=False), nn.Sigmoid(),
        )
    def forward(self, x): return self.net(x).view(-1)

def train_dcgan(dataloader, epochs=100, lr=2e-4, device='cuda'):
    G, D = Generator().to(device), Discriminator().to(device)
    criterion = nn.BCELoss()
    opt_G = torch.optim.Adam(G.parameters(), lr=lr, betas=(0.5, 0.999))
    opt_D = torch.optim.Adam(D.parameters(), lr=lr, betas=(0.5, 0.999))

    for epoch in range(epochs):
        for real_imgs, _ in dataloader:
            real_imgs = real_imgs.to(device)
            B = real_imgs.size(0)
            real_labels = torch.ones(B, device=device)
            fake_labels = torch.zeros(B, device=device)

            # Update D
            opt_D.zero_grad()
            noise = torch.randn(B, NZ, 1, 1, device=device)
            fake = G(noise).detach()
            loss_D = (criterion(D(real_imgs), real_labels) + criterion(D(fake), fake_labels)) / 2
            loss_D.backward(); opt_D.step()

            # Update G (non-saturating)
            opt_G.zero_grad()
            noise = torch.randn(B, NZ, 1, 1, device=device)
            loss_G = criterion(D(G(noise)), real_labels)
            loss_G.backward(); opt_G.step()

        if (epoch+1) % 10 == 0:
            print(f"Epoch {epoch+1}  D={loss_D.item():.4f}  G={loss_G.item():.4f}")

Convolutional VAE with reparameterisationpython

import torch
import torch.nn as nn
import torch.nn.functional as F

class ConvVAE(nn.Module):
    def __init__(self, latent_dim=128):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Conv2d(3, 32,  4, 2, 1), nn.ReLU(),
            nn.Conv2d(32, 64, 4, 2, 1), nn.ReLU(),
            nn.Conv2d(64, 128,4, 2, 1), nn.ReLU(),
            nn.Conv2d(128,256,4, 2, 1), nn.ReLU(),
        )
        self.fc_mu      = nn.Linear(256*4*4, latent_dim)
        self.fc_log_var = nn.Linear(256*4*4, latent_dim)
        self.fc_decode  = nn.Linear(latent_dim, 256*4*4)
        self.decoder = nn.Sequential(
            nn.ConvTranspose2d(256,128,4,2,1), nn.ReLU(),
            nn.ConvTranspose2d(128, 64,4,2,1), nn.ReLU(),
            nn.ConvTranspose2d(64,  32,4,2,1), nn.ReLU(),
            nn.ConvTranspose2d(32,   3,4,2,1), nn.Sigmoid(),
        )

    def encode(self, x):
        h = self.encoder(x).flatten(1)
        return self.fc_mu(h), self.fc_log_var(h)

    def reparameterise(self, mu, log_var):
        if self.training:
            return mu + (0.5 * log_var).exp() * torch.randn_like(mu)
        return mu

    def decode(self, z):
        return self.decoder(self.fc_decode(z).view(-1, 256, 4, 4))

    def forward(self, x):
        mu, log_var = self.encode(x)
        return self.decode(self.reparameterise(mu, log_var)), mu, log_var

def vae_loss(recon, x, mu, log_var, beta=1.0):
    recon_loss = F.binary_cross_entropy(recon, x, reduction='sum')
    kl_loss = -0.5 * torch.sum(1 + log_var - mu.pow(2) - log_var.exp())
    return recon_loss + beta * kl_loss

# FID computation (pip install torchmetrics[image])
# from torchmetrics.image.fid import FrechetInceptionDistance
# fid = FrechetInceptionDistance(feature=2048, normalize=True)
# fid.update(real_imgs, real=True)
# fid.update(fake_imgs, real=False)
# print(f"FID: {fid.compute():.2f}")

Knowledge check

What problem does the reparameterisation trick solve in VAE training?

Knowledge check

In GAN training, what is "mode collapse"?

Knowledge check

FID (Fréchet Inception Distance) measures:

← Computer Vision