Image Fundamentals & Preprocessing — AI Tutorials

Images as Tensors

A digital image is a 3-dimensional array (tensor) of shape (H, W, C):

H = height in pixels
W = width in pixels
C = number of channels (3 for RGB, 1 for grayscale)

Each element is a pixel value — typically an 8-bit integer in [0, 255] for uint8 images or a float in [0.0, 1.0] after normalisation.

Channel Ordering

Different libraries use different channel conventions:

Library	Default order	Tensor shape
PIL / torchvision	RGB	(H, W, C) → then (C, H, W)
OpenCV	BGR	(H, W, C)
PyTorch	RGB	(C, H, W)
TensorFlow	RGB	(H, W, C)

Always be explicit when converting between libraries — a common bug is feeding BGR images to a model trained on RGB.

Color Spaces

Space	Channels	Best for
RGB	Red, Green, Blue	Display, general DL
BGR	Blue, Green, Red	OpenCV default
HSV	Hue, Saturation, Value	Color-based segmentation, augmentation
LAB	Lightness, a, b	Perceptually uniform distance, CLAHE
Grayscale	Intensity	Edge detection, medical imaging

Geometric Transforms

Geometric transforms alter the spatial layout of pixels.

Transform	Parameters	Use case
Resize	target (H, W)	Fixed input size for models
RandomCrop	crop size	Training augmentation
CenterCrop	crop size	Validation / inference
RandomHorizontalFlip	probability	Doubles effective dataset
Rotate	angle range	Rotational invariance
Affine	scale, shear, translate	Geometric robustness

Advanced Augmentation

MixUp

Linearly interpolates two training samples:

$\tilde{x} = \lambda x_i + (1-\lambda) x_j, \quad \tilde{y} = \lambda y_i + (1-\lambda) y_j$

where $\lambda \sim \text{Beta}(\alpha, \alpha)$ with $\alpha = 0.2$ typically.

CutMix

Pastes a rectangular patch from one image into another:

$\tilde{x} = \mathbf{M} \odot x_i + (1-\mathbf{M}) \odot x_j$

where $\mathbf{M}$ is a binary mask. Labels are mixed proportional to patch area.

RandAugment

Applies $N$ randomly selected transforms from a predefined list, each at magnitude $M$ . Eliminates manual augmentation search.

Loading and inspecting imagespython

import numpy as np
from PIL import Image
import cv2
import torch
import torchvision.transforms.functional as TF

# PIL (RGB, HWC uint8)
pil_img = Image.open("photo.jpg").convert("RGB")
print(f"PIL size: {pil_img.size}")          # (W, H)
arr = np.array(pil_img)
print(f"numpy shape: {arr.shape}")          # (H, W, 3)

# OpenCV (BGR, HWC uint8)
bgr = cv2.imread("photo.jpg")
rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
hsv = cv2.cvtColor(bgr, cv2.COLOR_BGR2HSV)
lab = cv2.cvtColor(bgr, cv2.COLOR_BGR2LAB)

# Convert to PyTorch tensor (CHW, float [0,1])
tensor = TF.to_tensor(pil_img)       # (3, H, W), float32
print(f"Tensor shape: {tensor.shape}, range: [{tensor.min():.2f}, {tensor.max():.2f}]")

# Histogram Equalisation with CLAHE (LAB space)
def clahe_equalise(bgr_img: np.ndarray) -> np.ndarray:
    lab = cv2.cvtColor(bgr_img, cv2.COLOR_BGR2LAB)
    l, a, b = cv2.split(lab)
    clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
    l_eq = clahe.apply(l)
    lab_eq = cv2.merge([l_eq, a, b])
    return cv2.cvtColor(lab_eq, cv2.COLOR_LAB2BGR)

Albumentations augmentation pipelinepython

import albumentations as A
from albumentations.pytorch import ToTensorV2
import numpy as np
from PIL import Image
import torch

IMAGENET_MEAN = [0.485, 0.456, 0.406]
IMAGENET_STD  = [0.229, 0.224, 0.225]

# Training pipeline
train_transform = A.Compose([
    A.RandomResizedCrop(height=224, width=224, scale=(0.08, 1.0), ratio=(0.75, 1.33)),
    A.HorizontalFlip(p=0.5),
    A.ColorJitter(brightness=0.4, contrast=0.4, saturation=0.4, hue=0.1, p=0.8),
    A.ToGray(p=0.2),
    A.GaussianBlur(blur_limit=(3, 7), p=0.1),
    A.Normalize(mean=IMAGENET_MEAN, std=IMAGENET_STD),
    ToTensorV2(),
])

# Validation pipeline
val_transform = A.Compose([
    A.Resize(256, 256),
    A.CenterCrop(224, 224),
    A.Normalize(mean=IMAGENET_MEAN, std=IMAGENET_STD),
    ToTensorV2(),
])

img_np = np.array(Image.open("photo.jpg").convert("RGB"))  # HWC uint8
result = train_transform(image=img_np)
tensor = result["image"]   # CHW float32 tensor
print(tensor.shape, tensor.dtype)   # torch.Size([3, 224, 224]) torch.float32

# MixUp implementation
def mixup_data(x: torch.Tensor, y: torch.Tensor, alpha: float = 0.2):
    lam = torch.distributions.Beta(alpha, alpha).sample().item() if alpha > 0 else 1.0
    idx = torch.randperm(x.size(0))
    mixed_x = lam * x + (1 - lam) * x[idx]
    return mixed_x, y, y[idx], lam

def mixup_criterion(criterion, pred, y_a, y_b, lam):
    return lam * criterion(pred, y_a) + (1 - lam) * criterion(pred, y_b)

# CutMix implementation
def cutmix_data(x: torch.Tensor, y: torch.Tensor, alpha: float = 1.0):
    B, C, H, W = x.shape
    lam = torch.distributions.Beta(alpha, alpha).sample().item()
    idx = torch.randperm(B)
    cut_ratio = (1.0 - lam) ** 0.5
    cut_h, cut_w = int(H * cut_ratio), int(W * cut_ratio)
    cx, cy = torch.randint(W, (1,)).item(), torch.randint(H, (1,)).item()
    x1, x2 = max(cx - cut_w // 2, 0), min(cx + cut_w // 2, W)
    y1, y2 = max(cy - cut_h // 2, 0), min(cy + cut_h // 2, H)
    mixed_x = x.clone()
    mixed_x[:, :, y1:y2, x1:x2] = x[idx, :, y1:y2, x1:x2]
    actual_lam = 1 - (x2 - x1) * (y2 - y1) / (H * W)
    return mixed_x, y, y[idx], actual_lam

Knowledge check

A PyTorch model trained on ImageNet expects inputs normalised with ImageNet stats. You load an image with OpenCV. What is the correct processing order?

Knowledge check

MixUp training produces labels like [0.7, 0.3] for a two-class problem. What does this mean?

Knowledge check

What is the key advantage of CutMix over MixUp?

← Computer Vision