mini-diffusion — an 18.5M-parameter DDPM trained overnight on a laptop

A denoising diffusion model written from scratch in plain PyTorch. The U-Net, the noise schedule, four samplers and the training loop are all in the repository — diffusers is never imported.

Code: https://github.com/vous99/mini-diffusion

Trained on one MacBook Pro (M5 Pro, 24 GB, MPS) in a single overnight run. Every number below was measured on that run, not quoted from a paper.

Results

Metric	Value
FID-tv	30.12 (10,000 samples, DDIM-50, EMA weights)
Steps	22,098
Images seen	2.83M (17.4 epochs of CelebA train)
Best val loss	0.0352 (L_simple, MSE on epsilon)
Training time	~9 hours of compute at 87 images/s
Parameters	18,538,371

FID over training: 73.35 (step 2,500) → 50.58 → 43.10 → 39.37 → 37.69 → 35.57 → 34.67 → 35.17 (step 20,000). Most of the gain lands in the first third of the run; the curve flattens after roughly step 15,000.

FID-tv is not the FID of the literature. It uses torchvision's Inception-v3 weights rather than the TensorFlow-ported weights every published FID is built on. The numbers here are internally consistent — valid for comparing checkpoints, samplers and guidance scales — but not directly comparable to a paper's "FID 3.5".

What the model does

• Unconditional face generation at 64×64.
• Conditional generation on six CelebA attributes: Male, Smiling, Young, Eyeglasses, Blond_Hair, Bangs.
• Classifier-free guidance from the same weights — trained with 15% condition dropout against a learned null embedding, so one checkpoint serves both the conditional and unconditional branch.
• Deterministic DDIM with an eta parameter that reproduces ancestral DDPM exactly at eta=1.
• Karras sigma schedule with Heun's method.
• DDIM inversion, and therefore spherical interpolation between two real photographs.

Usage

git clone https://github.com/vous99/mini-diffusion && cd mini-diffusion
pip install -r requirements.txt

python -c "
from huggingface_hub import hf_hub_download
import shutil, os
os.makedirs('ckpt', exist_ok=True)
shutil.copy(hf_hub_download('vous99/mini-diffusion', 'best.pt'), 'ckpt/best.pt')
"

python sample.py                                   # an 8x8 grid
python sample.py --attrs "Male=1,Eyeglasses=1"     # conditional
python sample.py --guidance-sweep                  # the guidance-scale figure
python sample.py --sampler-comparison              # DDPM / DDIM / Heun

Sampling needs no dataset. Retraining does — prepare_data.py rebuilds it from the HuggingFace CelebA shards.

Architecture

U-Net over four resolutions, base_ch=64, channel_mults=(1,2,2,4) → 64@64² 128@32² 128@16² 256@8². Two ResBlocks per level down, three up, self-attention at 16² and 8² and in the middle block, head_dim=32, GroupNorm with 32 groups.

Conditioning is a Linear(6, 256) projection added to the sinusoidal timestep embedding, plus a learned null vector. Stable Diffusion instead cross-attends to a sequence of text tokens; six fixed flags are not a sequence, so addition is the honest analogue — the same mechanism, a simpler carrier.

The output convolution is zero-initialised, so the loss at step 0 is exactly E||eps||² = 1.0000. That single number confirms the target is epsilon, the data is scaled to [-1,1], and the reduction is a mean.

Training details

Schedule	cosine (Nichol & Dhariwal), T=1000
Prediction target	epsilon
Timestep sampling	stratified over the batch, not i.i.d. uniform
Optimizer	AdamW, lr 2e-4, betas (0.9, 0.999), weight decay 0, grad clip 1.0
LR schedule	linear warmup 500 steps → cosine decay to 2e-5
Batch	64 × 2 gradient accumulation = 128 effective
EMA	0.999 with warmup
Dropout	0.0
Augmentation	horizontal flip, p=0.5
Precision	bfloat16 autocast (GroupNorm stays fp32)

Limitations

• 64×64 only. Faces at higher resolution are out of scope for this budget.
• No text conditioning. The condition is six binary flags, not a prompt.
• Aligned frontal portraits only. CelebA is centred faces; profiles, multiple people and full bodies are out of distribution.
• Roughly 15–20% of samples collapse into noise. This is the dominant remaining defect.
• Colour casts. Even the cosine schedule ends at abar_T ≈ 2.4e-9 rather than 0, so the model never trains on pure noise but is handed pure noise at sampling time. With epsilon-prediction this shows up as a per-image brightness and colour bias. It receded substantially over training but has not disappeared; v-prediction with zero terminal SNR is the proper fix.
• This is 4.4% of the compute budget of the DDIM paper's CelebA-64 run. The register is a good 2016 GAN, not "indistinguishable from a photograph".

Licence and intended use

The weights inherit the CelebA licence: non-commercial research use only. This is a study artifact for understanding how diffusion models work, not a product component.

It generates synthetic faces of people who do not exist. Do not use it to impersonate real people or to produce material presented as a genuine photograph of anyone.

Checkpoint contents

best.pt is a torch.save dict with model (raw weights), ema (averaged weights — what you should sample from), cfg (the UNetConfig), diffusion (the DiffusionConfig), attributes (the six conditioning names, in order), iter and val_loss. Load it with checkpoint.py from the GitHub repository.