benchmarks/README.md

Face Restoration Benchmark

This directory is the fixed evaluation set for model-quality decisions. Images in inputs/ and references/ are intentionally ignored by Git because they may contain private portraits. Commit only manifest.csv metadata if it does not expose sensitive information.

Setup

1. Copy manifest.example.csv to manifest.csv.

2. Add 300-500 held-out portrait examples. Each input_path is the degraded input. Where available, reference_path is its aligned high-quality target.

3. Keep the same manifest for the baseline and every candidate checkpoint.

4. Run the validation gate before an evaluation or training decision:

   python tools/evaluate_restoration.py --manifest benchmarks/manifest.csv --dry-run
   

The benchmark must never overlap the training dataset. Use category to make sure blur, compression, low-light, old-photo and severe-damage cases are all represented.

Acceptance gate

A candidate may advance only if it has no regression in the identity review, does not regress median PSNR/SSIM where reference images exist, and is preferred by the side-by-side human review for artifacts around eyes, skin, teeth and hair.