src/eval/README.md

Eval CLI

Quick reference for the standalone evaluation tools in src/eval.

Commands

• photo-eval prep
  Download a small evaluation dataset (photos or dance) and optionally normalize.

• photo-eval single
  Upload one label set, classify a set of images, and write timestamped CSV + optional summary.

• photo-eval matrix
  Upload multiple label sets and run the same images against each; outputs a matrix CSV + optional summary.

Selection fields:

• Use --select-domain-n / --select-label-n and --min-domain-score / --min-label-score to control which tags show up in selected_domains and selected_labels columns.

API endpoints used

All eval commands call the running API (default http://localhost:7860):

• POST /api/v1/label-sets (upload a label set)
• POST /api/v1/label-sets/{label_set_hash}/activate (optional for single via --activate)
• POST /api/v1/classify (classify image with optional label_set_hash)

Examples

photo-eval prep --out data_eval --target photos --n 50 --normalize
photo-eval prep --out data_eval --target dance --n 50 --normalize

photo-eval single \
  --label-set label-dataset/personal-photos-lite-v1.json \
  --images data_eval/photos/normalized \
  --out-dir data_results \
  --summary

photo-eval matrix \
  --label-sets "label-dataset/personal-photos-*.json" \
  --images data_eval/photos/normalized \
  --out-dir data_results \
  --summary