src/eval/README.md

# Eval CLI

Quick reference for the standalone evaluation tools in `src/eval`.

## Commands

- `photo-eval prep`  
  Download a small evaluation dataset (photos or dance) and optionally normalize.

- `photo-eval single`  
  Upload one label set, classify a set of images, and write timestamped CSV + optional summary.

- `photo-eval matrix`  
  Upload multiple label sets and run the same images against each; outputs a matrix CSV + optional summary.

Selection fields:

- Use `--select-domain-n` / `--select-label-n` and `--min-domain-score` / `--min-label-score` to control
  which tags show up in `selected_domains` and `selected_labels` columns.

## API endpoints used

All eval commands call the running API (default `http://localhost:7860`):

- `POST /api/v1/label-sets` (upload a label set)
- `POST /api/v1/label-sets/{label_set_hash}/activate` (optional for `single` via `--activate`)
- `POST /api/v1/classify` (classify image with optional `label_set_hash`)

## Examples

```bash
photo-eval prep --out data_eval --target photos --n 50 --normalize
photo-eval prep --out data_eval --target dance --n 50 --normalize

photo-eval single \
  --label-set label-dataset/personal-photos-lite-v1.json \
  --images data_eval/photos/normalized \
  --out-dir data_results \
  --summary

photo-eval matrix \
  --label-sets "label-dataset/personal-photos-*.json" \
  --images data_eval/photos/normalized \
  --out-dir data_results \
  --summary
```