# DECISIONS.md

Running log of implementation decisions where the spec (`PROJECT_SPEC_dog_reunification_V1.md`)
left room, plus anything discovered that contradicts the spec. Newest entries at the bottom of
each section.

## Implementer choices (from spec §19)

- **Frontend styling: Tailwind CSS.** Faster to build responsive/mobile-first UI; utility classes
  keep styling co-located with components. (spec §5, §19)
- **Auth: JWT access tokens** (Bearer), password hashing via `passlib[bcrypt]`. Stateless, simple to
  consume from the SPA, and matches the OpenAPI/codegen story. Refresh tokens deferred. (spec §11, §19)
- **Dog-detector crop: planned, never built.** The intent was an optional crop-to-dog pass before
  embedding. It was not needed once the embedder was fine-tuned on full-body photos as well as face
  crops, so no detector module exists and `Picture.is_probably_not_dog` is never set. (spec §8, §19)

## Decisions where the spec left room

- **Python 3.13** is used (local interpreter); spec asked for 3.11+. No incompatibilities expected.
- **Finders without accounts do not create a `users` row.** Their contact lives on the `cases` row
  (`finder_name/finder_email/finder_phone`), per spec §7.1 note. De-dupe into lightweight finder users
  deferred. (spec §7.1)
- **Embeddings stored in their own `embeddings` table** (per spec §7.5/§7.6 refinement), vector as
  float32 BLOB, L2-normalized at write time so cosine == dot product.
- **Single config object** via `pydantic-settings`, loaded from `.env`. All thresholds/radius levels are
  named config values with documented defaults (spec §0, §15).

## Resolved spec questions

- `REVIEW_THRESHOLD` / `STRONG_THRESHOLD` govern the **case matcher** only. The public photo search
  returns the top N ranked results with no cutoff, deliberately: a page of low scores tells a
  searcher their dog is probably not in the database, where an empty page is ambiguous. Retune with
  `scripts/eval_matching.py` if the case workflow is put into real use (spec §9.3, §9.6).
- `RADIUS_LEVELS` default `[0, 10, 25, 50, 100, -1]` where `-1` == nationwide (no filter).
- Nearby-shelter lookup is a bundled static list for MVP (spec §12).

## Estimated-breed candidate gate (extends spec §9.3 metadata gate)

- **New swap point: `BreedClassifier`** (`backend/app/ml/breed.py`), structured exactly like
  `Embedder`: `Protocol` + `MockBreedClassifier` default + lazily-imported `HFBreedClassifier` +
  `get_breed_classifier()`/`reset_breed_classifier_cache()` singletons. It produces **breed labels
  only**, used purely as a candidate pre-filter — never for similarity (that matcher is selected
  separately).
- **Predictions stored in their own versioned `breed_predictions` table** (mirrors `embeddings`):
  per-picture, keyed by breed-model `name`/`version`, one row per top-K rank. Match time uses only
  the *active* breed model's rows; other versions count as "no predictions" → fail-open.
- **Fail-open, symmetric top-K gate** (open choice #1: chose **symmetric** for recall). A candidate
  is excluded **only on a definite conflict**: both query and candidate have predictions under the
  active model and their top-K label sets do not intersect. Missing predictions on either side
  never exclude — same recall-first discipline as `_metadata_compatible` and `geo.within_radius`.
  The gate runs after the geo + color/size gates and **before** the cosine work, so it prunes the
  expensive comparison. `BREED_FILTER_ENABLED=false` disables it entirely (predictions still
  stored).
- **`BREED_TOP_K` default 7** (open choice #3; anywhere in 5–10), exposed as config to tune vs recall.
- **Human-entered breed is deliberately not folded in** (open choice #2: **no**, settled).
  `KnownDog.breed` / `UnknownDog.est_breed` are free text, and owners are often wrong about mixed
  breeds. A confident-but-wrong human label would exclude the correct dog outright, so the gate stays
  purely model-driven to keep it recall-first.
- **Mock-first / HF opt-in.** Default `BREED_CLASSIFIER=mock` is deterministic and weight-free
  (reuses the mock embedder's pixel hashing so identical images yield identical breed sets — this
  keeps the seeded Rex match working). The real HF model
  (`jhoppanne/Dogs-Breed-Image-Classification-V1`, ~43 MB) is enabled only via config and lazily
  imports `torch`/`transformers`, exactly like the real embedders. Labels are read from
  `model.config.id2label` at load time (no hardcoded breed list).
- **Best-effort in the upload pipeline.** Breed prediction is wrapped in try/except so an upload
  never fails if the breed model errors or isn't downloaded; the embedding stays required.
- **Migrations:** `versions/` previously relied on `create_all`; added a baseline autogenerated
  migration covering the full schema including `breed_predictions`. Existing dev DBs predate the
  table — delete `backend/data/app.db` and re-seed, **or** run `alembic upgrade head` then
  `python -m scripts.process_dataset --all` to populate predictions for already-uploaded photos.

## Batch loading & test-data management (datasets)

- **New `datasets` table + nullable `dataset_id` FK** on `known_dogs`, `unknown_dogs`, `users`
  (Alembic `31362988755b`, chained on the `a18bdb0cac22` baseline; batch-mode ALTERs with **named**
  FK constraints since SQLite batch mode requires names). Cases/matches/pictures link transitively
  through the dogs/users, not via their own `dataset_id`.
- **Purge is application-level and transactional**, not DB-cascade. Pictures link to dogs by
  convention (`subject_type`/`subject_id`, no FK) and `embeddings`/`breed_predictions` FK pictures
  with no `ON DELETE`, so a clean purge deletes children before parents in one transaction
  (notifications → breed_predictions/embeddings → matches → pictures → cases → unknown_dogs →
  known_dogs → users → dataset) and returns per-type counts. Everything-or-nothing via
  commit/rollback. (`services/datasets.py`)
- **Purge also deletes media files.** The picture storage keys are collected *before* the rows are
  deleted, then the files are removed *after* the DB commit succeeds (best-effort: a failed unlink
  only leaves an orphan, never inconsistent data; a rolled-back purge deletes nothing). Count
  returned as `media_files`. Earlier builds left media on disk (the storage step was a no-op), which
  accumulated orphans across purges/re-seeds; `scripts/prune_media.py` removes any orphans not
  referenced by a `pictures` row (dry-run by default) and stays as a safety net for manual DB resets.
- **One dog per folder, multiple pictures (default).** The CSV `folder` column is the identity key
  (each DogFaceNet subfolder == one real dog). The loader **groups a folder's rows into a single
  `KnownDog`/`UnknownDog`** and attaches every image as a `Picture` (first = primary); a known dog
  gets one optional lost case, a found dog one found `Case`. This supersedes the original "one row ==
  one dog" behavior, which created a separate same-named dog per photo — wrong for multi-photo
  identities and bad for matching (1-picture dogs). Pass `group_by_folder=False` /
  `--one-dog-per-image` for the legacy per-row behavior. Owners/finders are still de-duped by email
  and linked to the dataset; `found_zip` equals the dog's known ZIP so default-radius (level 0 = same
  ZIP) matching works. **Note:** datasets loaded before this fix have one dog per image — purge and
  re-load them.
- **Holdout default:** 1 image, or 2 when a dog has 5+ images and `--holdout` is not given; always
  leaves ≥1 registration image (a single-image dog yields 0 holdouts, logged).
- **`--mark-lost` also opens a lost `Case`** per marked known dog (status `lost`, `last_known_zip`
  set) so the known dataset is actually matchable; `--mark-lost-pct` controls the share (default 100).
- **Embedding reuse:** extracted `embed_picture()` in `services/images.py`, used by both the upload
  pipeline and `embed-all` — embedding generation is never reimplemented.
- **Many-to-many match:** `run_matching_for_case` gained an optional `candidate_dataset_id` to scope
  the candidate pool; the `/admin/datasets/{id}/match?candidate_dataset_id=` endpoint passes it through.
- **Progress feedback = polling.** No realtime channel existed, so dataset loads run in a worker
  thread tracked by a process-local in-memory job registry (`services/jobs.py`); the dashboard polls
  `GET /admin/jobs/{id}`. Ephemeral by design (single-process local target, A8); a durable queue is a
  documented future swap.
- **Schema change ⇒ re-seed.** Existing dev DBs lack `dataset_id`; `create_all` does not ALTER, so
  delete `backend/data/app.db` and re-seed, or `alembic upgrade head`.

## Storage-only load (defer embeddings)

- `process_and_store_picture` gained `generate_embedding` / `generate_breed` flags (default `True`,
  so normal uploads/seed are unchanged). The batch loader's new `skip_embeddings` option sets both
  to `False` — images are validated, normalized (EXIF stripped), stored, and thumbnailed, but **no
  embeddings or breed predictions are generated**. Exposed as `--skip-embeddings` (CLI),
  `skip_embeddings` (load-job form), and a "Load images into storage only" checkbox in the
  dashboard. Matching needs embeddings, so "Run matching after load" is force-disabled when skipping.
- **Batch embedding generator:** the existing `POST /admin/datasets/{id}/embed-all` (idempotent —
  only embeds pictures lacking an embedding under the active model) is now surfaced as an **"Embed"
  button per dataset row on the Data Loading page**, next to "Purge" (in addition to the dataset
  detail page's "Run Embeddings"). This is the intended two-step flow: bulk-load into storage now,
  generate embeddings later. `embed-all` runs synchronously; fine for the mock embedder. (If a real
  CNN embedder on very large datasets makes the request too long, move `embed_all` onto the
  background job system used by load — a documented future step.)

## Real re-ID embedder: HuggingFace model (EMBEDDER=hf)

- **New `HFEmbedder`** (`backend/app/ml/embedder.py`) behind the existing `Embedder` swap point —
  no API/schema changes. It loads an HF image-classification model
  (`EMBEDDER_HF_MODEL`, default `jhoppanne/Dogs-Breed-Image-Classification-V1`) and uses its
  **penultimate, pre-classifier pooled features** as the re-ID vector: last hidden state →
  global-average-pool (CNN feature maps) or mean-over-tokens (transformer sequences) → L2-normalize.
  Breed *labels* are NOT used for similarity — that stays the separate `BreedClassifier`. Embedding
  `dim` is inferred at load with one dummy forward (architecture-agnostic).
- **torch/transformers lazily imported**, exactly like `HFBreedClassifier`; default
  `EMBEDDER=mock` still needs none of it. torch is installed CPU-only.
- **Versioning makes the swap safe:** embeddings are tagged `model_name="hf-embed"`,
  `model_version=<repo name>`. Matching only compares embeddings from the *active* model, so after
  switching `EMBEDDER=mock→hf` you must regenerate vectors (admin **embed-all**); old mock vectors
  are simply ignored, not mixed.
- **Same model, two roles:** this repo can use the model both as the breed gate
  (`BREED_CLASSIFIER=hf`, labels) and the re-ID embedder (`EMBEDDER=hf`, features). They load
  independent copies for interface cleanliness; sharing one load is a possible future optimization.
- **Perf note:** CPU embedding of the full DogFaceNet set (~8k images) via `embed-all` is slow and
  runs synchronously today — fine to run once, but a background-job version of `embed-all` is the
  documented next step for large real-model runs.

## Breed filtering (HF softmax labels)

- **Same HF model, the classifier head this time.** `BreedClassifier=hf` runs
  `jhoppanne/Dogs-Breed-Image-Classification-V1`'s softmax and stores the top-K labels in
  `breed_predictions` (separate from the re-ID *embeddings*). `BREED_TOP_K=10` so the UI can filter
  by top-1 through top-10.
- **Filter semantics (per-photo, recall-friendly):** a dog matches breed B at depth k if **any of
  its photos** predicts B within rank `< k`. `breed_k=1` = "top match"; 5/10 = top-K. Displayed
  `predicted_breeds` is the dog-level aggregate (distinct labels ordered by best rank then score
  across the dog's photos), so a dog matched on one photo's top-1 may show a different label first —
  expected for multi-photo dogs.
- **Active breed model is auto-detected from the DB** (`active_breed_model_in_db`, prefers non-mock,
  most-used) — reading predictions never loads the HF model. `estimated_breeds_for` was refactored
  to use this too, so match-candidate hydration no longer risks loading the breed model.
- **Endpoints:** `GET /admin/breeds` (distinct labels + counts) and
  `GET /admin/dogs?breed=&breed_k=` (filter). Profiles now include `predicted_breeds`.
- **Tests pin `BREED_CLASSIFIER=mock`** (conftest) so the suite never loads the HF model; the `.env`
  default is `hf` for real use. Generate predictions with `python -m scripts.process_dataset --all`
  (`--limit N` for a partial run).

## Public "search by photo" (home page)

- **What:** anonymous visitors can upload a dog photo (+ optional ZIP) on the home page and get the
  found/unknown dogs already in the system ranked by photo similarity. `POST /search/by-photo`
  (multipart `file`, optional `zip`, `top_k`). No auth, no DB writes.
- **Read-only by design:** it is a *lookup*, not a report. It never creates a `KnownDog`/`Case`, so
  it doesn't pollute the matching pipeline or notify anyone. The copy steers users who find a likely
  match to the real "report my dog as lost" flow (where confirmed, mediated matching happens).
- **Transient embedding:** `images.embed_bytes()` runs the *same* validate→normalize→encode pipeline
  as stored uploads, writes to a `tempfile`, embeds via the active model, deletes the temp file. The
  query photo is never persisted — same vector a stored picture would get, zero footprint.
- **Scoring reuses the matcher:** `matching.search_unknowns_by_vectors()` scores the active
  found/unknown pool (`pending|lost|at_shelter`) by the same dog-level **max cosine** aggregate the
  case matcher uses (`_dog_level_score`). No `REVIEW_THRESHOLD` cut here — this is a "show me the
  closest" ranked list, so we return the top-K regardless of absolute score (recall-first; the model
  eval showed scores run uniformly high, so a threshold would mostly mislead).
- **ZIP is optional + fail-open:** with a ZIP we scope to `DEFAULT_SEARCH_RADIUS=100` mi (fail-open
  on unknown ZIPs, like the case matcher) and surface per-result `distance_miles`; empty ZIP =
  nationwide. Results are always ranked by score, not distance.
- **Results carry their own photos** so the UI viewer (`PhotoTools` modal) shows every angle
  without a separate auth'd detail endpoint (the admin `/admin/dog/...` route stays admin-only).
- **Tests:** `backend/tests/test_search.py` (ranking, no-auth, persists-nothing, ZIP distance, bad
  image) and `frontend/src/components/__tests__/PhotoTools.test.tsx` (upload→render, photo viewer).

## Public breed-estimation toy (home page)

- **What:** a for-fun home-page widget — upload a photo, get the top N (3/5/7/10) breed labels.
  `POST /search/breed` (multipart `file`, `top_n`). No auth, no DB writes.
- **Reuses the breed swap point, not a new model:** `images.predict_breeds_bytes()` mirrors
  `embed_bytes` (same validate→normalize→encode→tempfile→cleanup) and calls the active
  `BreedClassifier` (`get_breed_classifier`). With `BREED_CLASSIFIER=hf` it's the real softmax; with
  `mock` (tests/offline) it's deterministic pseudo-labels. The endpoint slices to `top_n`, so the
  practical max is bounded by `BREED_TOP_K` (10 in `.env`).
- **Distinct from the match-time breed gate:** this is display/entertainment only and never touches
  matching; it just exposes the classifier the gate already uses.
- **Note:** the home page presents this together with photo-search in a single `PhotoTools` widget —
  one upload, a large centered preview, and buttons to run matching, breed estimation, or both.
- **Tests:** `test_search.py` (top_n length, deterministic labels, clamp to 10, bad image) +
  `frontend/src/components/__tests__/PhotoTools.test.tsx` (upload→ranked list, run-both).

## ZIP filtering fix (geo centroids were missing)

- **Bug:** photo-search ZIP filtering (and case-match radius) silently did nothing. Two causes:
  (1) the app default `zip_centroid_file` was CWD-relative (`./data/...`), so launching uvicorn from
  `backend/` looked in `backend/data/` and loaded **0 centroids**; (2) the shipped centroid CSV was a
  31-row national demo set, while the DogFaceNet demo data uses ~480 real Texas ZIPs — **0** of which
  had centroids. `within_radius` fails open on unknown ZIPs, so every candidate passed → ZIP no-op,
  distance always null.
- **Fix 1 (path):** `config.zip_centroid_file` now defaults to an **absolute** path resolved from the
  repo root (`parents[2]/data/zip_centroids.csv`), so it's found regardless of CWD. `.env` /
  `ZIP_CENTROID_FILE` still overrides.
- **Fix 2 (coverage):** a one-off generator produced `data/zip_centroids.csv` (539
  rows) covering the national demo ZIPs (kept verbatim so matching tests' distances are unchanged)
  plus every Texas ZIP in `prepare_dogfacenet.ZIP_CODES`, mapped to a **metro-level** centroid by
  ZIP3 prefix. Inter-metro distances are realistic (Houston↔Dallas ≈ 225 mi) so radius filtering
  works; intra-metro distance ≈ 0. A real per-ZIP gazetteer is a drop-in CSV replacement later.
- **Verified:** 1395/1395 demo dogs now have centroids; a Houston search at 100 mi keeps 382/1395
  (drops other metros). Test: `test_search_by_photo_zip_excludes_out_of_radius`.

## One model pass per stored image (embedding + breed together)

- **Problem:** every stored picture forwarded through the model TWICE — once in `HFEmbedder.embed`
  (penultimate features) and once in `HFBreedClassifier.predict` (logits) — even though both use the
  *same* model (`jhoppanne/Dogs-Breed-Image-Classification-V1`). Batch workflows were worse: running
  `embed_dataset` then `backfill_breeds` = two full passes over the dataset.
- **Fix:** `HFEmbedder.embed_and_breed(paths, top_k)` does ONE forward with `output_hidden_states=True`
  and returns both the pooled re-ID vector (from `hidden_states[-1]`) and the softmax breed labels
  (from `logits`). Verified the vector is **identical** to the old standalone `embed()` — existing
  embeddings stay valid, no regeneration needed.
- **Orchestration:** `images.embed_and_breed_picture(db, pic)` uses the single pass when
  `_same_hf_model()` (embedder=hf, breed=hf, same repo id); otherwise falls back to the two separate
  models (mock defaults / mismatched ids), embedding required + breed best-effort. Breed rows are
  tagged via `hf_breed_name_version()` so they match `HFBreedClassifier`'s tags exactly.
- **Wired in:** `process_and_store_picture` (live uploads) calls it when both are requested;
  `datasets.embed_all` (admin "Run Embeddings + Breeds" button) now reports `embedded` + `breeds`;
  and `scripts/process_dataset.py` is the batch path that **replaces** `embed_dataset` +
  `backfill_breeds` (one pass instead of two). `embed_dataset` remains for mock / split configs;
  `backfill_breeds` has since been removed.
- **Scope:** applies to pictures stored *in the system*. The public front-page toys (`embed_bytes`,
  `predict_breeds_bytes`) are intentionally separate one-off queries and out of scope.
- **Tests:** `test_embed_breed_single_pass.py` — a fake same-HF embedder proves the model runs
  exactly once and both rows are written with the right tags; plus the mock-fallback idempotency.

## Purge fix: datasets that share users (reused test emails)

- **Bug:** purging "DogFaceNet known" failed with `FOREIGN KEY constraint failed` on `DELETE FROM
  users` and rolled the whole transaction back (dataset left fully intact). Root cause: the prepare
  scripts mint owner emails `owner1@test.com…`, and `batch_loader._get_or_create_user` dedupes users
  by email — so a second dataset (MPDD) whose CSV reused those emails attached its 96 dogs to the
  FIRST dataset's users. Purging the first dataset then tried to delete users still owning the
  second dataset's dogs.
- **Fix (`purge_dataset`):** a user may be legitimately shared across datasets, so instead of force-
  deleting all of the dataset's users, only delete users with **no surviving references**
  (`KnownDog.owner_id` / `Case.person_id` after the dataset's own rows are gone) and **detach the
  rest** (`dataset_id = NULL`) so the dataset row can be removed without a dangling FK. New
  `users_detached` count is returned. Also extended match cleanup to remove matches whose
  **candidate** is one of the purged dataset's dogs (not just by case), so a cross-dataset match
  can't be left pointing at a deleted dog.
- **Verified live:** purged DogFaceNet known (1393 dogs, 5969 pictures + media, 11938 embeddings,
  59690 breed preds; 1297 users deleted, 96 detached). MPDD (#3/#4) fully intact, all owners valid,
  0 media orphans. "DogFaceNet unknown" purge was checked and had cleaned up correctly (no DB or
  media orphans).
- **Latent data-hygiene note:** reused test emails still ENTANGLE datasets (one user owning dogs in
  several). The purge now tolerates it, but future prep runs (YT-BB-Dog, Sibetan) will keep sharing
  `owner1@test.com…` unless the prepare scripts namespace emails per dataset. Recommended follow-up.
- **Test:** `test_purge_retains_user_shared_with_another_dataset` (shared owner across two datasets).

## Removed color/size as entered information (subjective/unreliable)

- **Why:** color and size are subjective, inconsistently reported, and were pruning candidates in the
  matcher — a wrong color/size guess could hide a correct match.
- **Matching:** deleted the `_metadata_compatible` color/size gate and the `apply_metadata_filter`
  path in `run_matching_for_case`. Matching now gates only on ZIP radius + estimated breed (model
  softmax), then ranks by embedding similarity. (Breed filtering was explicitly wanted; kept.)
- **Inputs removed:** `KnownDogCreate`/`KnownDogUpdate`, `LostCaseCreate.new_dog_color/size`,
  `FoundSightedCreate`, and the `/cases/found` + `/cases/sighted` form params no longer accept
  color/size. Frontend forms (MyDogs add-dog, ReportLost new-dog, ReportFound/Sighted) dropped the
  fields, and the now-always-empty color/size display was removed from the owner views (MyDogs card,
  DogDetail).
- **Deliberately kept:** the `color`/`size` DB columns (nullable) and the `*Out` response schemas —
  no destructive migration, and any already-loaded/legacy data still round-trips. Batch-loader CSVs
  and admin dataset views still carry their (often "unknown") values; these are inert now that
  matching ignores them. FastAPI/Pydantic ignore the extra fields, so old clients/CSVs don't break.
- **Verified:** 71 backend + 14 frontend tests pass, clean typecheck/build.

## Per-dataset + combined re-ID metrics (scripts/eval_datasets.py)

- New read-only eval over the loaded (known+unknown) dataset pairs, auto-detected by name
  ("<base> known" + "<base> unknown"). Reuses DB embeddings (active model).
- **Intra-subject similarity:** per dog, cosine over all pairs of its OWN photos (min/max/avg) →
  `eval_out/intra_subject_similarity.csv`, plus a per-dataset summary and a sampled different-dog
  baseline for context.
- **Retrieval quality:** found dog → ranked known gallery with the production dog-level max-cosine
  score; recall@1/5/10 (+counts), mean/median rank, MRR, true-vs-best-wrong separation. Reported per
  dataset AND combined (shared gallery; the other dataset's dogs become distractors). Identity is
  namespaced per source (`base:folder`) so colliding folder numbers across datasets never cross-match
  → this is how "filter matching by dataset" is realized for the eval. Summary →
  `eval_out/retrieval_summary.csv`.
- Measures the MODEL (pure embedding retrieval, full gallery). Deliberately does NOT apply the
  production ZIP-radius / breed gates: on this synthetic data each dog's found+known share a ZIP, so
  a ZIP gate would inflate recall as an artifact rather than reflect the model.

## Photo-level retrieval / CMC (scripts/eval_photo_retrieval.py)

- Different protocol from eval_datasets.py (which is dog-level: whole found dog vs whole known dogs).
  Here every INDIVIDUAL photo (on the recombined pre-split whole dogs) is a query matched against ALL
  other photos, its own excluded; a hit @k means a same-dog photo is in the top-k. Reported per
  dataset then combined (shared photo gallery; other dataset = distractors). Recall@1/5/10/20 + mean/
  median rank + MRR. Identity namespaced per source so folder-number collisions don't cross-match.
- `_cmc` computes each query's nearest-same-dog rank as `1 + #{sim > s_true}` with s_true and the
  count taken from the SAME similarity row (one matmul) — a first attempt used two matmuls and a
  float mismatch nudged true matches off rank 1 (recall@1 low by ~5pts). Verified elementwise
  identical to a brute-force argsort on MPDD.
- Results (hf-embed): MPDD r@1/5/10/20 = 65.1/81.0/87.3/91.8%; DogFaceNet 62.9/79.8/85.6/90.1%;
  Combined 62.7/79.5/85.3/89.8%. Lower than the dog-level eval (single-photo query is weaker than
  aggregating a whole dog + larger per-photo gallery). Summary -> eval_out/photo_retrieval_cmc.csv.

## Outlier photo removal + re-eval (scripts/remove_outliers.py)

- Quarantined the 22 flagged `removable_outlier` photos (10 MPDD + 12 DogFaceNet) — each the single
  isolated photo whose removal lifts its dog's MIN pairwise similarity back above 0.5. Files moved to
  `data/outlier_photos/` (reversible); Picture+Embedding+BreedPrediction rows deleted. Every affected
  dog kept >=2 photos. Verified 0 DB-keys-missing-on-disk afterward.
- Re-ran both eval scripts on the cleaned data (counts −22 exactly):
  - Consistency: dogs with min<0.5 fell MPDD 15->5, DFN 13->1 (remaining low-min dogs have >1 bad
    photo, so no single removable outlier left).
  - Photo-level CMC improved: MPDD r@1/5/10/20 65.1/81.0/87.3/91.8 -> 66.7/82.6/88.9/93.2; DFN
    62.9/79.8/85.6/90.1 -> 63.0/79.9/85.8/90.2; Combined 62.7/79.5/85.3/89.8 -> 62.9/79.6/85.5/90.0.
    MPDD moved most (small set); DFN barely (12/8363).
- NOTE (pre-existing, unrelated): ~5240 orphan media files on disk with no DB row (from the earlier
  DogFaceNet reload). Invisible to the DB-based evals; cleanup via scripts/prune_media.py when wanted.

## Fix: load progress bar freezes on large batch loads

Two independent causes, both hit only on LONG loads (many polls over many minutes):
- **Frontend (root cause):** the poll's `catch { clearInterval(t) }` permanently stopped polling on a
  SINGLE failed request, so the bar froze at its last value while the load kept running server-side.
  Now tolerates a run of transient failures (~10s / 20 polls) before giving up, and surfaces a "Lost
  connection" error only if the job is truly unreachable (`failsRef` counter).
- **Backend (why polls failed):** SQLite had no `busy_timeout` and used the default rollback journal,
  so the status poll's admin-auth SELECT could transiently hit "database is locked" during the load's
  long write transaction. Added `PRAGMA journal_mode=WAL` (readers don't block on the writer) +
  `PRAGMA busy_timeout=10000`. Bonus: the test suite got ~35% faster under WAL.
- Tests: new "keeps polling through a transient poll failure" AdminData test; all backend/frontend
  green.
- Not changed (documented follow-up): the whole load is still ONE transaction (commit at the end),
  so the SQLAlchemy session grows for very large loads. Periodic commit + expunge would bound memory
  and smooth progress further, at the cost of all-or-nothing semantics.

## Batch encoding progress bar (background embed job)

- The admin "Run Embeddings + Breeds" button was a synchronous blocking POST — no progress, and on a
  large dataset it just spun / could time out. Converted encoding to a background JOB reusing the
  same registry + polling the loader uses.
- Backend: `datasets.embed_all(db, dataset, *, progress=None, commit_every=25)` now commits +
  reports `{processed,total,dataset_id}` every 25 pictures (and `expunge_all` to bound the session)
  instead of one commit at the end — safe because encoding is additive/idempotent. New endpoint
  `POST /admin/datasets/{id}/embed-all-job` runs it in a worker thread and returns a job id; the old
  synchronous `embed-all` endpoint stays for small sets / tests.
- Frontend: shared `useJobPolling` hook (resilient polling, same transient-failure tolerance as the
  load bar) + `JobProgressBar` component. DatasetDetail's encode button now starts the job, shows a
  live bar, and prints the summary + refreshes stats on completion.
- Tests: `test_embed_all_reports_progress` (monotonic progress events), `test_embed_all_job_runs_in_
  background` (start->poll->done), and a DatasetDetail UI test (bar -> summary). All suites green.

## Palette: warm tan/brown theme (replaced generic blue/white)

- The app consistently uses `brand-*` and `gray-*` Tailwind tokens, so retheming is a two-scale
  change in tailwind.config.js — no component rewrites:
  - `brand` → warm tan→coffee-brown scale (was blue). Recolors buttons, links, active nav, focus
    rings, progress bars, and breed chips.
  - `gray` → warmed taupe/sand neutrals (overrides Tailwind's cool default gray). Warms every
    surface, border, and text token the app already uses (page bg gray-50 = warm cream, cards stay
    white for a paper-on-desk feel).
  - New `accent` = a muted dusty-denim blue (deliberately not the default bright blue); the 4
    generic `blue-100/blue-700` "found/unknown/open" badges now use it as a cool counterpoint.
- Contrast checked on the key pairs (brand-600/white ~6.6:1, brand-700 on brand-50, gray-900 on
  gray-50, accent-700 on accent-100) — all comfortably readable. tsc/tests/build green.

## Date plausibility gate in matching

- A dog can't be FOUND before it was LOST, so a lost dog is only compared to found dogs whose found
  date is on/after `lost_date - grace`. Added `_date_compatible(lost_date, found_date)` in
  matching.py, applied (pre-cosine, alongside the ZIP + breed gates) in BOTH directions of
  `run_matching_for_case`: lost-query→found-pool and found-query→lost-pool. Uses each case's
  `event_date` (the user-entered lost/found date, not the report timestamp).
- Recall-first / fail-open: missing dates never exclude; only a definite conflict (found earlier than
  lost minus grace) prunes. Config: `DATE_FILTER_ENABLED=true`, `MATCH_DATE_GRACE_DAYS=2` (grace
  absorbs imprecise dates). The admin test-match (pure re-ID check) intentionally skips this gate.
- Batch-loaded datasets set both cases' event_date to the load date, so their found/lost dates are
  equal → always compatible (dataset matching/eval unaffected).
- Tests: found-before-lost excluded despite identical photo; found within grace still matches.

## Narrow-first case matching + widen the constraints

Directive: searches start narrow (location + top-10 breeds + top-10 matches) then widen. Scope
chosen: lost/found case matching, MANUAL widen button (keep it). What was already true vs added:
- Location: already narrow-first (starts at radius_levels[0]=0) with the manual `/widen` ladder. ✓
- Top-10 breed gate: already applied (BREED_TOP_K=10, breed_filter_enabled). ✓
- Top-10 matches: `TOP_N` was 5 → now **10** (the narrow-pass result cap).
- "Then widen the constraints": the widen ladder previously loosened only location. Now the WIDEST
  rung (nationwide, radius -1) also **drops the breed gate** (`nationwide = radius == -1`; the breed
  gate is skipped there). So the manual widen progresses location → … → breed, with no schema change
  (state derived from the existing `search_radius_miles`). Date/ZIP/threshold gates unchanged.
- Tests: narrow pass caps at 10 matches; breed gate excludes a disjoint-breed dog at radius 0 but
  matches it once widened to nationwide.

## Privacy: shelter/vet location is public (only homes stay private)

- Reversed the earlier rule that hid `current_location_detail` (the shelter/vet holding a found
  dog). A shelter/vet is a public place, and telling a matched owner where to reclaim their dog is
  the point — so it's now included in the hydrated match candidate (hydrate.py) and `UnknownDogOut`.
  Home locations remain protected: never collected below ZIP granularity, contact stays mediated.
- Frontend: MatchCard shows "📍 Being held at <shelter/vet>"; the found-report field is relabeled
  (public, "use the shelter/vet name — not a home address").
- Test flipped: `test_unknown_dog_shelter_shown_but_location_stays_zip_level` asserts the shelter
  name IS returned while location stays ZIP-level. Updated model/schema/spec-§14 comments.

## Removed the "sighted" workflow (found = you have the dog)

- A found report now always means the reporter HAS the dog: in their own custody (`pending`) or
  dropped at a shelter/vet (`at_shelter`). Removed the seen-but-not-caught "sighted" flow entirely.
- Backend: deleted `POST /cases/sighted` and the shared `_create_found_sighted` helper (folded into a
  single `create_found_case`); removed `CaseType.sighted` and the now-unused `UnknownDogStatus.lost`
  ("sighted, not in custody"); dropped `lost` from the found/unknown matching pool filter (both the
  case matcher and the public photo search); renamed `FoundSightedCreate`→`FoundCreate`; updated
  seed.py's far-away demo dog from sighted→found.
- Data migration (raw SQL, enums already removed): 1 `sighted` case → `found`, 1 `lost` unknown →
  `pending`.
- Frontend: removed the `/report/sighted` route + home button; `ReportFound` is found-only (no `kind`
  prop); `api.createReport(kind)`→`api.createFound`; `CaseType` and MatchCard copy updated.
- All backend/frontend tests green; typecheck + build clean.

## Removed seeded demo data (generated-dot "dogs")

- The synthetic `make_sample_images` dogs (Rex, Maple, 2 demo found dogs) and their info were noise
  now that real datasets are loaded. New `scripts/purge_seed_data.py` deletes every non-dataset dog
  (`dataset_id IS NULL`) + its pictures/embeddings/breeds/cases/matches/notifications/media, plus the
  demo owner users — but ONLY non-admin users owning no dataset dog, so the admin and the 96
  detached (dataset_id-NULL) MPDD owners are preserved.
- Ran it: removed 2 known + 2 unknown dogs, 2 owners (alice/bob), 3 cases, 1 match, 4 pictures + 4
  media files. Verified 0 non-dataset dogs remain, admin intact, datasets intact (1489 known + 1489
  unknown), media 8862 = 8862 on disk (0 orphans).
- `scripts/seed.py` slimmed to create ONLY the admin account (no synthetic dogs), so re-seeding can't
  reintroduce them; README updated. `make_sample_images.py` stays (the test suite uses it).
- All backend tests green.

## Generated realistic cases for every loaded dog (one-off script, since removed)

- One LOST case per known dog + one FOUND case per unknown dog, rebuilt cleanly (old cases/matches
  purged first). Pairing is by dataset + folder identity ("DogN" ↔ "Test found dog N").
- Constraints honored: lost dates within the last 2 days; each found date is on/after its paired
  lost date and never in the future; each found ZIP equals the paired lost ZIP or a neighboring ZIP
  (another ZIP sharing the metro centroid — 40% of found reports); known-dog ZIPs (which were null)
  taken from the paired found dog's ZIP.
- Each known dog got a fresh owner `owner{i}` (owner0…owner1488) as its owner + the lost case's
  reporter. The ~2825 old shared/entangled test owners (now orphaned) were removed; admin kept.
- Verified all 1489 pairs: dates in range, found≥lost, all found ZIPs same-metro as their lost ZIP,
  every known dog lost with an owner. Cases are NOT auto-matched (run matching separately if wanted).

## Admin dashboard: all cases, owners, and run-matching-on-demand

- **All Cases** (`/admin/cases`, page + `GET /admin/cases`): every case, newest first, hydrated with
  its dog (thumb/name/kind), reporter (owner for lost, finder for found), ZIP, date, status, and
  match count. Filter by type (all/lost/found), paginated. Replaced the old thin `list[CaseOut]`.
- **Owners** (`/admin/owners`, page + `GET /admin/owners`): each owner with the dogs they own
  (thumb/name/status), paginated.
- **Run matching on demand** (`POST /admin/cases/{id}/run-match`): runs the PRODUCTION matcher
  (`run_matching_for_case` — narrow-first ZIP/breed/date gates, persists ranked Match rows) for a
  chosen case and returns the hydrated matches. Surfaced as a "Run matching" button per row on the
  All Cases page, opening a modal of the ranked candidates (MatchCard). Distinct from Test Matching
  (`/admin/test-match`), which is the non-persisting pure-re-ID diagnostic.
- Admin dashboard links updated (All Cases / All Dogs / Owners / Test Matching / Data Loading).
- Tests: `test_admin_all_cases_and_owners`, `test_admin_run_case_match`. All suites green
  (79 backend, 16 frontend); verified live on the 2978-case dataset.

## Fix: run-matching 500 (matching must not load the model) + case detail view

- **Bug:** `POST /admin/cases/{id}/run-match` 500'd. Root cause: `run_matching_for_case` called
  `get_embedder()` / `get_breed_classifier()` just to read the active model's name/version — which
  loads the HF model over the network. On a server without a HF token that gets rate-limited/offline,
  the load fails → 500. Matching only needs the PRECOMPUTED embeddings, never the model.
- **Fix:** added `active_embed_model_in_db(db)` (mirrors `active_breed_model_in_db`) and switched
  `run_matching_for_case` + `rank_dog_against_opposite` to pick the model from stored rows — no model
  load, works fully offline. Removed the now-dead `_active_breed_model()`. The public photo search
  keeps `get_embedder()` (it must embed the uploaded photo anyway). Verified run-match returns 200
  with the hf-embed vectors under `EMBEDDER=mock HF_HUB_OFFLINE=1` (zero network calls).
- **Case detail:** new `GET /admin/cases/{id}` returns the case + its subject dog (all photos) + its
  persisted matches (each candidate hydrated with photos). All-Cases rows are now clickable →
  CaseDetailModal showing the dog's photos and the ranked matches, with a Run/Re-run matching button
  inside. Tests: `test_admin_case_detail`; all suites green (80 backend, 16 frontend).

## All cases moved to Houston ZIPs

- The case generator drew ZIPs from `HOUSTON_ZIPS` (the 77002–77099 subset of the
  prepare list) instead of the spread-across-Texas dataset ZIPs. Conventions kept: one deterministic
  ZIP per dog (per-folder seeded), found ZIP = same or a neighboring (other Houston) ZIP (~40%),
  owner ZIP = the dog's ZIP, lost dates within the last 2 days, found ≥ lost.
- Owner creation is now idempotent (upsert `owner{i}@example.com`) so re-running doesn't hit the
  email UNIQUE constraint.
- Re-ran: 1489 lost + 1489 found cases, all ZIPs Houston (95 distinct), 0 constraint violations.
- Consequence: all Houston ZIPs share one centroid, so at radius 0 every dog is in range of every
  other — ZIP no longer narrows the pool; matching leans on photos (+ breed/date gates). Matches were
  cleared by the rebuild; re-run matching (per case in the admin All Cases view, or in bulk) if wanted.

## Side-by-side photo comparison (case dog vs each match)

- `MatchCard` now takes optional `queryPhotos` + `queryLabel`. When present, clicking a match's
  thumbnail opens a TWO-COLUMN modal: the case's own dog photos on the left, the candidate match's
  photos on the right (with the % match). Without those props it falls back to the single-dog gallery.
- Owner view (CaseDetail): new owner/admin-authorized `GET /cases/{id}/dog` returns the case's dog
  (kind + all photos); CaseDetail fetches it and passes it to each MatchCard.
- Admin view (AdminCases → CaseDetailModal): passes the case's subject-dog photos (already in the
  detail payload) to each MatchCard. So comparison works in both regular and admin case viewing.
- Tests: `test_case_dog_endpoint_returns_photos_and_guards_owner` (photos + 403 for non-owner). All
  suites green (81 backend, 16 frontend).

## Removed human-entered breed; display the model's estimated breed (top 3)

- Owners/finders are unreliable on breed, so it's no longer collected. Removed the breed input from
  `KnownDogCreate`/`KnownDogUpdate`, `LostCaseCreate.new_dog_breed`, `FoundCreate.est_breed`, the
  `/cases/found` `est_breed` form param, and the frontend forms (MyDogs add-dog, ReportLost new-dog).
- Everywhere breed was shown, it now shows the **top-3 estimated breeds** (model's HF-softmax
  predictions, already surfaced as `estimated_breeds` / `predicted_breeds`): MatchCard (card + modal
  + comparison), DogPhotosModal, PhotoSearch results, AdminDogs cards, AdminTestMatch candidates,
  DogDetail (removed the human "Breed" field), and the ReportLost dog picker. Shared
  `estimatedBreeds()` helper (top-3, "Breed not estimated" fallback).
- DB columns (`KnownDog.breed`, `UnknownDog.est_breed`) kept nullable for backward compat (same as
  color/size); they're just no longer collected or displayed. Matching never used human breed anyway
  (the gate uses the model's `breed_predictions`).
- All suites green (81 backend, 16 frontend), clean build.

## Found-report 500 fix + owner re-run + admin delete buttons

- **Found-report 500 (root cause):** each found report ran `rematch_open_lost_cases_against`, which
  re-ran FULL matching for EVERY open lost case (1490 now) — O(cases×candidates), holding the SQLite
  write lock for minutes → timeout/500 and locking out other requests (the Houston co-location made
  every dog a radius-0 candidate, worsening it). The found case's OWN matching already surfaces +
  notifies matching lost dogs, so the auto-rematch is redundant. Made it bounded + geo-filtered and
  OFF by default (`REMATCH_MAX_CASES=0`); raise it to opt in for small sets. Also fixed a latent FK
  bug: `run_matching_for_case` deleted prior pending matches without detaching notifications that
  referenced them (nulls `Notification.match_id` first now) — this surfaced once re-matching a case
  that had produced a strong-match notification.
- **Owner re-run matching:** `POST /cases/{id}/rematch` (owner/admin, re-runs at current radius) +
  a "Re-run matching" button on the owner CaseDetail (next to Widen). Two tests updated to re-run
  explicitly now that auto-rematch is off by default.
- **Admin delete (temporary, for re-demoing):** `DELETE /admin/dogs/{kind}/{id}` (dog + pictures +
  media + embeddings + breeds + its cases + matches referencing it) and `DELETE /admin/cases/{id}`
  (case + matches, dog kept). Delete buttons on AdminDogs cards and AdminCases rows.
- Tests: owner rematch (+403), admin delete dog/case (+404/400). All green (84 backend, 16 frontend).

## Admin dashboard as a button hub + admin nav + delete people

- Admin home (`Admin.tsx`) is now just a grid of bigger link cards (Pending Matches, All Cases, All
  Dogs, Owners, Test Matching, Data Loading). The pending-matches review view moved to its own page
  `AdminPending.tsx` at `/admin/pending`.
- Nav: when an admin is logged in, "My Dogs"/"My Cases" are hidden (only the Admin link + Log out).
- Delete people: `DELETE /admin/owners/{id}` (+ `delete_person` service) removes a person and their
  dogs/pictures/media/embeddings/breeds/cases/matches; refuses to delete an admin. Delete button per
  owner on the Owners page (dogs + cases already had delete buttons).
- Test: `test_admin_delete_person` (deletes owner+dogs, 404 on re-delete, 400 on admin). All green.

## 1-decimal match % + tie-break by most-recent report

- Match percentages now show one decimal (e.g. 94.4%) so close scores stop collapsing onto the same
  integer: ConfidenceBar, MatchCard header, and the PhotoSearch result badges.
- Matching ranking (`run_matching_for_case`): primary sort is the similarity ROUNDED to one decimal
  (matches the display); candidates that tie at that precision are ordered most-recently-reported
  first, using the candidate's case `created_at` (fetched alongside the date-gate query; ties with no
  report time sort last). `CandidateScore` gained a `reported_at` field.
- Tests: `test_equal_score_matches_break_ties_by_most_recent_report`; two frontend tests updated for
  the decimal. All green (87 backend, 16 frontend).