---
title: PawTrace
emoji: 🐕
colorFrom: yellow
colorTo: green
sdk: docker
app_port: 7860
pinned: false
short_description: Find a lost dog by its image — AI re-identification demo
---

# PawTrace — image-based dog re-identification

Upload a photo of a dog and PawTrace ranks a database of found dogs by how closely each one matches
that specific dog, not just its breed. It runs two pipelines over a shared ResNet-101 backbone: an
image similarity search fine-tuned to tell individuals apart, and a breed classifier.

**Live demo:** https://pawtrace.predx.com/ — read-only, searching 1,000 sample dogs (3,474 photos).

## The model

A ResNet-101 model initialized from a dog-breed classifier
([jhoppanne/Dogs-Breed-Image-Classification-V1](https://huggingface.co/jhoppanne/Dogs-Breed-Image-Classification-V1))
and fine-tuned with batch-hard triplet loss, producing a 2,048-dimension L2-normalized embedding per
image. Two images of the same dog land close together; two look-alikes do not.

Measured on 419 held-out dogs (2,176 photos) under the standard re-ID protocol, where every image
queries all the others and is excluded from its own gallery:

| Split    | Rank-1 | Top-5 | Top-10 | mAP   |
|----------|--------|-------|--------|-------|
| Face     | 90.9%  | 97.0% | 98.8%  | 65.0% |
| Body     | 93.3%  | 97.1% | 98.1%  | 66.4% |
| Combined | 92.4%  | 97.0% | 98.2%  | 65.6% |

Fine-tuning lifted overall Rank-1 from 71.6% to 92.6%, a gain of 21 points over the pre-trained
backbone. The gain is much larger on full-body photos (67.6% → 93.3%) than on faces
(84.3% → 91.2%), because a breed-oriented encoder already preserves enough facial detail to
distinguish individuals, while it collapses body shots of the same breed together.

Training code is in [`backend/scripts/train_reid.py`](./backend/scripts/train_reid.py).

> **Caveat worth stating.** The training and test sets draw heavily on YT-BB-Dog, where every image
> of a dog comes from a single video, so background is constant within an identity. These numbers
> therefore describe retrieval within a session, and the real lost-dog case — a photo taken today
> against one taken last week, somewhere else — should be expected to be harder. Measuring that gap
> properly needs a dataset following the same dogs across days and locations at usable resolution,
> which no public set currently offers.

## Architecture

```
React + Vite + TS (SPA, Tailwind)  ──HTTP/JSON──>  FastAPI
  PC + mobile camera capture                         ├─ Auth (JWT, bcrypt)
                                                     ├─ Dogs / Cases / Matches (SQLAlchemy → SQLite)
                                                     ├─ Image pipeline: validate→normalize→store→embed
                                                     ├─ Matching: ZIP-radius filter → cosine → rank
                                                     ├─ Notifier (console email default; SMTP optional)
                                                     └─ Geo (ZIP centroids + haversine)
```

A dog's match score is the maximum cosine similarity across every (query photo × candidate photo)
pair, so one weak upload cannot drag down a dog that also has a good photo, and extra angles only
help. Embeddings are stored tagged with model name and version, so vectors from different model
generations are never compared.

Four seams are interfaces with config-selected implementations: `Embedder`, `BreedClassifier`,
`StorageBackend` (local→S3), and `VectorIndex` (NumPy brute force→FAISS). Search is exact
brute-force cosine, which is correct and fast at this scale.

## Running locally

Requires Python 3.11+ (developed on 3.13) and Node 18+ (developed on 24).

```bash
cd backend
python -m venv .venv
# Windows PowerShell:   .venv\Scripts\Activate.ps1
# Windows git-bash:     source .venv/Scripts/activate
# macOS/Linux:          source .venv/bin/activate
pip install -r requirements.txt

cp ../.env.example ../.env     # optional; sane defaults apply with no .env
python -m scripts.seed         # seeds the admin account
uvicorn app.main:app --reload --port 8000    # API + docs at /docs
```

```bash
cd frontend
npm install
npm run dev        # http://localhost:5173, proxies /api to :8000
```

`scripts.seed` creates `admin@example.com` and prints a randomly generated password once. Set
`SEED_ADMIN_PASSWORD` beforehand to choose your own.

The re-ID model is not used by default locally — see Configuration. To run against the same weights
as the demo, place `best.pt` at the repo root and set `EMBEDDER=reid`.

## Configuration

Tunables live in [`.env.example`](./.env.example). The two that matter most:

| Variable | Values | Deployed as |
|---|---|---|
| `EMBEDDER` | `mock` \| `hf` \| `reid` | `reid` (the fine-tuned checkpoint) |
| `BREED_CLASSIFIER` | `mock` \| `hf` | `hf` |

`mock` is a deterministic stand-in that needs no model weights and exists so the test suite runs
fast and offline. `hf` downloads the breed model from Hugging Face on first use. `reid` loads the
fine-tuned checkpoint from `REID_MODEL_PATH`.

Other settings cover match thresholds, `TOP_N`, `RADIUS_LEVELS`, image size limits, and
storage/notifier selection.

## Demo mode

With `DEMO_MODE=true` (how the public demo runs), middleware rejects every mutating HTTP request, so
the database cannot be modified from the UI, a direct API call, or curl. The only exceptions are the
two photo-search endpoints, which persist nothing. Safety is enforced on the server, not by hiding
buttons in the interface.

## Tests

```bash
cd backend && pytest -q      # 106 tests
cd frontend && npx vitest run # 22 tests
```

Tests use the mock embedder and breed classifier, so they need no weights, no downloads, and no GPU.

## Data & privacy

- Uploaded images are auto-oriented and stripped of EXIF/GPS on ingest; only a processed JPEG and
  thumbnail are stored on disk, with the path in the database rather than the blob.
- Contact is mediated: home addresses and exact shelter/vet locations are never shown to the
  other party, only ZIP-level location. Reports are rate-limited.
- `data/zip_centroids.csv` is a representative sample of US ZIP centroids. Drop in a full
  public-domain dataset with the same `zip,lat,lng` columns for complete coverage.

## Admin and batch loading

Bulk data is managed as datasets, each load grouped, inspectable, and purgeable as a unit, under
`/admin`. Load from a folder of dog-identity subfolders (one subfolder per dog), then generate
embeddings and breed predictions in a single pass per image:

```bash
cd backend
python -m scripts.load_dataset --folder /path/to/dogs --type unknown \
    --name "My dataset" --csv /path/to/dogs/found_dogs.csv
python -m scripts.process_dataset --all      # or --dataset-id N
```

Admin endpoints cover `GET/DELETE /admin/datasets[/{id}]`, embedding, matching, and async loading via
`POST /admin/datasets/load` with `GET /admin/jobs/{id}` for progress. Design notes are in
[`DECISIONS.md`](./DECISIONS.md).

## Project layout

```
backend/   FastAPI app, services, ML interfaces, Alembic migrations, tests, scripts
frontend/  React + Vite + TS SPA (Tailwind)
data/      zip_centroids.csv, media/ (gitignored)
```

## Credits

Built on open models and datasets from the research community: DogFaceNet, the Multi-Pose Dog
Dataset, YT-BB-Dog, Stanford Dogs, and the Hugging Face breed classifier above. Full attributions and
licenses are on the demo's Credits page.