| --- |
| title: PawTrace |
| emoji: 🐕 |
| colorFrom: yellow |
| colorTo: green |
| sdk: docker |
| app_port: 7860 |
| pinned: false |
| short_description: Find a lost dog by its image — AI re-identification demo |
| --- |
| |
| # PawTrace — image-based dog re-identification |
|
|
| Upload a photo of a dog and PawTrace ranks a database of found dogs by how closely each one matches |
| that specific dog, not just its breed. It runs two pipelines over a shared ResNet-101 backbone: an |
| image similarity search fine-tuned to tell individuals apart, and a breed classifier. |
|
|
| **Live demo:** https://pawtrace.predx.com/ — read-only, searching 1,000 sample dogs (3,474 photos). |
|
|
| ## The model |
|
|
| A ResNet-101 model initialized from a dog-breed classifier |
| ([jhoppanne/Dogs-Breed-Image-Classification-V1](https://huggingface.co/jhoppanne/Dogs-Breed-Image-Classification-V1)) |
| and fine-tuned with batch-hard triplet loss, producing a 2,048-dimension L2-normalized embedding per |
| image. Two images of the same dog land close together; two look-alikes do not. |
|
|
| Measured on 419 held-out dogs (2,176 photos) under the standard re-ID protocol, where every image |
| queries all the others and is excluded from its own gallery: |
|
|
| | Split | Rank-1 | Top-5 | Top-10 | mAP | |
| |----------|--------|-------|--------|-------| |
| | Face | 90.9% | 97.0% | 98.8% | 65.0% | |
| | Body | 93.3% | 97.1% | 98.1% | 66.4% | |
| | Combined | 92.4% | 97.0% | 98.2% | 65.6% | |
|
|
| Fine-tuning lifted overall Rank-1 from 71.6% to 92.6%, a gain of 21 points over the pre-trained |
| backbone. The gain is much larger on full-body photos (67.6% → 93.3%) than on faces |
| (84.3% → 91.2%), because a breed-oriented encoder already preserves enough facial detail to |
| distinguish individuals, while it collapses body shots of the same breed together. |
|
|
| Training code is in [`backend/scripts/train_reid.py`](./backend/scripts/train_reid.py). |
|
|
| > **Caveat worth stating.** The training and test sets draw heavily on YT-BB-Dog, where every image |
| > of a dog comes from a single video, so background is constant within an identity. These numbers |
| > therefore describe retrieval within a session, and the real lost-dog case — a photo taken today |
| > against one taken last week, somewhere else — should be expected to be harder. Measuring that gap |
| > properly needs a dataset following the same dogs across days and locations at usable resolution, |
| > which no public set currently offers. |
|
|
| ## Architecture |
|
|
| ``` |
| React + Vite + TS (SPA, Tailwind) ──HTTP/JSON──> FastAPI |
| PC + mobile camera capture ├─ Auth (JWT, bcrypt) |
| ├─ Dogs / Cases / Matches (SQLAlchemy → SQLite) |
| ├─ Image pipeline: validate→normalize→store→embed |
| ├─ Matching: ZIP-radius filter → cosine → rank |
| ├─ Notifier (console email default; SMTP optional) |
| └─ Geo (ZIP centroids + haversine) |
| ``` |
|
|
| A dog's match score is the maximum cosine similarity across every (query photo × candidate photo) |
| pair, so one weak upload cannot drag down a dog that also has a good photo, and extra angles only |
| help. Embeddings are stored tagged with model name and version, so vectors from different model |
| generations are never compared. |
|
|
| Four seams are interfaces with config-selected implementations: `Embedder`, `BreedClassifier`, |
| `StorageBackend` (local→S3), and `VectorIndex` (NumPy brute force→FAISS). Search is exact |
| brute-force cosine, which is correct and fast at this scale. |
|
|
| ## Running locally |
|
|
| Requires Python 3.11+ (developed on 3.13) and Node 18+ (developed on 24). |
|
|
| ```bash |
| cd backend |
| python -m venv .venv |
| # Windows PowerShell: .venv\Scripts\Activate.ps1 |
| # Windows git-bash: source .venv/Scripts/activate |
| # macOS/Linux: source .venv/bin/activate |
| pip install -r requirements.txt |
| |
| cp ../.env.example ../.env # optional; sane defaults apply with no .env |
| python -m scripts.seed # seeds the admin account |
| uvicorn app.main:app --reload --port 8000 # API + docs at /docs |
| ``` |
|
|
| ```bash |
| cd frontend |
| npm install |
| npm run dev # http://localhost:5173, proxies /api to :8000 |
| ``` |
|
|
| `scripts.seed` creates `admin@example.com` and prints a randomly generated password once. Set |
| `SEED_ADMIN_PASSWORD` beforehand to choose your own. |
|
|
| The re-ID model is not used by default locally — see Configuration. To run against the same weights |
| as the demo, place `best.pt` at the repo root and set `EMBEDDER=reid`. |
|
|
| ## Configuration |
|
|
| Tunables live in [`.env.example`](./.env.example). The two that matter most: |
|
|
| | Variable | Values | Deployed as | |
| |---|---|---| |
| | `EMBEDDER` | `mock` \| `hf` \| `reid` | `reid` (the fine-tuned checkpoint) | |
| | `BREED_CLASSIFIER` | `mock` \| `hf` | `hf` | |
|
|
| `mock` is a deterministic stand-in that needs no model weights and exists so the test suite runs |
| fast and offline. `hf` downloads the breed model from Hugging Face on first use. `reid` loads the |
| fine-tuned checkpoint from `REID_MODEL_PATH`. |
|
|
| Other settings cover match thresholds, `TOP_N`, `RADIUS_LEVELS`, image size limits, and |
| storage/notifier selection. |
|
|
| ## Demo mode |
|
|
| With `DEMO_MODE=true` (how the public demo runs), middleware rejects every mutating HTTP request, so |
| the database cannot be modified from the UI, a direct API call, or curl. The only exceptions are the |
| two photo-search endpoints, which persist nothing. Safety is enforced on the server, not by hiding |
| buttons in the interface. |
|
|
| ## Tests |
|
|
| ```bash |
| cd backend && pytest -q # 106 tests |
| cd frontend && npx vitest run # 22 tests |
| ``` |
|
|
| Tests use the mock embedder and breed classifier, so they need no weights, no downloads, and no GPU. |
|
|
| ## Data & privacy |
|
|
| - Uploaded images are auto-oriented and stripped of EXIF/GPS on ingest; only a processed JPEG and |
| thumbnail are stored on disk, with the path in the database rather than the blob. |
| - Contact is mediated: home addresses and exact shelter/vet locations are never shown to the |
| other party, only ZIP-level location. Reports are rate-limited. |
| - `data/zip_centroids.csv` is a representative sample of US ZIP centroids. Drop in a full |
| public-domain dataset with the same `zip,lat,lng` columns for complete coverage. |
|
|
| ## Admin and batch loading |
|
|
| Bulk data is managed as datasets, each load grouped, inspectable, and purgeable as a unit, under |
| `/admin`. Load from a folder of dog-identity subfolders (one subfolder per dog), then generate |
| embeddings and breed predictions in a single pass per image: |
|
|
| ```bash |
| cd backend |
| python -m scripts.load_dataset --folder /path/to/dogs --type unknown \ |
| --name "My dataset" --csv /path/to/dogs/found_dogs.csv |
| python -m scripts.process_dataset --all # or --dataset-id N |
| ``` |
|
|
| Admin endpoints cover `GET/DELETE /admin/datasets[/{id}]`, embedding, matching, and async loading via |
| `POST /admin/datasets/load` with `GET /admin/jobs/{id}` for progress. Design notes are in |
| [`DECISIONS.md`](./DECISIONS.md). |
|
|
| ## Project layout |
|
|
| ``` |
| backend/ FastAPI app, services, ML interfaces, Alembic migrations, tests, scripts |
| frontend/ React + Vite + TS SPA (Tailwind) |
| data/ zip_centroids.csv, media/ (gitignored) |
| ``` |
|
|
| ## Credits |
|
|
| Built on open models and datasets from the research community: DogFaceNet, the Multi-Pose Dog |
| Dataset, YT-BB-Dog, Stanford Dogs, and the Hugging Face breed classifier above. Full attributions and |
| licenses are on the demo's Credits page. |
|
|
