README.md

metadata

title: PawTrace
emoji: 🐕
colorFrom: yellow
colorTo: green
sdk: docker
app_port: 7860
pinned: false
short_description: Find a lost dog by its image — AI re-identification demo

PawTrace — image-based dog re-identification

Upload a photo of a dog and PawTrace ranks a database of found dogs by how closely each one matches that specific dog, not just its breed. It runs two pipelines over a shared ResNet-101 backbone: an image similarity search fine-tuned to tell individuals apart, and a breed classifier.

Live demo: https://pawtrace.predx.com/ — read-only, searching 1,000 sample dogs (3,474 photos).

The model

A ResNet-101 model initialized from a dog-breed classifier (jhoppanne/Dogs-Breed-Image-Classification-V1) and fine-tuned with batch-hard triplet loss, producing a 2,048-dimension L2-normalized embedding per image. Two images of the same dog land close together; two look-alikes do not.

Measured on 419 held-out dogs (2,176 photos) under the standard re-ID protocol, where every image queries all the others and is excluded from its own gallery:

Split	Rank-1	Top-5	Top-10	mAP
Face	90.9%	97.0%	98.8%	65.0%
Body	93.3%	97.1%	98.1%	66.4%
Combined	92.4%	97.0%	98.2%	65.6%

Fine-tuning lifted overall Rank-1 from 71.6% to 92.6%, a gain of 21 points over the pre-trained backbone. The gain is much larger on full-body photos (67.6% → 93.3%) than on faces (84.3% → 91.2%), because a breed-oriented encoder already preserves enough facial detail to distinguish individuals, while it collapses body shots of the same breed together.

Training code is in backend/scripts/train_reid.py.

Caveat worth stating. The training and test sets draw heavily on YT-BB-Dog, where every image of a dog comes from a single video, so background is constant within an identity. These numbers therefore describe retrieval within a session, and the real lost-dog case — a photo taken today against one taken last week, somewhere else — should be expected to be harder. Measuring that gap properly needs a dataset following the same dogs across days and locations at usable resolution, which no public set currently offers.

Architecture

React + Vite + TS (SPA, Tailwind)  ──HTTP/JSON──>  FastAPI
  PC + mobile camera capture                         ├─ Auth (JWT, bcrypt)
                                                     ├─ Dogs / Cases / Matches (SQLAlchemy → SQLite)
                                                     ├─ Image pipeline: validate→normalize→store→embed
                                                     ├─ Matching: ZIP-radius filter → cosine → rank
                                                     ├─ Notifier (console email default; SMTP optional)
                                                     └─ Geo (ZIP centroids + haversine)

A dog's match score is the maximum cosine similarity across every (query photo × candidate photo) pair, so one weak upload cannot drag down a dog that also has a good photo, and extra angles only help. Embeddings are stored tagged with model name and version, so vectors from different model generations are never compared.

Four seams are interfaces with config-selected implementations: Embedder, BreedClassifier, StorageBackend (local→S3), and VectorIndex (NumPy brute force→FAISS). Search is exact brute-force cosine, which is correct and fast at this scale.

Running locally

Requires Python 3.11+ (developed on 3.13) and Node 18+ (developed on 24).

cd backend
python -m venv .venv
# Windows PowerShell:   .venv\Scripts\Activate.ps1
# Windows git-bash:     source .venv/Scripts/activate
# macOS/Linux:          source .venv/bin/activate
pip install -r requirements.txt

cp ../.env.example ../.env     # optional; sane defaults apply with no .env
python -m scripts.seed         # seeds the admin account
uvicorn app.main:app --reload --port 8000    # API + docs at /docs

cd frontend
npm install
npm run dev        # http://localhost:5173, proxies /api to :8000

scripts.seed creates admin@example.com and prints a randomly generated password once. Set SEED_ADMIN_PASSWORD beforehand to choose your own.

The re-ID model is not used by default locally — see Configuration. To run against the same weights as the demo, place best.pt at the repo root and set EMBEDDER=reid.

Configuration

Tunables live in .env.example. The two that matter most:

Variable	Values	Deployed as
EMBEDDER	mock | hf | reid	reid (the fine-tuned checkpoint)
BREED_CLASSIFIER	mock | hf	hf

mock is a deterministic stand-in that needs no model weights and exists so the test suite runs fast and offline. hf downloads the breed model from Hugging Face on first use. reid loads the fine-tuned checkpoint from REID_MODEL_PATH.

Other settings cover match thresholds, TOP_N, RADIUS_LEVELS, image size limits, and storage/notifier selection.

Demo mode

With DEMO_MODE=true (how the public demo runs), middleware rejects every mutating HTTP request, so the database cannot be modified from the UI, a direct API call, or curl. The only exceptions are the two photo-search endpoints, which persist nothing. Safety is enforced on the server, not by hiding buttons in the interface.

Tests

cd backend && pytest -q      # 106 tests
cd frontend && npx vitest run # 22 tests

Tests use the mock embedder and breed classifier, so they need no weights, no downloads, and no GPU.

Data & privacy

• Uploaded images are auto-oriented and stripped of EXIF/GPS on ingest; only a processed JPEG and thumbnail are stored on disk, with the path in the database rather than the blob.
• Contact is mediated: home addresses and exact shelter/vet locations are never shown to the other party, only ZIP-level location. Reports are rate-limited.
• data/zip_centroids.csv is a representative sample of US ZIP centroids. Drop in a full public-domain dataset with the same zip,lat,lng columns for complete coverage.

Admin and batch loading

Bulk data is managed as datasets, each load grouped, inspectable, and purgeable as a unit, under /admin. Load from a folder of dog-identity subfolders (one subfolder per dog), then generate embeddings and breed predictions in a single pass per image:

cd backend
python -m scripts.load_dataset --folder /path/to/dogs --type unknown \
    --name "My dataset" --csv /path/to/dogs/found_dogs.csv
python -m scripts.process_dataset --all      # or --dataset-id N

Admin endpoints cover GET/DELETE /admin/datasets[/{id}], embedding, matching, and async loading via POST /admin/datasets/load with GET /admin/jobs/{id} for progress. Design notes are in DECISIONS.md.

Project layout

backend/   FastAPI app, services, ML interfaces, Alembic migrations, tests, scripts
frontend/  React + Vite + TS SPA (Tailwind)
data/      zip_centroids.csv, media/ (gitignored)

Credits

Built on open models and datasets from the research community: DogFaceNet, the Multi-Pose Dog Dataset, YT-BB-Dog, Stanford Dogs, and the Hugging Face breed classifier above. Full attributions and licenses are on the demo's Credits page.