RUN_GUIDE.md

# Running Chipoint locally

Point a photo at it, get back a latitude/longitude. Runs on a GPU (fast) or CPU (slow but works).

## 1. What you need

- **~16 GB free disk** for the weights (3 gallery indexes + 3 heads).
- **A GPU with ~8 GB VRAM** is ideal. CPU-only works too — expect ~1–2 min per image instead of a second.
- **~16 GB RAM.** The gallery indexes are memory-mapped, so they don't all have to sit in RAM at once.
- Python 3.10+.

## 2. Install

```bash
pip install torch numpy pandas pillow open_clip_torch timm transformers huggingface_hub modelscope
```

(For an AMD GPU, install the ROCm build of torch instead of the default CUDA one.)

## 3. Get the weights

```bash
huggingface-cli download chiikabu-labs/chipoint --local-dir chipoint
cd chipoint
```

## 4. Encoders

The three encoders download automatically on first run — nothing to accept or configure:
- `ViT-SO400M-16-SigLIP2-256` and `ViT-gopt-16-SigLIP2-256` (open, via open_clip)
- `facebook/dinov3-vitl16-pretrain-lvd1689m` (pulled from ModelScope, ungated)

If you'd rather use a local copy of DINOv3, set `export DINOV3_PATH=/path/to/dinov3-vitl16` before running.

## 5. Run

```bash
python run_chipoint.py my_photo.jpg
# or several / a glob:
python run_chipoint.py photos/*.jpg
```

Output, one line per image:

```
my_photo.jpg    47.83400, -70.46100    https://maps.google.com/?q=47.83400,-70.46100
```

The first run downloads the encoder weights and takes a minute to warm up.

**Geolocating many photos? Pass them all in one command** (`python run_chipoint.py photos/*.jpg`). Each 5 GB gallery index is read from disk only **once for the whole batch**. All images are scored against it in a single pass, so 50 images cost roughly the same as 1, not 50×. Running the script once per image re-reads all 15 GB every time, which is far slower.

## How it works

The image is encoded by three frozen vision models, each embedding is passed through a small trained
projection head, and each head retrieves its nearest neighbours from a gallery of 4.9M geotagged images.
The candidate lists are fused, then re-scored with a density-weighted consensus. The candidates vote on
a region, with each vote scaled by how sparse its area is in the gallery, so over-photographed places don't
dominate. The winning candidate's GPS is the answer. Nothing is trained at query time; it's all retrieval
plus that scoring rule. This is the exact pipeline behind the benchmark numbers (37.6 % within 25 km on OSV-5M).

## Notes

- Best on outdoor photos, streets, landscapes, towns. It's a region/city model.
- Most efficient at daytime photos. Nighttime photos are a struggle for the model, however during twilight/sunrise the model may still perform as expected.
- CPU works but the g-opt encoder is large; a GPU is much happier.
- Environment variables: `CHIPOINT_DIR` (weights folder, defaults to the script's own directory), `DINOV3_PATH` (local DINOv3 copy).