File size: 2,938 Bytes
90c4a87 b2ece46 90c4a87 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | # Running Chipoint locally
Point a photo at it, get back a latitude/longitude. Runs on a GPU (fast) or CPU (slow but works).
## 1. What you need
- **~16 GB free disk** for the weights (3 gallery indexes + 3 heads).
- **A GPU with ~8 GB VRAM** is ideal. CPU-only works too — expect ~1–2 min per image instead of a second.
- **~16 GB RAM.** The gallery indexes are memory-mapped, so they don't all have to sit in RAM at once.
- Python 3.10+.
## 2. Install
```bash
pip install torch numpy pandas pillow open_clip_torch timm transformers huggingface_hub modelscope
```
(For an AMD GPU, install the ROCm build of torch instead of the default CUDA one.)
## 3. Get the weights
```bash
huggingface-cli download chiikabu-labs/chipoint --local-dir chipoint
cd chipoint
```
## 4. Encoders
The three encoders download automatically on first run — nothing to accept or configure:
- `ViT-SO400M-16-SigLIP2-256` and `ViT-gopt-16-SigLIP2-256` (open, via open_clip)
- `facebook/dinov3-vitl16-pretrain-lvd1689m` (pulled from ModelScope, ungated)
If you'd rather use a local copy of DINOv3, set `export DINOV3_PATH=/path/to/dinov3-vitl16` before running.
## 5. Run
```bash
python run_chipoint.py my_photo.jpg
# or several / a glob:
python run_chipoint.py photos/*.jpg
```
Output, one line per image:
```
my_photo.jpg 47.83400, -70.46100 https://maps.google.com/?q=47.83400,-70.46100
```
The first run downloads the encoder weights and takes a minute to warm up.
**Geolocating many photos? Pass them all in one command** (`python run_chipoint.py photos/*.jpg`). Each 5 GB gallery index is read from disk only **once for the whole batch**. All images are scored against it in a single pass, so 50 images cost roughly the same as 1, not 50×. Running the script once per image re-reads all 15 GB every time, which is far slower.
## How it works
The image is encoded by three frozen vision models, each embedding is passed through a small trained
projection head, and each head retrieves its nearest neighbours from a gallery of 4.9M geotagged images.
The candidate lists are fused, then re-scored with a density-weighted consensus. The candidates vote on
a region, with each vote scaled by how sparse its area is in the gallery, so over-photographed places don't
dominate. The winning candidate's GPS is the answer. Nothing is trained at query time; it's all retrieval
plus that scoring rule. This is the exact pipeline behind the benchmark numbers (37.6 % within 25 km on OSV-5M).
## Notes
- Best on outdoor photos, streets, landscapes, towns. It's a region/city model.
- Most efficient at daytime photos. Nighttime photos are a struggle for the model, however during twilight/sunrise the model may still perform as expected.
- CPU works but the g-opt encoder is large; a GPU is much happier.
- Environment variables: `CHIPOINT_DIR` (weights folder, defaults to the script's own directory), `DINOV3_PATH` (local DINOv3 copy).
|