Running Chipoint locally

Point a photo at it, get back a latitude/longitude. Runs on a GPU (fast) or CPU (slow but works).

1. What you need

~16 GB free disk for the weights (3 gallery indexes + 3 heads).
A GPU with ~8 GB VRAM is ideal. CPU-only works too — expect ~1–2 min per image instead of a second.
~16 GB RAM. The gallery indexes are memory-mapped, so they don't all have to sit in RAM at once.
Python 3.10+.

2. Install

pip install torch numpy pandas pillow open_clip_torch timm transformers huggingface_hub modelscope

(For an AMD GPU, install the ROCm build of torch instead of the default CUDA one.)

3. Get the weights

huggingface-cli download chiikabu-labs/chipoint --local-dir chipoint
cd chipoint

4. Encoders

The three encoders download automatically on first run — nothing to accept or configure:

ViT-SO400M-16-SigLIP2-256 and ViT-gopt-16-SigLIP2-256 (open, via open_clip)
facebook/dinov3-vitl16-pretrain-lvd1689m (pulled from ModelScope, ungated)

If you'd rather use a local copy of DINOv3, set export DINOV3_PATH=/path/to/dinov3-vitl16 before running.

5. Run

python run_chipoint.py my_photo.jpg
# or several / a glob:
python run_chipoint.py photos/*.jpg

Output, one line per image:

my_photo.jpg    47.83400, -70.46100    https://maps.google.com/?q=47.83400,-70.46100

The first run downloads the encoder weights and takes a minute to warm up.

Geolocating many photos? Pass them all in one command (python run_chipoint.py photos/*.jpg). Each 5 GB gallery index is read from disk only once for the whole batch. All images are scored against it in a single pass, so 50 images cost roughly the same as 1, not 50×. Running the script once per image re-reads all 15 GB every time, which is far slower.

How it works

The image is encoded by three frozen vision models, each embedding is passed through a small trained projection head, and each head retrieves its nearest neighbours from a gallery of 4.9M geotagged images. The candidate lists are fused, then re-scored with a density-weighted consensus. The candidates vote on a region, with each vote scaled by how sparse its area is in the gallery, so over-photographed places don't dominate. The winning candidate's GPS is the answer. Nothing is trained at query time; it's all retrieval plus that scoring rule. This is the exact pipeline behind the benchmark numbers (37.6 % within 25 km on OSV-5M).

Notes

Best on outdoor photos, streets, landscapes, towns. It's a region/city model.
Most efficient at daytime photos. Nighttime photos are a struggle for the model, however during twilight/sunrise the model may still perform as expected.
CPU works but the g-opt encoder is large; a GPU is much happier.
Environment variables: CHIPOINT_DIR (weights folder, defaults to the script's own directory), DINOV3_PATH (local DINOv3 copy).