| # RUN GUIDE — chipoint-2 street-view + general-photo arms |
|
|
| ## Files |
| - `street_gallery/proj_head_gallery_512_{dinov3,so400m256,gopt}.npy` — street gallery |
| projection vectors (4.89M OSV-5M street frames, unit-norm 512-d fp16, |
| the street-arm index) |
| - `street_gallery/gallery_all_C_so400m256.npy` — street gallery coordinates (4.89M x 2) |
| - `weights/proj_head_{dinov3,so400m256,gopt}.pt` — projection heads (tower -> 512-d) |
| - `weights/streetview_k512_joint.pt` — street-view reranker arm (K512, 23 features) |
| - `photo_gallery/` — unified photo gallery, 14,541,898 rows (MP14 10.9M + |
| MP16-Pro 4.1M): `emb_*.npy` (512-d fp16 projected), `ll_*.npy` (lat/lon), |
| `ids_*.txt` (source ids, row-aligned), `manifest.json` |
| - `weights/photoarm_k768_thr.pt` / `weights/photoarm_k768_sr.pt` / `weights/photoarm_k512_thr.pt` — |
| general-photo reranker arms (23 features) |
| - `weights/ghead_merged_z4/z6/z8/z10.pt`, `weights/ghead_merged_reg.pt`, |
| `features/gallery_merged_cells.npz` — merged-head feature stack (required for the |
| 23-feature contract) |
| - `weights/router_osv_flickr.pt` — street/photo router (DINOv3 512-d -> logit; |
| >0 photo, else street) |
|
|
| ## Load |
| ```bash |
| python run_chipoint.py weights/streetview_k512_joint.pt # street arm: verify + schema |
| python run_chipoint.py weights/photoarm_k768_thr.pt # photo arm: verify + schema |
| ``` |
| Checkpoint dict: `state` (SR), `thr_state` (ThrNet), `mu`/`sd` |
| (normalizers), `nf`, `names`, `thr_km`. |
|
|
| ## Feature contract (nf = 23) |
| 18 base features over the candidate pool (per-encoder sims, margins, |
| ranks, cross-encoder agreement, vote counts, spatial density) + 5 |
| merged-head features from `ghead_merged_z4/z6/z8/z10/reg.pt` mapped by |
| `gallery_merged_cells.npz`. Feed order is `ckpt['names']`; standardize with |
| `(X - mu) / sd` before scoring. |
|
|
| ## Eval protocol (numbers in README) |
| Street arm: OSV-5M 4.89M gallery; 210,122-image sequence-stratified test |
| split. Retrieval: query tower embedding -> `proj_head_*` -> 512-d, top-K |
| against `street_gallery/proj_head_gallery_512_*` (same tower space), rerank with the |
| SR/ThrNet head on the 23-feature stack, threshold at 1/25/200/750/2500 km. |
|
|
| Photo arm: unified 14.5M gallery (`photo_gallery/emb_*.npy`, 512-d fp16); |
| im2gps3k (2997) / yfcc4k (4536) queries; blocklisted honest protocol |
| (184 exact-id leaks removed); 5-fold CV for the headline numbers. |
|
|
