Upload REPRODUCE.md with huggingface_hub

REPRODUCE.md

@@ -0,0 +1,101 @@
+# Reproduce / Train MapTokenGS — agent guide
+
+Self-contained instructions to go from the two HuggingFace repos to a trained model.
+Two repos make up the release (split for licensing — Waymo cannot be public):
+
+| repo | visibility | holds |
+|---|---|---|
+| `ChenmingWu/mapgs-maptokengs` | public | **all code** + Argoverse-2 data (`data/av2_shard_*.tar`) + checkpoints (`runs/*.safetensors`) + this guide |
+| `ChenmingWu/mapgs-waymo` | private | Waymo data only (`data/waymo_shard_*.tar`) |
+
+You need **both** for full Waymo+AV2 training; AV2-only training works from the public repo alone.
+
+---
+
+## Step 1 — Download both repos
+```bash
+pip install -U "huggingface_hub[cli]"
+huggingface-cli login            # needed for the PRIVATE waymo repo
+
+# code + AV2 + checkpoints (public)
+huggingface-cli download ChenmingWu/mapgs-maptokengs --repo-type model --local-dir mapgs-maptokengs
+# Waymo data (private — requires your token with read access)
+huggingface-cli download ChenmingWu/mapgs-waymo      --repo-type model --local-dir mapgs-waymo
+```
+
+## Step 2 — Reconstruct + MERGE the data
+Every shard tar stores paths as `data/unified/<dataset>/<split>/<clip_id>/…`, so untarring all
+shards **from the same working directory** rebuilds one merged tree. Use the code repo as the root:
+```bash
+cd mapgs-maptokengs
+# AV2 (train+val) -> data/unified/av2/{train,val}/...
+for f in data/av2_shard_*.tar;            do tar -xf "$f"; done
+# Waymo (train)   -> data/unified/waymo/train/...   (merges into the SAME data/unified/)
+for f in ../mapgs-waymo/data/waymo_shard_*.tar; do tar -xf "$f"; done
+rm -f data/*_shard_*.tar                  # optional: drop the tarballs
+
+# verify the merge (expect ~1822 av2/train, 309 av2/val, ~4590 waymo/train clips):
+for d in data/unified/av2/train data/unified/av2/val data/unified/waymo/train; do
+  echo "$d: $(ls data/unified/${d#data/unified/} 2>/dev/null | wc -l) clips"; done
+```
+Each `<clip_id>/` contains `meta.pt` (poses, intrinsics, HD-map ground field, MAGT anchors,
+dynamic boxes) + `images/<frame>_<cam>.jpg`. That's the format `UnifiedClipDataset` reads.
+
+## Step 3 — Environment
+Python ≥ 3.11, a CUDA GPU. From the `mapgs-maptokengs` root:
+```bash
+pip install torch --index-url https://download.pytorch.org/whl/cu128   # match your CUDA
+pip install gsplat                       # 3DGS rasterizer (builds against CUDA)
+pip install git+https://github.com/rahul-goel/fused-ssim.git           # TokenGS needs fused_ssim
+pip install einops safetensors imageio numpy matplotlib transformers huggingface_hub
+pip install -e _tokengs_repo             # the TokenGS backbone package (Apache-2.0)
+export PYTHONPATH=$PWD                    # so `mapgs`, `maptokengs`, `scripts` import
+```
+`transformers` is only for the L_vert mono-depth prior (Depth-Anything-V2-Small, auto-downloaded
+on first use).
+
+## Step 4 — Backbone weights (warm-start)
+Training **warm-starts from NVIDIA TokenGS `dl3dv_base`** (research-only; NOT redistributed here —
+get it from the official TokenGS release, github.com/nv-tlabs/TokenGS). Two options:
+- **From the pretrained backbone:** pass `--ckpt /path/to/dl3dv_base.safetensors`.
+- **Continue from our finetuned model (no NVIDIA download needed):** our checkpoints in `runs/` are
+  full weights — pass `--ckpt runs/maptokengs_fixed_best.safetensors` to continue/eval directly.
+
+## Step 5 — Train
+```bash
+PYTHONPATH=$PWD python scripts/finetune_maptokengs.py \
+  --ckpt /path/to/dl3dv_base.safetensors \
+  --roots data/unified/av2,data/unified/waymo \
+  --val-roots data/unified/av2 \
+  --height 448 --width 768 --n-in 8 \
+  --iters 1200 --eval-every 400 --eval-clips 64 \
+  --out runs/my_run.safetensors
+```
+- Evaluates on **held-out scenes** (`av2/val`, disjoint from train) and writes the best held-out
+  checkpoint to `runs/my_run_best.safetensors`.
+- **Early-stop ~1000–1200 steps:** held-out PSNR peaks around step 1000 then *overfits* on this
+  6.4k-clip set (it dropped 17.98→15.39 by step 3000) — the `*_best.safetensors` captures the peak,
+  so always use that, not the final.
+- AV2-only (skip Waymo): `--roots data/unified/av2`.
+- All losses are on by default: MAGT-anchored map tokens, map-depth + free-space, extrapolation
+  warp, free-token visibility, PointForward dynamics, L_vert, tempering. Flags: `--lam-md --lam-fs
+  --lam-vis --lam-ext --lam-vert`, ramps `--extrap-ramp --vert-ramp --dyn-ramp`.
+
+## Step 6 — Evaluate / visualize / infer
+```bash
+# qualitative grid (GT vs predicted novel view vs depth) + out-of-trajectory strip:
+PYTHONPATH=$PWD python scripts/visualize_maptokengs.py \
+  --ckpt runs/maptokengs_fixed_best.safetensors --s 2.0      # -> runs/viz_nvs.png, viz_oot.png
+```
+The training script already prints held-out-SCENE PSNR/SSIM (mean±std) BEFORE/AFTER. Our best
+held-out-scene result: **17.98 ± 1.4 PSNR / 0.597 SSIM** (`runs/maptokengs_fixed_best.safetensors`).
+
+## Notes / gotchas
+- **Resolution** is capped by stored data: width ≤ ~768 (AV2 stored 775w), height ≤ ~683 (Waymo).
+  The loader does aspect-preserving cover+crop, so AV2 (portrait) and Waymo (landscape) aren't
+  distorted. Going higher needs re-processing from native sensor frames via `scripts/*_to_unified`.
+- **Eval is held-out SCENES** (av2/val), `s` pinned to `s_max`. Do not eval on train clips
+  (in-sample numbers are inflated).
+- **Licenses:** Waymo data — do NOT redistribute (regenerate from your own Waymo access via
+  `mapgs/data/convert/waymo_to_unified.py`). AV2 — CC BY-NC-SA 4.0 (attribute Argoverse 2).
+  Checkpoints — derived from NVIDIA TokenGS, research-only / non-commercial.