model card

README.md

@@ -0,0 +1,160 @@
+---
+license: mit
+library_name: pytorch
+pipeline_tag: image-to-image
+tags:
+  - satellite-imagery
+  - remote-sensing
+  - super-resolution
+  - image-outpainting
+  - flow-matching
+  - generative-model
+---
+
+# Genesis
+
+**A Generative Engine for Hierarchical Satellite Image Synthesis** — ACM SIGSPATIAL 2026.
+
+[![Project Page](https://img.shields.io/badge/Project-Page-2ea44f)](https://subash-khanal.github.io/genesis)
+![arXiv](https://img.shields.io/badge/arXiv-coming%20soon-b31b1b)
+[![Code](https://img.shields.io/badge/GitHub-Code-181717?logo=github)](https://github.com/subash-khanal/genesis)
+
+![Genesis: sparse seed tiles → full multi-scale quadtree](assets/task.png)
+
+Genesis completes a sparse set of seed satellite tiles into a full, zoomable Web-Mercator
+**quadtree** — filling every scale and location. Two flow-matching *JiT* operators do the work: a
+**parent-conditional super-resolution** model (one 256×256 parent tile → the 512×512 mosaic of its
+four children, one zoom level down) and a **mask-based outpainting** model (256×256 completion
+under quadrant masks). Composed into a pyramid engine, they bring the seeds to a common working
+level, greedily outpaint to complete that level, super-resolve down to the leaf tiles, and fill the
+coarser levels by downsampling — yielding a pyramid that is consistent across scales and honors
+every seed. Models are trained on the Git-10M global tile corpus.
+
+*Subash Khanal, Yangzhi Cui, Daniel Cher, Eric Xing, Brian Wei, Srikumar Sastry, Nathan Jacobs —
+Washington University in St. Louis.*
+
+![The Genesis pyramid-completion pipeline](assets/pipeline.png)
+
+## Checkpoints
+
+All models sample with **50 flow-matching steps, cfg = 1.0** (the paper eval setting) and load
+**EMA** weights. Main runs are trained to 800k steps; the SR ablation grid (JiT-B/16) to 500k,
+toggling {DINOv3 conditioning on/off} × {LPIPS loss on/off}. The main SR models are
+DINOv3-conditioned (+LPIPS); the main outpainting models are no-DINO.
+
+| Path | Model / arch | Size | Train steps | What it's for |
+|---|---|---:|---:|---|
+| `main/superresolution/main_B16/sr-full-tile-stage3-step0800000.ckpt` | SR · JiT-B/16 | 3.3 GB | 800k | **Main SR** (DINOv3 + LPIPS), 256→512 |
+| `main/superresolution/main_H16/sr-full-tile-stage3-step0800000.ckpt` | SR · JiT-H/16 | 23 GB | 800k | **Main SR**, largest model |
+| `main/outpainting/main_B16/op-new-stage3-step0800000.ckpt` | OP · JiT-B/16 | 3.6 GB | 800k | **Main outpainting** (no-DINO, quadrant masks) |
+| `main/outpainting/main_H16/op-new-stage3-step0800000.ckpt` | OP · JiT-H/16 | 26 GB | 800k | **Main outpainting**, largest model |
+| `ablations/abl_nodino_nolpips/sr-full-tile-stage3-step0500000.ckpt` | SR · JiT-B/16 | 3.2 GB | 500k | Ablation: DINOv3 off · LPIPS off |
+| `ablations/abl_dino_nolpips/sr-full-tile-stage3-step0500000.ckpt` | SR · JiT-B/16 | 3.3 GB | 500k | Ablation: DINOv3 on · LPIPS off |
+| `ablations/abl_nodino_lpips/sr-full-tile-stage3-step0500000.ckpt` | SR · JiT-B/16 | 3.2 GB | 500k | Ablation: DINOv3 off · LPIPS on |
+| `ablations/abl_dino_lpips/sr-full-tile-stage3-step0500000.ckpt` | SR · JiT-B/16 | 3.3 GB | 500k | Ablation: DINOv3 on · LPIPS on |
+| `shared/git10m_quad_meta.json` | metadata | 1.7 GB | — | Git-10M split metadata (paper eval) |
+| `shared/hierarchy_index_quad.pkl` | metadata | 0.6 GB | — | Parent↔child quadtree index (paper eval) |
+| `dinov3_vitl16_pretrain_sat493m-eadcf0ff.pth` | DINOv3 ViT-L/16 (SAT-493M) | 1.2 GB | — | Frozen conditioning encoder for SR |
+| `example_sr_op.py` | script | — | — | Runnable quick start (below) |
+
+## Quick start
+
+Install the code repo (checkpoints download from this HF repo automatically):
+
+```bash
+git clone https://github.com/subash-khanal/genesis && cd genesis
+uv sync && source .venv/bin/activate    # Python 3.13, torch 2.8 cu128
+```
+
+Then run one SR ×2 pass and one quadrant-outpainting pass on a **live Esri World Imagery tile**
+(the Sydney Opera House waterfront at zoom 16 serves as the dummy input — the script fetches it
+over the network):
+
+```python
+import os, sys
+sys.path.insert(0, "src")
+sys.path.insert(0, "demos")   # demos/ last = highest priority (its utils/ package must win)
+
+from huggingface_hub import hf_hub_download
+from PIL import Image
+from utils.genesis_common import (
+    fetch_tile_at_zoom, inference_device,
+    load_sr_denoiser, load_op_denoiser,
+    run_superresolution, run_outpainting,
+)
+
+device = inference_device()   # cuda if available, else cpu
+
+sr_ckpt = hf_hub_download("MVRL/genesis", "main/superresolution/main_B16/sr-full-tile-stage3-step0800000.ckpt")
+op_ckpt = hf_hub_download("MVRL/genesis", "main/outpainting/main_B16/op-new-stage3-step0800000.ckpt")
+dino    = hf_hub_download("MVRL/genesis", "dinov3_vitl16_pretrain_sat493m-eadcf0ff.pth")
+
+lat, lon, zoom = -33.8568, 151.2153, 16
+tile = fetch_tile_at_zoom(lon, lat, zoom)                 # one real 256×256 Esri tile
+
+# SR: z16 parent → 512×512 mosaic of its four z17 children (steps=50, cfg=1.0 defaults)
+sr_model, _ = load_sr_denoiser(sr_ckpt, "JiT-B/16", dino_weights=dino, device=device)
+run_superresolution(sr_model, tile, device, target_zoom=zoom + 1).save("genesis_sr_out.png")
+
+# OP: keep the upper-left quadrant real, outpaint the other three (white = hole)
+mask = Image.new("L", (256, 256), 255)
+mask.paste(0, (0, 0, 128, 128))
+op_model, _ = load_op_denoiser(op_ckpt, "JiT-B/16", dino_weights="", device=device)
+run_outpainting(op_model, tile, mask, device, tile_zoom=zoom).save("genesis_op_out.png")
+```
+
+Or as a one-liner — download [`example_sr_op.py`](example_sr_op.py) from this repo into the
+checkout root and run:
+
+```bash
+python example_sr_op.py
+```
+
+> Live imagery: Esri World Imagery — Source: Esri, Maxar, Earthstar Geographics, and the GIS User
+> Community.
+
+## Demos
+
+Interactive Gradio apps live in the code repo — see
+[`demos/README.md`](https://github.com/subash-khanal/genesis/blob/main/demos/README.md).
+The main one is the **full-pyramid builder** (`demos/app_pyramid_unified.py`): click a satellite
+map to drop sparse seed tiles and watch Genesis complete an entire 4-level quadtree live,
+tile-by-tile, with a stitched-PNG download. Companion apps cover single-tile 256→512 SR, quadrant
+outpainting, a large lateral-outpainting grid, a deep 6-level pyramid, and no-model interpolation
+baselines.
+
+## Reproduce the paper
+
+One command per paper table — see
+[`docs/EVALUATION.md`](https://github.com/subash-khanal/genesis/blob/main/docs/EVALUATION.md)
+in the code repo:
+`bash src/eval/run_all_sr.sh` · `run_all_op.sh` · `run_all_ablation.sh` · `run_all_dense500.sh`,
+each with the paper-exact defaults baked in.
+
+## Citation
+
+```bibtex
+@inproceedings{khanal2026genesis,
+  title     = {{Genesis}: A Generative Engine for Hierarchical Satellite Image Synthesis},
+  author    = {Khanal, Subash and Cui, Yangzhi and Cher, Daniel and Xing, Eric and
+               Wei, Brian and Sastry, Srikumar and Jacobs, Nathan},
+  booktitle = {ACM SIGSPATIAL International Conference on Advances in
+               Geographic Information Systems (SIGSPATIAL)},
+  year      = {2026},
+}
+```
+
+## License & acknowledgments
+
+The Genesis code and checkpoints are released under the **MIT License** (see the
+[code repo](https://github.com/subash-khanal/genesis)).
+
+`dinov3_vitl16_pretrain_sat493m-eadcf0ff.pth` is the satellite-pretrained (SAT-493M) ViT-L/16
+encoder from Meta AI's [DINOv3](https://github.com/facebookresearch/dinov3) release, redistributed
+here unmodified for convenience; it remains governed by Meta's DINOv3 license, not MIT. At
+inference the SR model loads it via `torch.hub.load('facebookresearch/dinov3', 'dinov3_vitl16',
+weights=...)`.
+
+Training data comes from the [Git-10M](https://huggingface.co/datasets/lcybuaa/Git-10M) global
+satellite-tile corpus. Demo/example imagery is fetched live from Esri World Imagery (attribution
+above).