Add arXiv link and BibTeX citation

README.md

@@ -21,7 +21,7 @@ pipeline_tag: depth-estimation
 <a title="Website" href="https://prs-eth.github.io/PaGeR/" target="_blank" rel="noopener noreferrer" style="display: inline-block;">
     <img src="https://img.shields.io/badge/%E2%99%A5%20Project%20-Website-blue" alt="Website">
 </a>
-<a title="arXiv" href="https://arxiv.org/abs/TBD" target="_blank" rel="noopener noreferrer" style="display: inline-block;">
+<a title="arXiv" href="https://arxiv.org/abs/2605.26368" target="_blank" rel="noopener noreferrer" style="display: inline-block;">
     <img src="https://img.shields.io/badge/%F0%9F%93%84%20Read%20-Paper-AF3436" alt="arXiv">
 </a>
 <a title="Hugging Face" href="https://huggingface.co/spaces/prs-eth/PaGeR" target="_blank" rel="noopener noreferrer" style="display: inline-block;">
@@ -41,40 +41,52 @@ pipeline_tag: depth-estimation
 </a>
 </p>
 
-`PaGeR` is the **unified** geometry-estimation checkpoint released with our paper *Unified Panoramic Geometry Estimation via Multi-View Foundation Models* — [arXiv (TBD)](https://arxiv.org/abs/TBD).
+`PaGeR` is the **unified** geometry-estimation checkpoint released with our paper:
+
+- **Paper:** *Unified Panoramic Geometry Estimation via Multi-View Foundation Models* — [arXiv:2605.26368](https://arxiv.org/abs/2605.26368)
 
 From a single equirectangular (ERP) panorama, one forward pass returns:
 
-- **Scale-invariant (SI) depth** at full panoramic resolution, from the dense depth head.
-- **Metric depth** in metres, computed as `SI_depth × exp(log_scale)` with a single global `log_scale` from a parallel coarse-scale head. Two scale heads ship (indoor / outdoor); exactly one runs per panorama, picked manually (`--scene_mode indoor|outdoor`) or auto-routed by a zero-shot [CLIP ViT-B/32](https://github.com/openai/CLIP) classifier (via [`open_clip`](https://github.com/mlfoundations/open_clip)) that scores the 4 equatorial cubemap faces against "indoor scene" / "outdoor scene" prompts.
-- **Surface normals** as unit vectors in the panorama's world frame.
-- **Sky mask** for filling unbounded regions in the depth and normal outputs.
+- **Scale-invariant (SI) depth** at full panoramic resolution, predicted by the dense depth head.
+- **Metric depth** in metres, obtained by multiplying the SI depth with a single global log-scale predicted by a parallel coarse metric-scale head. Two such scale heads are trained — one for indoor, one for outdoor scenes — and exactly one runs per panorama (see routing below).
+- **Surface normals** as unit vectors in the panorama's world frame,
+- **Sky segmentation** for masking unbounded depth regions.
+
+So the unified PaGeR checkpoint emits *both* the SI depth map and the metric depth map in one shot: the dense head fixes geometry, the scale head fixes absolute scale. If you only need metric depth and don't want to manage the indoor/outdoor scale routing, the depth-only [`prs-eth/PaGeR-metric-depth`](https://huggingface.co/prs-eth/PaGeR-metric-depth) checkpoint predicts metric depth directly in a single head.
+
+Indoor and outdoor scenes are served by twin scale heads, so a single checkpoint covers both regimes. The active head can be selected manually or routed automatically — see [Model Details](#model-details) below for how that routing is done at inference time.
 
-If you only need metric depth in a single direct head (no scale routing), use [`prs-eth/PaGeR-metric-depth`](https://huggingface.co/prs-eth/PaGeR-metric-depth). Browse the full [PaGeR HF collection](https://huggingface.co/collections/prs-eth/pager) or try the [interactive demo](https://huggingface.co/spaces/prs-eth/PaGeR).
+You can also browse the rest of our [PaGeR HF collection](https://huggingface.co/collections/prs-eth/pager) or try the [interactive demo](https://huggingface.co/spaces/prs-eth/PaGeR).
 
 ## Model Details
 
 - **Developed by:** [Vukasin Bozic](https://vulus98.github.io/), [Isidora Slavkovic](https://linkedin.com/in/isidora-slavkovic), [Dominik Narnhofer](https://scholar.google.com/citations?user=tFx8AhkAAAAJ&hl=en), [Nando Metzger](https://nandometzger.github.io/), [Denis Rozumny](https://rozumden.github.io/), [Konrad Schindler](https://scholar.google.com/citations?user=FZuNgqIAAAAJ), [Nikolai Kalischek](https://scholar.google.com/citations?user=XwzlnZoAAAAJ&hl=de).
-- **Model type:** Feed-forward, multi-view foundation-model adaptation for single-image panoramic geometry (depth + normals + sky + metric scale).
-- **Backbone:** [Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3) (`da3-giant`, ViT-Giant), repurposed for cubemap-based multi-view processing.
-- **I/O:** ERP panorama in (up to 3K) → 6-face cubemap at 504 px/face → ERP outputs (SI depth, metric depth, normals, sky mask).
-- **Routing note:** The paper's per-domain numbers were produced with the scale head selected manually per dataset; the CLIP router is an inference-time convenience (demo / CLI), lives outside the checkpoint, and can be overridden via `--scene_mode {auto,indoor,outdoor}`.
-- **License:** [CC BY-NC 4.0](LICENSE) — academic / non-commercial only, inherited from the `da3-giant` backbone (CC BY-NC 4.0). The runtime CLIP ViT-B/32 weights are MIT-licensed and travel separately. Commercial use is not permitted.
+- **Model type:** Feed-forward, multi-view foundation-model adaptation for single-image panoramic geometry estimation (depth + normals + sky + metric scale).
+- **Backbone:** [Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3) (`da3-giant`, ViT-Giant), repurposed for cubemap-based multi-view processing of the panorama.
+- **Inputs:** A single ERP panorama, internally projected onto a 6-face cubemap at 504 px per face.
+- **Outputs (in one forward pass):**
+  - **Scale-invariant (SI) depth map** at panoramic resolution, from the dense depth head.
+  - **Metric depth** (metres), computed as `SI_depth * exp(log_scale)` where `log_scale` is the single global log-scale predicted by the active (indoor or outdoor) coarse metric-scale head. Both the SI and metric maps share the same dense geometry; the scale head only injects absolute scale.
+  - **Surface normals** as unit vectors in the panorama's world frame.
+  - **Sky mask** for filling/masking unbounded regions in the depth and normal outputs.
+- **Indoor / outdoor routing (inference-time add-on, not part of the paper):** The paper's per-domain numbers were produced with the indoor or outdoor scale head selected manually per dataset. For convenience in the released demo and CLI, a small zero-shot [CLIP ViT-B/32](https://github.com/openai/CLIP) classifier (loaded via [`open_clip`](https://github.com/mlfoundations/open_clip)) can auto-pick between the twin scale heads at inference time, by scoring the 4 equatorial cubemap faces against two text-prompt centroids ("indoor scene" vs. "outdoor scene"). The router lives outside the checkpoint and can be overridden by the user (`--scene_mode {auto,indoor,outdoor}`).
+- **Resolution:** Designed for high-resolution ERP inputs, up to 3K.
+- **License:** [CC BY-NC 4.0](LICENSE) — academic / non-commercial use only. The released weights are derivative works of the [Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3) `da3-giant` backbone, released by ByteDance under CC BY-NC 4.0, and inherit that restriction. The CLIP ViT-B/32 weights used for the indoor/outdoor router are loaded from [`open_clip`](https://github.com/mlfoundations/open_clip) at runtime; they are MIT-licensed and travel separately, so they do not propagate any additional restriction. Commercial use is not permitted.
+- **Resources for more information:** [Project Website](https://prs-eth.github.io/PaGeR/), [Paper](https://arxiv.org/abs/2605.26368), [Code](https://github.com/prs-eth/PaGeR).
+
 ### Other released checkpoints
 
-| Checkpoint | SI depth | Metric depth | Normals | Sky |
+| Checkpoint | Hugging Face id | Depth | Normals | Sky |
 |---|---|---|---|---|
-| **PaGeR** *(this card, recommended)* — [`prs-eth/PaGeR`](https://huggingface.co/prs-eth/PaGeR) | ✅ | ✅ (SI × CLIP-routed indoor / outdoor scale head) | ✅ | ✅ |
-| PaGeR-Metric-Depth — [`prs-eth/PaGeR-metric-depth`](https://huggingface.co/prs-eth/PaGeR-metric-depth) | — | ✅ (single direct head) | — | — |
-| PaGeR-Normals — [`prs-eth/PaGeR-normals`](https://huggingface.co/prs-eth/PaGeR-normals) | — | — | ✅ | — |
+| **PaGeR** *(this card, recommended)* | [`prs-eth/PaGeR`](https://huggingface.co/prs-eth/PaGeR) | ✅ | ✅ | ✅ |
+| PaGeR-Metric-Depth | [`prs-eth/PaGeR-metric-depth`](https://huggingface.co/prs-eth/PaGeR-metric-depth) | ✅ (metric) |  |  |
+| PaGeR-Normals | [`prs-eth/PaGeR-normals`](https://huggingface.co/prs-eth/PaGeR-normals) |  | ✅ |  |
 
 ## Usage
 
-A minimal Python snippet that runs the unified model on a single panorama:
+A minimal Python snippet that runs the unified model on a single panorama and produces metric depth, surface normals, and a sky mask in one forward pass. The snippet assumes you have [cloned the repository](https://github.com/prs-eth/PaGeR) and `pip install -e .` ed it, so that `src.pager` is importable; checkpoint weights and config are streamed from the Hub on first use.
 
 ```python
-from pathlib import Path
-
 import matplotlib.pyplot as plt
 import numpy as np
 import torch
@@ -89,6 +101,7 @@ from src.utils.utils import prepare_depth_for_logging, prepare_normals_for_loggi
 checkpoint = "prs-eth/PaGeR"          # or a local directory
 device = torch.device("cuda")
 
+# 1. Load the model config from the Hub and instantiate Pager.
 config_path = hf_hub_download(repo_id=checkpoint, filename="config.yaml")
 cfg = OmegaConf.load(config_path)
 
@@ -96,23 +109,48 @@ pager = Pager(checkpoint, cfg=cfg, device=device)
 pager.get_intrinsics_extrinsics(image_size=cfg.face_size, fov=getattr(cfg, "cube_fov", 90.0))
 pager.model.to(device).eval()
 
-panorama = np.array(Image.open("examples/example_1.jpg").convert("RGB")) / 255.0
+# 2. Load a panorama and project it to the 6-face cubemap PaGeR consumes.
+panorama = np.array(Image.open("assets/examples/apartment_synth.jpg").convert("RGB")) / 255.0
 panorama = torch.from_numpy(panorama).permute(2, 0, 1).float() * 2 - 1
 rgb_cubemap = erp_to_cubemap(panorama, face_w=cfg.face_size,
                              fov=getattr(cfg, "cube_fov", 90.0)).unsqueeze(0).to(device)
 
+# 3. Run one forward pass. The unified checkpoint carries both indoor and
+#    outdoor scale heads; pass ``skip_heads`` to keep exactly one of them
+#    (here: force the outdoor head by skipping ``scale_indoor``). The full
+#    CLI in ``inference.py`` instead routes each panorama automatically via
+#    a small CLIP ViT-B/32 classifier on the cubemap faces.
 with torch.inference_mode():
     pred = pager(rgb_cubemap, dtype=torch.float16, skip_heads={"scale_indoor"})
 
+# 4. Convert raw head outputs into ERP-resolution arrays:
+#    - depth: SI depth × exp(log_scale) → metric depth (metres), with the
+#      predicted sky region filled to ``MAX_DEPTH`` via a soft alpha blend.
+#    - normals: unit vectors in the panorama's world frame, sky-filled.
 cmap = plt.get_cmap("Spectral")
 H, W = panorama.shape[-2:]
-depth_metric, _ = prepare_depth_for_logging(
+depth_metric, depth_viz = prepare_depth_for_logging(
     pager, pred["depth"][0], pred["sky"][0], (H, W), cmap,
     log_scale=pred["scale"],
 )
-normals, _ = prepare_normals_for_logging(
+normals, normals_viz = prepare_normals_for_logging(
     pager, pred["normals"][0], pred["sky"][0], (H, W),
 )
 ```
 
-`depth_metric` is a `(1, H, W)` float32 array of metric depth (metres); `normals` is a `(3, H, W)` unit-normal field. Both already have the predicted sky region filled in. See the [GitHub repository](https://github.com/prs-eth/PaGeR) for the full CLI (`inference.py`), evaluation scripts, the Gradio demo (`app.py`), and the point-cloud exporter.
+`depth_metric` is a `(1, H, W)` float32 array of metric depth (metres); `normals` is a `(3, H, W)` unit-normal field. Both already have the predicted sky region filled in. The `*_viz` companions are uint8 RGB previews (Spectral-coloured for depth, per-sample rescaled for normals). See the [GitHub repository](https://github.com/prs-eth/PaGeR) for the full CLI (`inference.py`), evaluation scripts, the Gradio demo (`app.py`), and the point-cloud exporter.
+
+## Citation
+
+If you use this checkpoint in your work, please cite:
+
+```bibtex
+@article{bozic2026pager,
+  title   = {Unified Panoramic Geometry Estimation via Multi-View Foundation Models},
+  author  = {Bozic, Vukasin and Slavkovic, Isidora and Narnhofer, Dominik and
+             Metzger, Nando and Rozumny, Denis and Schindler, Konrad and
+             Kalischek, Nikolai},
+  journal = {arXiv preprint arXiv:2605.26368},
+  year    = {2026}
+}
+```