📟 PaGeR — Unified Panoramic Geometry Estimation Model Card

PaGeR is the unified geometry-estimation checkpoint released with our paper:

• Paper: Unified Panoramic Geometry Estimation via Multi-View Foundation Models — arXiv (TBD)

From a single equirectangular (ERP) panorama, one forward pass returns:

• Scale-invariant depth at full panoramic resolution,
• Metric depth in metres via a parallel coarse scale head,
• Surface normals as unit vectors in the panorama's world frame,
• Sky segmentation for masking unbounded depth regions.

Indoor and outdoor scenes are served by twin scale heads selected at inference time by a lightweight Places365 classifier, so a single checkpoint covers both regimes.

You can also browse the rest of our PaGeR HF collection or try the interactive demo.

Model Details

• Developed by: Vukasin Bozic, Isidora Slavkovic, Dominik Narnhofer, Nando Metzger, Denis Rozumny, Konrad Schindler, Nikolai Kalischek.
• Model type: Feed-forward, multi-view foundation-model adaptation for single-image panoramic geometry estimation (depth + normals + sky + metric scale).
• Backbone: Depth Anything 3 (da3-giant, ViT-Giant), repurposed for cubemap-based multi-view processing of the panorama.
• Inputs: A single ERP panorama, internally projected onto a 6-face cubemap at 504 px per face.
• Outputs (in one forward pass):
  • Scale-invariant depth map at panoramic resolution.
  • Metric depth (metres), produced by combining the depth map with the selected indoor / outdoor scale head.
  • Surface normals as unit vectors in the panorama's world frame.
  • Sky mask for filling/masking unbounded regions in the depth and normal outputs.
• Indoor / outdoor routing: A lightweight Places365 classifier auto-selects between the twin scale heads at inference time; the routing can be overridden by the user (--scene_mode {auto,indoor,outdoor}).
• Resolution: Designed for high-resolution ERP inputs, up to 3K.
• License: CC BY-NC 4.0 — academic / non-commercial use only. The released weights are derivative works of the Depth Anything 3 da3-giant backbone, released by ByteDance under CC BY-NC 4.0, and inherit that restriction. Commercial use is not permitted.
• Resources for more information: Project Website, Paper, Code.

Other released checkpoints

Checkpoint	Hugging Face id	Depth	Normals	Sky
PaGeR (this card, recommended)	prs-eth/PaGeR	✅	✅	✅
PaGeR-Metric-Depth	prs-eth/PaGeR-metric-depth	✅ (metric)
PaGeR-Normals	prs-eth/PaGeR-normals		✅

Usage

A minimal Python snippet that runs the unified model on a single panorama:

from pathlib import Path

import matplotlib.pyplot as plt
import numpy as np
import torch
from huggingface_hub import hf_hub_download
from omegaconf import OmegaConf
from PIL import Image

from src.pager import Pager
from src.utils.geometry_utils import erp_to_cubemap
from src.utils.utils import prepare_depth_for_logging, prepare_normals_for_logging

checkpoint = "prs-eth/PaGeR"          # or a local directory
device = torch.device("cuda")

config_path = hf_hub_download(repo_id=checkpoint, filename="config.yaml")
cfg = OmegaConf.load(config_path)

pager = Pager(checkpoint, cfg=cfg, device=device)
pager.get_intrinsics_extrinsics(image_size=cfg.face_size, fov=getattr(cfg, "cube_fov", 90.0))
pager.model.to(device).eval()

panorama = np.array(Image.open("examples/example_1.jpg").convert("RGB")) / 255.0
panorama = torch.from_numpy(panorama).permute(2, 0, 1).float() * 2 - 1
rgb_cubemap = erp_to_cubemap(panorama, face_w=cfg.face_size,
                             fov=getattr(cfg, "cube_fov", 90.0)).unsqueeze(0).to(device)

with torch.inference_mode():
    pred = pager(rgb_cubemap, dtype=torch.float16, skip_heads={"scale_indoor"})

cmap = plt.get_cmap("Spectral")
H, W = panorama.shape[-2:]
depth_metric, _ = prepare_depth_for_logging(
    pager, pred["depth"][0], pred["sky"][0], (H, W), cmap,
    log_scale=pred["scale"],
)
normals, _ = prepare_normals_for_logging(
    pager, pred["normals"][0], pred["sky"][0], (H, W),
)

depth_metric is a (1, H, W) float32 array of metric depth (metres); normals is a (3, H, W) unit-normal field. Both already have the predicted sky region filled in. See the GitHub repository for the full CLI (inference.py), evaluation scripts, the Gradio demo (app.py), and the point-cloud exporter.