| --- |
| license: apache-2.0 |
| tags: |
| - onnx |
| - camera-calibration |
| - gravity-estimation |
| - computer-vision |
| --- |
| |
| # GeoCalib field net, ONNX export |
|
|
| The convolutional half of [GeoCalib](https://github.com/cvg/GeoCalib) (Veicht et al., ECCV 2024, ETH CVG) as a single self-contained ONNX file: image in, four dense perspective fields out. Pair it with a small Levenberg-Marquardt solver on those fields to estimate roll, pitch, and gravity direction from a single image with no torch dependency. |
|
|
| Exported for [Caliscope](https://github.com/mprib/caliscope)'s board-free vertical estimation. |
|
|
| ## What it is |
|
|
| GeoCalib runs in two stages: a network predicts, per pixel, the projected world-up direction (`up_field`), the latitude of the viewing ray (`latitude_field`), and a confidence for each; a Levenberg-Marquardt optimizer then fits camera tilt (and optionally focal) to those fields. This export is the network stage only, from the official `pinhole` weights. The solver stage is ~300 lines of numpy, shipped with the consumer (Caliscope). |
|
|
| ## Contract |
|
|
| - **Input** `image`: `(1, 3, H, W)` float32 RGB in `[0, 1]`. H and W are dynamic but must be multiples of 32. GeoCalib's preprocessing resizes the short side to 320 with bilinear+antialias and rounds edges to multiples of 32; match it for best results. |
| - **Outputs**: `up_field` `(1, 2, H, W)`, `up_confidence` `(1, H, W)`, `latitude_field` `(1, 1, H, W)` (radians), `latitude_confidence` `(1, H, W)`. |
| - One graph serves all resolutions and orientations (exported with torch.onnx dynamo, dynamic shapes). |
| - fp32, 118 MB, weights embedded — no external data file. |
|
|
| ## Differences from upstream |
|
|
| Upstream GeoCalib is stochastic at inference: its Hamburger/NMF decoder draws random bases on every forward, so identical inputs give slightly different fields run to run (~0.005–0.2° gravity wobble downstream). This export freezes one seeded basis (seed 0) as a graph constant, making it deterministic. Field parity against frozen-basis torch is ~1e-6; gravity parity through the same solver is ~1e-5 degrees — four orders of magnitude below upstream's own run-to-run noise. |
|
|
| ## Provenance |
|
|
| - Upstream weights: `geocalib-pinhole` v1.0 release, [cvg/GeoCalib](https://github.com/cvg/GeoCalib) at commit `747054d`. |
| - Export procedure: wrap the model's backbone, low-level encoder, and perspective decoder in a module returning the four fields; replace each NMF2D `_build_bases` with a constant seeded basis (`torch.Generator().manual_seed(0)`, normalized); export with `torch.onnx.export(dynamo=True)`, dynamic height/width, `external_data=False`. Validated against torch on real frames before publishing. |
| - SHA-256: `a724447e1bf7138352e5e70bbe0f011e6fb02909edd3148a5d0f46e8d153ae7b` |
|
|
| ## License and attribution |
|
|
| Apache 2.0, as a derivative of GeoCalib (© ETH Zurich, Computer Vision and Geometry Group). If you use this model, cite the original work: |
|
|
| ```bibtex |
| @inproceedings{veicht2024geocalib, |
| author = {Alexander Veicht and Paul-Edouard Sarlin and Philipp Lindenberger and Marc Pollefeys}, |
| title = {{GeoCalib: Single-image Calibration with Geometric Optimization}}, |
| booktitle = {ECCV}, |
| year = {2024} |
| } |
| ``` |
|
|
