pinned
Sleeping
Agents
1
Calibench Leaderboard
🏆
Compare camera / camera-IMU calibration tools
AI & ML interests
None defined yet.
Recent Activity
Organization Card
Calibench: Calibration Benchmark for Visual-Inertial Sensors
Calibench is an open benchmark for camera and camera–IMU (visual-inertial) calibration. We host evaluation datasets, run a public leaderboard comparing calibration toolboxes, and welcome community datasets and results.
🔗 Project site: https://openmvis.com
Repositories
📦 calibench-vi/datasets |
Official evaluation datasets — our own captures (CC-BY-4.0) + download recipes for third-party sets |
🤝 calibench-vi/community |
Community-contributed calibration datasets |
📊 calibench-vi/results |
Leaderboard result records (one per toolbox × dataset) |
🏆 calibench-vi/leaderboard |
Interactive results leaderboard |
Benchmark datasets at a glance
14 physical datasets / rigs — the 13 behind the 61 leaderboard configs plus the
newly hosted virig-5imus-4cams (5 IMUs, not yet benchmarked) — from 195°-fisheye
to RGB-D, global- and rolling-shutter, camera-only to 5-IMU rigs. Browse the dataset wiki and sensor wiki (which sensor is in which dataset, and what calibrations exist for it). Full
machine-readable catalog: datasets.json
in calibench-vi/results.
| Dataset | Cameras | Shutter | Lens | IMU(s) | Board | License |
|---|---|---|---|---|---|---|
VI-rig 3-cam (ours, virig-4imus-3cams; ArUco variant virig-4imus-3cams-aruco) |
FLIR Blackfly S BFS-U3-16S2M 720×540 @30 + RealSense T265 2×848×800 @30 | GS | std + 163° fisheye | GX3-25 @500 Hz, GX3-35 @100 Hz, MTi-100 @400 Hz, BMI055 @200 Hz | AprilGrid 6×6 (+ ArUco) | CC BY 4.0 |
VI-rig 4-cam (ours, virig-4imus-4cams) |
ELP stereo 2×640×480 @25 + T265 | RS + GS | std + 163° fisheye | same 4 IMUs | AprilGrid 6×6 | CC BY 4.0 |
VI-rig 5-IMU (ours, virig-5imus-4cams) |
2× RealSense T265 = 4×848×800 @30 (two stereo fisheye pairs) | GS | 163° fisheye | GX3-25 @500 Hz, GX3-35 @100 Hz, MTi @400 Hz, 2× T265 BMI055 @200 Hz (5 IMUs) | AprilGrid 6×6 | CC BY 4.0 |
| EuRoC MAV | 2× Aptina MT9V034 752×480 @20 | GS | standard | ADIS16448 @200 Hz | AprilGrid + checkerboard | CC BY |
| TUM-VI | 2× IDS uEye UI-3241LE 1024² @20 (+512² variant) | GS | 195° fisheye | BMI160 @200 Hz | AprilGrid 6×6 | CC BY 4.0 |
| TUM-RS | 2× uEye 1280×1024 @20 (right cam RS in eval seqs) | GS calib / RS eval | 126° | BMI160 @200 Hz | AprilGrid 6×6 | not stated |
| TUM-VIE (frame cams) | 2× uEye 1024² @20 | GS | 101°×76° | BMI160 @200 Hz | AprilGrid 6×6 | CC BY 4.0 |
| TUM RGB-D | Kinect v1 / Asus Xtion 640×480 @30 (RGB + IR) | RS | standard | — | checkerboards | CC BY 4.0 |
| Double Sphere calib (TUM) | IDS uEye body, 5 lenses | GS | 195/183/150/126/122° | — | AprilGrid 6×6 | not stated |
| Monado MSD (MIC) | Valve Index stereo 2×960² @54 | GS | ~120° fisheye | — (camera-only here) | AprilGrid (3 cm) | CC BY 4.0 |
| Hilti 2022 | Alphasense Core 5× 720×540 @40 | GS | 126°×92° | BMI085 @400 Hz | AprilGrid 7×12 (15 cm) | CC BY-NC-SA |
| Hilti 2023 handheld | Alphasense Core 5× 720×540 @10 | GS | ~126°×92° | BMI085 @200 Hz | AprilGrid 6×6 | CC BY-NC-SA |
| PennCOSYVIO | 3× GoPro Hero 4 1920×1080 @30 | RS | 118° wide | — | checkerboard (10.8 cm) | citation-only |
| Urban fisheye (ImprovedOcamCalib) | Lensation BF2M15520 / BF2M12520 + GoPro Hero | GS | ~185° | — | checkerboards | GPL v2 |
Our captures are hosted (CC-BY-4.0); third-party sets ship as download recipes from their original hosts — we never re-host third-party bytes.
Participate
Anyone with a (free) Hugging Face account can contribute — no org membership needed, everything goes through Pull Requests on each repo's Community tab. Three ways in:
- 📊 Submit your calibration results. Run your toolbox on any Calibench
dataset and PR the output records to
yangyulin/calibench-submissions. CI validates the schema and re-runs the evaluation (simulation vs. held-out ground-truth; real data via consistency / cross-tool agreement), then publishes a verified record tocalibench-vi/results. - 🤝 Contribute a new dataset. PR your rig to
calibench-vi/communitywith a dataset card + license (data you can't relicense ships as a download recipe, not re-hosted bytes). Once merged it joins the catalog and becomes an eligible benchmark target. - 🏆 Climb the leaderboard. Verified results (1) flow straight into the live leaderboard — ranked per task-track over a fixed roster, so numbers are comparable and can't be self-reported. New datasets (2) expand the rosters everyone competes on.
Full how-to: community design & contribution guide. Questions and discussion: use each repo's Community tab.
How to cite
If you use Calibench, cite (1) the benchmark, (2) the MVIS paper for our
MVIS-rig captures (virig-4imus-3cams, virig-4imus-3cams-aruco,
virig-4imus-4cams; the newer virig-5imus-4cams has no dataset paper yet — cite
Calibench), and (3) each upstream dataset you use (each dataset card lists the
required citations). A CITATION.cff ships with calibench-vi/datasets (powers
the "Cite this dataset" button).
@misc{calibench2026,
author = {Yang, Yulin and others}, % TODO: full author list
title = {Calibench: A Camera-IMU Calibration Benchmark},
year = {2026},
howpublished = {Hugging Face Datasets},
note = {https://huggingface.co/calibench-vi},
doi = {TODO} % filled by HF "Generate DOI"
}
@article{Yang2024MVIS, % cite for our virig-4imus-3cams / virig-4imus-4cams captures
author = {Yang, Yulin and Geneva, Patrick and Huang, Guoquan},
title = {Multi-visual-inertial system: Analysis, calibration, and estimation},
journal = {The International Journal of Robotics Research}, year = {2024},
doi = {10.1177/02783649241245726}
}
License: datasets we host are released under CC-BY-4.0; third-party datasets remain under their own licenses and are linked, not re-hosted.
models 0
None public yet