Spaces:
Running on Zero
Running on Zero
File size: 2,278 Bytes
861c9ce 0122a25 861c9ce 0122a25 861c9ce 0122a25 7266be4 0122a25 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | ---
title: Map Det3D
emoji: 🌍
colorFrom: red
colorTo: indigo
sdk: gradio
sdk_version: 5.49.1
python_version: '3.12'
app_file: app.py
pinned: false
license: apache-2.0
short_description: Streaming multi-view 3D object detection and tracking
models:
- RoyYang0714/Map-Det3D
- facebook/map-anything
---
# Map-Det3D
Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection
from Streaming Inputs.
The demo streams the posed RGB frames of a scene through the model one at a
time and lifts every detection into a single metric world frame, so the boxes
accumulate into one consistent 3D layout with persistent track IDs. Results are
shown in an embedded [Rerun](https://rerun.io) viewer.
- 🌟 [GitHub Repository](https://github.com/cvg/Map-Det3D)
- 📄 [Paper](https://arxiv.org/abs/2608.12179)
## Running it locally
```bash
pip install -r requirements.txt
python app.py
```
## Notes on the deployment
- The Space needs **ZeroGPU** hardware. `@spaces.GPU` requests GPU time that
scales with the number of frames, up to the 300 s ceiling.
- `sdk_version` is pinned to a Gradio 5 release because `gradio_rerun==0.26.0`
is a Gradio 5 custom component. It has to stay on the same version as the
`rerun-sdk` the model code logs with, so bumping one means bumping all three.
- `vis4d_cuda_ops/` shadows the package of the same name, because pip cannot
install the real one here: it has no wheels, and pip compiles it in an
isolated build environment that resolves its own newest torch while the Space
caps the runtime torch lower, so the extension fails to import with
`undefined symbol: c10::NotImplementedError`. The local package compiles only
the upstream CPU sources, against the torch the Space runs, and routes
deformable attention through the grid-sample implementation already in
`mapdet3d`. Detections and tracks come out identical; see its docstring.
- `mapdet3d/` is vendored from the research repository so the Space does not
depend on a published package.
- The model weights are pulled from
[`RoyYang0714/Map-Det3D`](https://huggingface.co/RoyYang0714/Map-Det3D) at
start-up, which also pulls
[`facebook/map-anything`](https://huggingface.co/facebook/map-anything)
(CC-BY-NC-4.0) as the geometry backbone.
|