README.md

---
title: AQ3D 3D Instance Segmentation
emoji: 🪑
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 6.26.0
app_file: app.py
short_description: Segment indoor 3D scans into object instances
python_version: "3.12"
startup_duration_timeout: 30m
---

# AQ3D — Adaptive Query Transformer for 3D Instance Segmentation

Upload an indoor **surface mesh** (`.ply` / `.obj` / `.glb`) and AQ3D labels every
object in it with one of the 198 ScanNet200 instance classes.

* Paper: [AQ3D: Adaptive Query Transformer for 3D Instance Segmentation](https://huggingface.co/papers/2608.30618)
* Code: [kenomo/aq3d](https://github.com/kenomo/aq3d) · Weights: [kenomo/aq3d](https://huggingface.co/kenomo/aq3d)

This Space runs the **`aq3d_scannet200_volt`** checkpoint (Volt-B point transformer
backbone + AQTD adaptive-query decoder, 44.1 mAP on ScanNet200 val).

## How the pipeline is reproduced

The upstream inference stack depends on several compiled extensions that have no
wheels for the Space's Blackwell GPU (`segmentator`, `torch_scatter`,
`torch_geometric`, `flash-attn`). They are replaced here by exact,
dependency-free equivalents:

| upstream | here |
| --- | --- |
| `segmentator.segment_mesh` (C++/libtorch Felzenszwalb–Huttenlocher) | Numba port in `superpoints.py` (same edge order, weights, two-pass union–find) |
| `torch_scatter` / `torch_geometric.nn.pool.fps` | `nnutils.py` (`scatter_*` via `scatter_add_`/`scatter_reduce_`, greedy FPS) |
| `flash_attn_varlen_qkvpacked_func` | `F.scaled_dot_product_attention` per sequence (identical for batch size 1) |
| `timm` blocks | inlined (`Mlp`, identity LayerScale/DropPath at eval) |

Everything else — `MeanCoord` → `NormalizeColor(-1, 1)` → `GridSample(0.02)` with
FNV hashing, superpoint attention pooling, superpoint NMS (0.8), adaptive top-k,
mask scoring, and the point-count threshold — follows the official ScanNet200
validation configuration verbatim.

## Example scans

The bundled examples are CC BY 4.0 photogrammetry / LiDAR room scans from
[Objaverse](https://objaverse.allenai.org/), redistributed via Zenodo:

* **Attic** — *"my room and the mess therein"*, [Zenodo record 10380976](https://zenodo.org/records/10380976) — CC BY 4.0
* **Historic building interior** — iPhone LiDAR capture, [Zenodo record 10325220](https://zenodo.org/records/10325220) — CC BY 4.0

## Caveats

AQ3D is trained on ScanNet200, i.e. RGB-D reconstructions of real rooms in metres
with Z up. Meshes far outside that domain (synthetic scenes, single objects,
non-metric units) will still run but the labels will be unreliable. Use the
**Scene orientation & scale** accordion if the automatic guess is wrong.