README.md

metadata

title: AQ3D 3D Instance Segmentation
emoji: 🪑
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 6.26.0
app_file: app.py
short_description: Segment indoor 3D scans into object instances
python_version: '3.12'
startup_duration_timeout: 30m

AQ3D — Adaptive Query Transformer for 3D Instance Segmentation

Upload an indoor surface mesh (.ply / .obj / .glb) and AQ3D labels every object in it with one of the 198 ScanNet200 instance classes.

• Paper: AQ3D: Adaptive Query Transformer for 3D Instance Segmentation
• Code: kenomo/aq3d · Weights: kenomo/aq3d

This Space runs the aq3d_scannet200_volt checkpoint (Volt-B point transformer backbone + AQTD adaptive-query decoder, 44.1 mAP on ScanNet200 val).

How the pipeline is reproduced

The upstream inference stack depends on several compiled extensions that have no wheels for the Space's Blackwell GPU (segmentator, torch_scatter, torch_geometric, flash-attn). They are replaced here by exact, dependency-free equivalents:

upstream	here
segmentator.segment_mesh (C++/libtorch Felzenszwalb–Huttenlocher)	Numba port in superpoints.py (same edge order, weights, two-pass union–find)
torch_scatter / torch_geometric.nn.pool.fps	nnutils.py (scatter_* via scatter_add_/scatter_reduce_, greedy FPS)
flash_attn_varlen_qkvpacked_func	F.scaled_dot_product_attention per sequence (identical for batch size 1)
timm blocks	inlined (Mlp, identity LayerScale/DropPath at eval)

Everything else — MeanCoord → NormalizeColor(-1, 1) → GridSample(0.02) with FNV hashing, superpoint attention pooling, superpoint NMS (0.8), adaptive top-k, mask scoring, and the point-count threshold — follows the official ScanNet200 validation configuration verbatim.

Example scans

The bundled examples are CC BY 4.0 photogrammetry / LiDAR room scans from Objaverse, redistributed via Zenodo:

• Attic — "my room and the mess therein", Zenodo record 10380976 — CC BY 4.0
• Historic building interior — iPhone LiDAR capture, Zenodo record 10325220 — CC BY 4.0

Caveats

AQ3D is trained on ScanNet200, i.e. RGB-D reconstructions of real rooms in metres with Z up. Meshes far outside that domain (synthetic scenes, single objects, non-metric units) will still run but the labels will be unreliable. Use the Scene orientation & scale accordion if the automatic guess is wrong.