ReDiCo: Relay-Mediated Visual Updating for 3D Referring Segmentation
Official pretrained checkpoints for ReDiCo, a 3D Referring Expression Segmentation (3D-RES / 3D-GRES) method.
Model Description
3D Referring Expression Segmentation (3D-RES) aims to produce point-level masks for objects described by natural language in complex 3D scenes. Existing query-based methods usually keep visual tokens largely fixed during decoding, while dense visual self-attention can refresh them but at $O(N_{sp}^2)$ cost.
ReDiCo (Relay attention with Diversity and Coverage regularization) reformulates visual token updating as token-relay-token communication: language-guided relay tokens first aggregate global scene information, then distribute the refined context back to the visual stream, reducing the dominant interaction from $O(N_{sp}^2)$ to $O(N_q N_{sp})$ with $N_q \ll N_{sp}$. Geometry-aware relay attention injects 3D spatial proximity into relay assignments, while coverage and diversity regularization on the stage-1 relay-to-visual assignment jointly prevent region omission and relay redundancy.
With 3D+2D inputs, ReDiCo achieves 62.8% Acc@0.25 / 51.7% mIoU on 3D-RES (ScanRefer) and 73.8% Acc@0.25 / 53.7% mIoU on 3D-GRES (Multi3DRefer), reducing FLOPs by 33.5% vs. dense self-attention.
Two checkpoints:
| File | Task | Dataset |
|---|---|---|
redico_res.pth |
3D-RES | ScanRefer |
redico_gres.pth |
3D-GRES | Multi3DRefer |
Metrics (val)
| Task | Dataset | mIoU | Acc@0.5 | Acc@0.25 |
|---|---|---|---|---|
| 3D-RES | ScanRefer | 51.7 | 57.1 | 62.8 |
| 3D-GRES | Multi3DRefer | 53.7 | 54.1 | 73.8 |
Usage
# clone code
git clone https://github.com/songchuanle-1/ReDiCo.git
cd ReDiCo
# download checkpoints
wget https://huggingface.co/chuanle/ReDiCo/resolve/main/redico_res.pth
wget https://huggingface.co/chuanle/ReDiCo/resolve/main/redico_gres.pth
# inference — 3D-RES (ScanRefer)
python tools/test.py configs/redico.yaml --checkpoint redico_res.pth --gpu_ids 0 --out ./output
# inference — 3D-GRES (Multi3DRefer)
python tools/test.py configs/redico_gres.yaml --checkpoint redico_gres.pth --gpu_ids 0 --out ./output
Data and pretrained backbone (sp_unet_backbone.pth) download links are in the GitHub repo README (Baidu Netdisk mirror).