README.md

---
license: mit
tags:
  - 3d-vision
  - referring-expression-segmentation
  - point-cloud
  - scanrefer
  - multi3drefer
library_name: generic
---

# ReDiCo: Relay-Mediated Visual Updating for 3D Referring Segmentation

Official pretrained checkpoints for **ReDiCo**, a 3D Referring Expression Segmentation (3D-RES / 3D-GRES) method.

[Code](https://github.com/songchuanle-1/ReDiCo)

![framework](https://raw.githubusercontent.com/songchuanle-1/ReDiCo/main/docs/framework.png)

## Model Description

3D Referring Expression Segmentation (3D-RES) aims to produce point-level masks for objects described by natural language in complex 3D scenes. Existing query-based methods usually keep visual tokens largely fixed during decoding, while dense visual self-attention can refresh them but at $O(N_{sp}^2)$ cost.

**ReDiCo** (**Re**lay attention with **Di**versity and **Co**verage regularization) reformulates visual token updating as *token-relay-token* communication: language-guided relay tokens first aggregate global scene information, then distribute the refined context back to the visual stream, reducing the dominant interaction from $O(N_{sp}^2)$ to $O(N_q N_{sp})$ with $N_q \ll N_{sp}$. Geometry-aware relay attention injects 3D spatial proximity into relay assignments, while coverage and diversity regularization on the stage-1 relay-to-visual assignment jointly prevent region omission and relay redundancy.

With 3D+2D inputs, ReDiCo achieves **62.8%** Acc@0.25 / **51.7%** mIoU on 3D-RES (ScanRefer) and **73.8%** Acc@0.25 / **53.7%** mIoU on 3D-GRES (Multi3DRefer), reducing FLOPs by **33.5%** vs. dense self-attention.

Two checkpoints:

| File | Task | Dataset |
|---|---|---|
| `redico_res.pth` | 3D-RES | ScanRefer |
| `redico_gres.pth` | 3D-GRES | Multi3DRefer |

## Metrics (val)

| Task | Dataset | mIoU | Acc@0.5 | Acc@0.25 |
|---|---|---|---|---|
| 3D-RES | ScanRefer | 51.7 | 57.1 | 62.8 |
| 3D-GRES | Multi3DRefer | 53.7 | 54.1 | 73.8 |

## Usage

```bash
# clone code
git clone https://github.com/songchuanle-1/ReDiCo.git
cd ReDiCo

# download checkpoints
wget https://huggingface.co/chuanle/ReDiCo/resolve/main/redico_res.pth
wget https://huggingface.co/chuanle/ReDiCo/resolve/main/redico_gres.pth

# inference — 3D-RES (ScanRefer)
python tools/test.py configs/redico.yaml --checkpoint redico_res.pth --gpu_ids 0 --out ./output

# inference — 3D-GRES (Multi3DRefer)
python tools/test.py configs/redico_gres.yaml --checkpoint redico_gres.pth --gpu_ids 0 --out ./output
```

Data and pretrained backbone (`sp_unet_backbone.pth`) download links are in the [GitHub repo README](https://github.com/songchuanle-1/ReDiCo) (Baidu Netdisk mirror).