---
license: apache-2.0
library_name: pytorch
tags:
- 3d
- point-cloud
- scannet
- 3d-question-answering
- 3d-dense-captioning
---

# PerLA: Perceptive 3D Language Assistant (CVPR 2025)

Checkpoints for [PerLA](https://github.com/gfmei/PerLA), a 3D language assistant
for question answering and dense captioning in indoor scenes.
[Paper](https://arxiv.org/abs/2411.19774) · [Code](https://github.com/gfmei/PerLA) ·
[Project page](https://gfmei.github.io/PerLA/)

Each checkpoint holds the trained PerLA weights (detector, merger, Q-Former,
prompt encoder and projections) and its model settings. The frozen OPT-1.3B
language model is loaded separately from `facebook/opt-1.3b`.

| File | Benchmark | CIDEr | BLEU-4 | METEOR | ROUGE-L |
| --- | --- | ---: | ---: | ---: | ---: |
| `perla_scanqa.pth` | ScanQA val | 77.06 | 13.91 | 15.46 | 37.00 |

The model is trained on ScanQA from the pretrained detector, without generalist
pretraining (single run, seed 0; best validation epoch).

## Usage

```bash
git clone https://github.com/gfmei/PerLA.git && cd PerLA
hf download RyanMei/PerLA perla_scanqa.pth --local-dir ckpts
bash scripts/eval.sh scanqa ckpts/perla_scanqa.pth
```

See the [code repository](https://github.com/gfmei/PerLA) for installation,
data preparation and training.

## Citation

```bibtex
@inproceedings{mei2025PerLA,
  title     = {PerLA: Perceptive 3D Language Assistant},
  author    = {Guofeng Mei and Wei Lin and Luigi Riz and Yujiao Wu and Fabio Poiesi and Yiming Wang},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2025}
}
```