File size: 1,631 Bytes
35dd0e5 13fe35b 35dd0e5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | ---
license: apache-2.0
library_name: pytorch
tags:
- 3d
- point-cloud
- scannet
- 3d-question-answering
- 3d-dense-captioning
---
# PerLA: Perceptive 3D Language Assistant (CVPR 2025)
Checkpoints for [PerLA](https://github.com/gfmei/PerLA), a 3D language assistant
for question answering and dense captioning in indoor scenes.
[Paper](https://arxiv.org/abs/2411.19774) · [Code](https://github.com/gfmei/PerLA) ·
[Project page](https://gfmei.github.io/PerLA/)
Each checkpoint holds the trained PerLA weights (detector, merger, Q-Former,
prompt encoder and projections) and its model settings. The frozen OPT-1.3B
language model is loaded separately from `facebook/opt-1.3b`.
| File | Benchmark | CIDEr | BLEU-4 | METEOR | ROUGE-L |
| --- | --- | ---: | ---: | ---: | ---: |
| `perla_scanqa.pth` | ScanQA val | 77.06 | 13.91 | 15.46 | 37.00 |
The model is trained on ScanQA from the pretrained detector, without generalist
pretraining (single run, seed 0; best validation epoch).
## Usage
```bash
git clone https://github.com/gfmei/PerLA.git && cd PerLA
hf download RyanMei/PerLA perla_scanqa.pth --local-dir ckpts
bash scripts/eval.sh scanqa ckpts/perla_scanqa.pth
```
See the [code repository](https://github.com/gfmei/PerLA) for installation,
data preparation and training.
## Citation
```bibtex
@inproceedings{mei2025PerLA,
title = {PerLA: Perceptive 3D Language Assistant},
author = {Guofeng Mei and Wei Lin and Luigi Riz and Yujiao Wu and Fabio Poiesi and Yiming Wang},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2025}
}
```
|