File size: 1,867 Bytes
0e39cc8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | ---
license: apache-2.0
tags:
- segmentation
- efficientsam
- aria-digital-twin
- reproducibility
---
# Checkpoints — Projection-Assisted Segmentation Reproduction
Model weights for reproducing projection-assisted gaze-prompted segmentation on
the Aria Digital Twin dataset.
Companion data: [`tianxia2/projseg-adt-seq144-subset`](https://huggingface.co/datasets/tianxia2/projseg-adt-seq144-subset).
## Files
| File | Size | What |
|---|---|---|
| `efficient_sam_vitt.pt` | 40 MB | **EfficientSAM ViT-Tiny.** The anchor segmenter, prompted with a single gaze point. Unmodified upstream weights from [yformer/EfficientSAM](https://github.com/yformer/EfficientSAM) (Apache-2.0), redistributed for convenience. |
| `refinenetgaze.pth` | 5.9 MB | **RefineNetGaze.** Optional U-Net that refines a reused mask from RGB + prior mask + gaze heatmap (5-channel input, base=16). Trained by us on ADT. |
## Usage
```bash
huggingface-cli download tianxia2/projseg-checkpoints --local-dir checkpoints
```
The reproduction artifact expects them at `checkpoints/` (override with
`PROJSEG_CKPT_ROOT`).
## Notes
- `refinenetgaze.pth` is stored as a **plain tensor state dict**, so it loads
under PyTorch ≥ 2.6's default `weights_only=True`. Construct the model as
`RefineNetGaze(base=16)`.
- Refinement is **optional and off by default**. On the reported EfficientSAM +
`dense_gt` configuration it *lowers* mIoU (0.3283 → 0.3111), because it was
trained against a different anchor-mask distribution than the one ESAM
anchors produce. It is shipped for completeness and ablation, not as part of
the headline result.
- EfficientSAM is class-agnostic. Prompted with one gaze point it segments
*some* coherent region around that point, which does not always coincide with
the annotated ADT instance — hence modest absolute mIoU on this benchmark.
|