---
license: other
license_name: mixed-see-card
library_name: numpy
tags:
- image-segmentation
- open-vocabulary-segmentation
- computational-pathology
- training-free
- arxiv:2611.05599
pipeline_tag: image-segmentation
---

# CAPTOR pretrained tables

Term-by-word tables for **CAPTOR** (Caption-Anchored Patch-Term Occurrence
Representation), text-encoder-free grounding by counting caption co-occurrence
over a frozen image encoder's visual words. Each table holds a frozen encoder's
visual words (k-means centroids of its patch tokens) and one signature per term,
counted from a caption corpus. With the encoder, a table scores every term at every
patch of an image; no text encoder is involved.

Authors: Elad Yoshai and Natan T. Shaked, Tel Aviv University.
Paper: https://arxiv.org/abs/2611.05599
Code: https://github.com/EladYoshai/captor

| file | encoder | K × vocabularies | caption corpus | terms | reference result |
|---|---|---|---|---|---|
| `coco_dinov2reg_k640.npz` | DINOv2 ViT-B/14 with registers, 196 px | 640 × 4, joint words | COCO train2017, 15,000 images | 78 COCO objects | COCO val2017 reporting half: mIoU 0.5654 |
| `pathcap_conch_k80.npz` | CONCH image trunk | 80 × 4 | PathCap, 32,463 figures | 25 tissue terms | BCSS, 62 held-out patients: mIoU 0.4966 |

`MANIFEST.json` gives each file's settings, size and SHA-256.

## Use

```bash
pip install git+https://github.com/EladYoshai/captor.git
```

```python
from captor import CAPTOR

model = CAPTOR.from_pretrained("coco_dinov2reg_k640")    # fetched from this repository
seg = model.segment("photo.jpg", ["cat", "dog", "bed"])
```

Each file is a plain `.npz` (no pickle): `centroids` (V, K, D), `signatures`
(V, K, T), `terms`, corpus moments, optional joint-word arrays, and a JSON `meta`
entry with the read-out settings. The format is documented in `captor/io.py`.

## Licences

The tables contain no images. They are statistics counted from caption corpora over
frozen encoders' features, and are to be used under the terms of those sources:

- `coco_dinov2reg_k640`: COCO captions (CC BY 4.0) and DINOv2 (Apache 2.0).
- `pathcap_conch_k80`: PathCap captions (CC BY-NC 2.0; Sun et al., PathAsst, 2023)
  over CONCH features (CONCH: CC BY-NC-ND 4.0, gated). Non-commercial research use only.

The encoders themselves are not included and must be obtained from their own
model pages under their own terms.

## Citation

```bibtex
@article{yoshai2026captor,
  title   = {{CAPTOR}: Text-Encoder-Free Grounding by Counting Caption Co-occurrence
             over the Visual Words of a Frozen Encoder},
  author  = {Yoshai, Elad and Shaked, Natan T.},
  journal = {arXiv preprint arXiv:2611.05599},
  year    = {2026}
}
```