ORGANIC-3D encoder (enc4_organic3d)
A 3D CT image–text encoder that scores any sentence against a CT volume and follows the image evidence for each finding: its score drops when the finding's organ is erased, stays when another organ is erased, and prefers the sentence that names the correct side. Paper: ORGANIC-3D: Grounding 3D CT Vision Language Models in Organ-Level Image–Text Coupling.
· Code: https://github.com/lzl199704/organic3d
· Benchmark: https://huggingface.co/datasets/zllhhhh/organic3d-benchmark
· VLM: https://huggingface.co/zllhhhh/organic3d-vlm
Research use only. Not for clinical use.
Usage
pip install -e <organic3d repository> # the `organic` package and its requirements
python organic3d.py encoder --ct scan.nii.gz --region chest --totalseg \
--sentence "There is a nodule in the right upper lobe."
The script downloads this repository, resamples the CT to 1 mm RAS, centres a 384 mm crop with TotalSegmentator (or an
estimate without it) and prints the score of every finding of the region (Δf = S(V, "There is f.") − S(V, "No f."))
and of each --sentence. From Python: from organic3d import Encoder.
Model
| Image tower | COLIPRI 3D ViT (Primus-M), 192³ input at 2 mm, fine-tuned |
| Anatomy tokens | 144 learned queries (16 organ groups × 8, left and right lung and kidney separate, + 16 global), 2 cross-attention layers |
| Score | each sentence is routed to the tokens of the organs it names and pools them with text-conditioned weights; cosine to the CXR-BERT sentence embedding / temperature |
| Files | model.safetensors (272 M parameters, fp32), config.json, organic3d.py |
Training
Public data only: the CT-RATE and Merlin training splits (3 : 2). Contrastive loss, classification loss and the four
ORGANIC losses (organ-erasure with a control organ, mirror, matched-pair, anatomy-attention), 12,000 steps on 4 × H100.
Config: configs/encoder/enc4_organic3d.yaml in the code repository.
Results (paper Table 1, %)
| CT-RATE | RAD-ChestCT | Merlin | |
|---|---|---|---|
| MLC-AUROC | 87.7 | 79.7 | 87.7 |
| Matched-pair AUROC (MP) | 82.5 | 78.2 | 86.5 |
| Organ-erasure ΔAUROC (OE) | 28.5 | 18.4 | 21.3 |
| Laterality accuracy (Side) | 76.9 | – | 63.8 |
Lung lobe erasure hit rate 74.7 (CT-RATE); lesion AUROC 79.5 and lesion-erasure hit rate 74.9 (ReXGroundingCT); zero-shot tumour detection 75.0 and lesion-erasure hit rate 70.7 (AbdomenAtlas 3.0).
Limitations
Trained on chest CT from one centre (CT-RATE) and abdominal CT from one centre (Merlin) with report-derived labels; performance elsewhere is unknown. Laterality is far from perfect (Side 76.9 / 63.8). Licence terms of COLIPRI, CT-RATE and Merlin apply.
Citation
@article{liu2026organic3d,
title = {ORGANIC3D: Grounding 3D CT Vision Language Models in Organ Level Image Text Coupling},
author = {Liu, Zelong and Ahmed, Hamza and Lee, Wynn Wingyi and Torjman, Mickael and Dimitrakakis, Alexander and
Lu, Jiahua and Deyer, Timothy and Taouli, Bachir and Fayad, Zahi A. and Mei, Xueyan},
journal = {arXiv preprint arXiv:2611.23529},
year = {2026}
}
- Downloads last month
- 5