Configuration Parsing Warning:In adapter_config.json: "peft.task_type" must be a string

ORGANIC-3D VLM (vlm4_organic3d)

A 3D CT vision–language model that answers yes/no finding questions and writes findings reports, and whose answers depend on the image: it withdraws 96–99% of correctly stated findings when their organ is erased. The frozen ORGANIC-3D encoder (https://huggingface.co/zllhhhh/organic3d-encoder) feeds its 144 anatomy tokens, in a fixed organ order, through an MLP into Qwen3.5-4B with LoRA. Paper: ORGANIC-3D: Grounding 3D CT Vision Language Models in Organ-Level Image–Text Coupling.

· Code: https://github.com/lzl199704/organic3d

· Benchmark:https://huggingface.co/datasets/zllhhhh/organic3d-benchmark

Research use only. Not for clinical use. Generated reports can contain findings that are not in the image and miss findings that are.

Usage

pip install -e <organic3d repository>
python organic3d.py vlm --ct scan.nii.gz --region abdomen --totalseg --report \
    --question "Is there a kidney mass in this CT scan? Answer Yes or No."

The script downloads this repository, the encoder and Qwen/Qwen3.5-4B (pinned revision), and prints P(Yes) for every finding of the region, any --question, and with --report an answer-first report (the findings it answers Yes to, then the report, started with "Trachea" on chest and "Lower thorax:" on abdominal CT).

Files

adapter_model.safetensors / adapter_config.json (LoRA, r 32, α 64; 244 MB), projector.safetensors (MLP, 34 MB), config.json, organic3d.py. The base model is not included.

Training

Public data only: CT-RATE and Merlin training splits. Language-modelling loss on yes/no answers and answer-first reports, the dependence margin (the correct answer must be more likely on the patient's own scan than on a matched partner's scan or on the scan with the finding's organ erased, and unchanged when a control organ is erased), the answer-attention loss and a report margin. 10,000 steps on 4 × B200, encoder frozen. Config: configs/vlm/vlm4_organic3d.yaml in the code repository.

Results (paper Tables 2 and 3, %)

CT-RATE Merlin ReXGroundingCT
Answer AUROC 86.3 87.6 79.5
Balanced accuracy / F1 78.9 / 58.7 80.0 / 40.6 72.1 / 49.0
Matched-pair AUROC (MP) 82.8 86.0 74.1
Organ-erasure hit rate (OEH) 96.8 96.2 98.9
Free-text report F1 / CRG 56.8 / 52.9 37.7 / 49.4 –
Finding-based report F1 / IA-F1 59.6 / 37.6 48.2 / 37.4 –

Licence terms of Qwen3.5 (Apache-2.0), COLIPRI, CT-RATE and Merlin apply.

Citation

@article{liu2026organic3d,
  title   = {ORGANIC3D: Grounding 3D CT Vision Language Models in Organ Level Image Text Coupling},
  author  = {Liu, Zelong and Ahmed, Hamza and Lee, Wynn Wingyi and Torjman, Mickael and Dimitrakakis, Alexander and
             Lu, Jiahua and Deyer, Timothy and Taouli, Bachir and Fayad, Zahi A. and Mei, Xueyan},
  journal = {arXiv preprint arXiv:2611.23529},
  year    = {2026}
}
Downloads last month
4
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for zllhhhh/organic3d-vlm

Finetuned
Qwen/Qwen3.5-4B
Adapter
(1,207)
this model

Dataset used to train zllhhhh/organic3d-vlm

Paper for zllhhhh/organic3d-vlm