Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,588 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: mixed-source-licenses
|
| 4 |
+
license_link: LICENSE
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
+
pretty_name: MedHEB-Bench
|
| 8 |
+
size_categories:
|
| 9 |
+
- 100K<n<1M
|
| 10 |
+
tags:
|
| 11 |
+
- medical
|
| 12 |
+
- multimodal
|
| 13 |
+
- retrieval
|
| 14 |
+
- embedding
|
| 15 |
+
- benchmark
|
| 16 |
+
- 2D
|
| 17 |
+
- 3D
|
| 18 |
+
- radiology
|
| 19 |
+
- pathology
|
| 20 |
+
configs:
|
| 21 |
+
- config_name: 2D-test
|
| 22 |
+
data_files:
|
| 23 |
+
- split: test
|
| 24 |
+
path: "2D_Task/**/*.json"
|
| 25 |
+
- config_name: 2D-train
|
| 26 |
+
data_files:
|
| 27 |
+
- split: train
|
| 28 |
+
path: "Train_Data/2D_Task/**/*.jsonl"
|
| 29 |
+
- config_name: 3D-test
|
| 30 |
+
data_files:
|
| 31 |
+
- split: test
|
| 32 |
+
path: "3D_Task/**/*.json"
|
| 33 |
+
- config_name: 3D-train
|
| 34 |
+
data_files:
|
| 35 |
+
- split: train
|
| 36 |
+
path: "Train_Data/3D_Task/**/*.jsonl"
|
| 37 |
+
- config_name: Text-test
|
| 38 |
+
data_files:
|
| 39 |
+
- split: test
|
| 40 |
+
path: "Text_Task/**/*.json"
|
| 41 |
+
- config_name: Text-train
|
| 42 |
+
data_files:
|
| 43 |
+
- split: train
|
| 44 |
+
path: "Train_Data/Text_Task/**/*.jsonl"
|
| 45 |
+
---
|
| 46 |
+
|
| 47 |
+
# MedHEB-Bench
|
| 48 |
+
|
| 49 |
+
This repository hosts the full **MedHEB-Bench** benchmark — a multimodal
|
| 50 |
+
medical embedding / retrieval benchmark spanning 2D imaging, 3D imaging,
|
| 51 |
+
and biomedical text. A small reviewer-facing sample (≈ 2.5 GB) is
|
| 52 |
+
available separately at
|
| 53 |
+
[MEDHEB/MedHEB-Bench_Sample](https://huggingface.co/datasets/MEDHEB/MedHEB-Bench_Sample).
|
| 54 |
+
|
| 55 |
+
## Repository layout
|
| 56 |
+
|
| 57 |
+
```
|
| 58 |
+
.
|
| 59 |
+
├── 2D_Task/
|
| 60 |
+
├── 2D_Images/
|
| 61 |
+
├── 3D_Task/
|
| 62 |
+
├── 3D_Images/
|
| 63 |
+
├── Text_Task/
|
| 64 |
+
├── Train_Data/
|
| 65 |
+
└── Preprocess/
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
### JSON record format
|
| 70 |
+
|
| 71 |
+
**Test JSON** (under `2D_Task/`, `3D_Task/`, `Text_Task/`):
|
| 72 |
+
|
| 73 |
+
```json
|
| 74 |
+
{
|
| 75 |
+
"qry_inst": "Find the same category image with the provided image:",
|
| 76 |
+
"qry_text": "",
|
| 77 |
+
"qry_img_path": "6cd606dc52e9.png",
|
| 78 |
+
"tgt_text": ["", "", "", "", ""],
|
| 79 |
+
"tgt_img_path": [
|
| 80 |
+
"82bb8a01935f.png",
|
| 81 |
+
"87b671c6d4c5.png",
|
| 82 |
+
"..."
|
| 83 |
+
]
|
| 84 |
+
}
|
| 85 |
+
```
|
| 86 |
+
|
| 87 |
+
**Train JSONL** (under `Train_Data/`) — chat-style records with
|
| 88 |
+
`messages` / `images` / `positive_messages`:
|
| 89 |
+
|
| 90 |
+
```json
|
| 91 |
+
{
|
| 92 |
+
"messages": [
|
| 93 |
+
{"role": "system", "content": "Represent the given image with the following question:"},
|
| 94 |
+
{"role": "user", "content": "<image>What is the diabetic retinopathy severity level observed in this retinal image?"}
|
| 95 |
+
],
|
| 96 |
+
"images": ["./2D_Images/APTOS/train_images/0024cdab0c1e.png"],
|
| 97 |
+
"positive_messages": [[{"role": "user", "content": "Mild"}]]
|
| 98 |
+
}
|
| 99 |
+
```
|
| 100 |
+
|
| 101 |
+
Image paths in test JSONs resolve relative to the matching sub-folder
|
| 102 |
+
under `2D_Images/` / `3D_Images/`. Train JSONL paths are relative to the
|
| 103 |
+
repository root.
|
| 104 |
+
|
| 105 |
+
## Restricted datasets — image archives NOT redistributed
|
| 106 |
+
|
| 107 |
+
Image archives for the following datasets are **not redistributed** here
|
| 108 |
+
because their upstream license / data-use agreements restrict re-hosting.
|
| 109 |
+
A `LICENSE_NOTICE.md` placeholder remains in each folder pointing to the
|
| 110 |
+
official source. Task-definition JSONs are kept so the benchmark
|
| 111 |
+
structure is visible.
|
| 112 |
+
|
| 113 |
+
- `2D_Images/ChestImagrome/` — Chest-ImaGenome (PhysioNet)
|
| 114 |
+
- `2D_Images/chexpert_plus/` — CheXpert-Plus (Stanford AIMI)
|
| 115 |
+
- `2D_Images/MIMIC-CXR-Report/` — MIMIC-CXR (PhysioNet)
|
| 116 |
+
- `2D_Images/MIMIC-CXR-T/` — MIMIC-CXR-LT (PhysioNet)
|
| 117 |
+
- `2D_Images/MIMIC-CXR-VQA/` — MIMIC-CXR-VQA (PhysioNet)
|
| 118 |
+
- `2D_Images/VindrCXR/` — VinDr-CXR (PhysioNet)
|
| 119 |
+
- `2D_Images/VindrMammo/` — VinDr-Mammo (PhysioNet)
|
| 120 |
+
- `2D_Images/RadImageNet-VQA/` — RadImageNet-VQA (HuggingFace)
|
| 121 |
+
- `3D_Images/CT_RATE/` — CT-RATE (HuggingFace)
|
| 122 |
+
- `3D_Images/MRNet/` — MRNet (Stanford ML Group)
|
| 123 |
+
|
| 124 |
+
# Dataset Download
|
| 125 |
+
|
| 126 |
+
All test sets used in our benchmark can be downloaded through the links provided in the table. Please note that, for some datasets, we use redistributed versions rather than the original official download sources: Kvasir, LC25000, and MRNet are obtained from Kaggle, while PublicHealthQA, CirrMRI600, and VQA_RAD are obtained from Hugging Face.
|
| 127 |
+
|
| 128 |
+
| Dataset | Dim | Modality | Download |
|
| 129 |
+
| --- | --- | --- | --- |
|
| 130 |
+
| MIMIC-CXR-LT | 2D | Chest X-ray | [](https://physionet.org/content/mimic-cxr/2.1.0/) |
|
| 131 |
+
| ISIC-2019 | 2D | Skin disease image | [](https://challenge.isic-archive.com/data/#2019) |
|
| 132 |
+
| Brain-Tumor-MRI | 2D | Brain MRI | [](https://www.kaggle.com/datasets/masoudnickparvar/brain-tumor-mri-dataset) |
|
| 133 |
+
| PathMNIST | 2D | Colon Pathology | [](https://zenodo.org/records/10519652) |
|
| 134 |
+
| ChestMNIST(NIH-ChestXray14) | 2D | Chest X-ray | [](https://zenodo.org/records/10519652) |
|
| 135 |
+
| DermaMNIST(HAM10000) | 2D | Skin Disease image | [](https://zenodo.org/records/10519652) |
|
| 136 |
+
| OCTMNIST(OCT) | 2D | Optical Coherence Tomography | [](https://zenodo.org/records/10519652) |
|
| 137 |
+
| TissueMNIST(BBBC051) | 2D | Kidney Cortex Microscope | [](https://zenodo.org/records/10519652) |
|
| 138 |
+
| BloodMNIST | 2D | Blood Cell image | [](https://zenodo.org/records/10519652) |
|
| 139 |
+
| OrganAMNIST(LiTS) | 2D | Abdominal CT scans(arxial) | [](https://zenodo.org/records/10519652) |
|
| 140 |
+
| OrganCMNIST(LiTS) | 2D | Abdominal CT scans(Coronal) | [](https://zenodo.org/records/10519652) |
|
| 141 |
+
| OrganSMNIST(LiTS) | 2D | Abdominal CT scans(Sagittal) | [](https://zenodo.org/records/10519652) |
|
| 142 |
+
| Kvasir | 2D | Gastrointestinal (GI) tract | [](https://datasets.simula.no/kvasir/) [](https://www.kaggle.com/datasets/meetnagadia/kvasir-dataset?select=kvasir-dataset) |
|
| 143 |
+
| APTOS-2019 | 2D | Optical Coherence Tomography | [](https://www.kaggle.com/competitions/aptos2019-blindness-detection/overview) |
|
| 144 |
+
| PanNuke | 2D | Pathdology | [](https://huggingface.co/datasets/RationAI/PanNuke) |
|
| 145 |
+
| VinDr-CXR | 2D | Chest X-ray | [](https://physionet.org/content/vindr-cxr/1.0.0/) |
|
| 146 |
+
| VinDr-Mammo | 2D | Breast X-ray | [](https://physionet.org/content/vindr-mammo/1.0.0/) |
|
| 147 |
+
| Chest-imagenome | 2D | Chest X-ray | [](https://physionet.org/content/chest-imagenome/1.0.0/) |
|
| 148 |
+
| Kvasir-SEG | 2D | Gastrointestinal (GI) tract | [](https://datasets.simula.no/kvasir-seg/) |
|
| 149 |
+
| USforKidneyMSBench(CT2USforKidneySeg) | 2D | Ultrasound | [](https://github.com/zekikus/MedSegBench) |
|
| 150 |
+
| UltrasoundNerveMSBench | 2D | Ultrasound | [](https://github.com/zekikus/MedSegBench) |
|
| 151 |
+
| UWSkinCancerMSBench | 2D | Skin Image | [](https://github.com/zekikus/MedSegBench) |
|
| 152 |
+
| Isic2018MSBench(ISIC 2018 Challenge) | 2D | Skin Image | [](https://github.com/zekikus/MedSegBench) |
|
| 153 |
+
| PolypGenMSBench | 2D | Gastrointestinal (GI) tract | [](https://github.com/zekikus/MedSegBench) |
|
| 154 |
+
| MIMIC-CXR | 2D | Chest X-ray | [](https://physionet.org/content/mimic-cxr/2.1.0/) |
|
| 155 |
+
| USData | 2D | Ultrasound | [](https://github.com/LijunRio/Ultrasound-Report-Generation) |
|
| 156 |
+
| Path-VQA | 2D | Pathdology | [](https://huggingface.co/datasets/flaviagiammarino/path-vqa) |
|
| 157 |
+
| PMC-VQA | 2D | Multi_modalities | [](https://huggingface.co/datasets/RadGenome/PMC-VQA) |
|
| 158 |
+
| ROCO-VQA | 2D | Multi_modalities | [](https://huggingface.co/datasets/adishourya/ROCO-QA) |
|
| 159 |
+
| MedPIX-VQA | 2D | Multi_modalities | [](https://huggingface.co/datasets/adishourya/MEDPIX-ClinQA) |
|
| 160 |
+
| RadImageNet-VQA | 2D | Multi_modalities | [](https://huggingface.co/datasets/raidium/RadImageNet-VQA) |
|
| 161 |
+
| VQA_RAD | 2D | Multi_modalities | [](https://osf.io/89kps/overview) [](https://huggingface.co/datasets/flaviagiammarino/vqa-rad)|
|
| 162 |
+
| MIMIC-CXR-VQA | 2D | Chest X-ray | [](https://physionet.org/content/medical-cxr-vqa-dataset/1.0.0/) |
|
| 163 |
+
| MedMCQA | Text | Biological text | [](https://drive.usercontent.google.com/download?id=15VkJdq5eyWIkfb_aoD3oS8i4tScbHYky&export=download&authuser=0) |
|
| 164 |
+
| PubMedQA | Text | Biological text | [](https://huggingface.co/datasets/qiaojin/PubMedQA) |
|
| 165 |
+
| MedicalQARetrieval | Text | Biological text | [](https://huggingface.co/datasets/mteb/medical_qa) |
|
| 166 |
+
| PublicHealthQA | Text | Biological text | [](https://www.kaggle.com/datasets/xhlulu/covidqa) [](https://huggingface.co/datasets/xhluca/publichealth-qa/discussions) |
|
| 167 |
+
| MMDental | 3D | Dental CBCT | [](https://springernature.figshare.com/articles/dataset/MMDental_-_A_multimodal_dataset_of_tooth_CBCT_images_with_expert_medical_records/28505276?file=53187695) |
|
| 168 |
+
| CT-RATE | 3D | Chest CT | [](https://huggingface.co/datasets/ibrahimhamamci/CT-RATE) |
|
| 169 |
+
| MRNet | 3D | Knee CT | [](https://aimi.stanford.edu/datasets/mrnet-knee-mris) [](https://www.kaggle.com/datasets/cjinny/mrnet-v1)|
|
| 170 |
+
| CirrMRI600 | 3D | 3D liver MRI | [](https://osf.io/cuk24/overview) [](https://huggingface.co/datasets/onkarsus13/AllMedicalCombine/tree/main)|
|
| 171 |
+
| OrganMNIST(LiTS) | 3D | Abdominal CT scans | [](https://zenodo.org/records/10519652) |
|
| 172 |
+
| NoduleMNIST | 3D | Chest CT | [](https://zenodo.org/records/10519652) |
|
| 173 |
+
| SynapseMNIST(MitoEM dataset) | 3D | Pathdology | [](https://zenodo.org/records/10519652) |
|
| 174 |
+
| RadGenomme-Chest-CT | 3D | Chest CT | [](https://huggingface.co/datasets/RadGenome/RadGenome-ChestCT) |
|
| 175 |
+
| M3D | 3D | Multi_modalities | [](https://www.modelscope.cn/datasets/GoodBaiBai88/M3D-Cap/tree/master/M3D_Cap/ct_case) |
|
| 176 |
+
| BraTS-PED | 3D | Brain MRI | [](https://www.synapse.org/Synapse:syn51514108) |
|
| 177 |
+
| SynthRAD2023 | 3D | Brain and Pelvis MRI,CT | [](https://zenodo.org/records/7260705) |
|
| 178 |
+
| HaN-Seg | 3D | Head and Neck MRI,CT | [](https://zenodo.org/records/7442914) |
|
| 179 |
+
| Chexpert_plus | 2D | Chest X-ray | [](https://stanfordaimi.azurewebsites.net/datasets/5158c524-d3ab-4e02-96e9-6ee9efc110a1) |
|
| 180 |
+
| Retinal Disease Classification | 2D | Optical Coherence Tomography | [](https://www.kaggle.com/datasets/andrewmvd/retinal-disease-classification) |
|
| 181 |
+
| LC25000 | 2D | Pathlogy | [](https://academictorrents.com/details/7a638ed187a6180fd6e464b3666a6ea0499af4af) [](https://www.kaggle.com/datasets/javaidahmadwani/lc25000) |
|
| 182 |
+
| OmniMedVQA | 2D | Multi_modalities | [](https://huggingface.co/datasets/foreverbeliever/OmniMedVQA) |
|
| 183 |
+
| BraTS-MEN | 3D | Brain MRI | [](https://www.synapse.org/Synapse:syn51514106) |
|
| 184 |
+
|
| 185 |
+
# Dataset Preprocess
|
| 186 |
+
|
| 187 |
+
The following datasets are supported and should be processed using the provided scripts in [`Preprocess/`](Preprocess):
|
| 188 |
+
|
| 189 |
+
- BraTS-MEN
|
| 190 |
+
- BraTS-PED
|
| 191 |
+
- CT-RATE
|
| 192 |
+
- ChirrMRI600
|
| 193 |
+
- HaN-Seg
|
| 194 |
+
- MedMNIST
|
| 195 |
+
- MRNet
|
| 196 |
+
- MMDental
|
| 197 |
+
- RadGenomme-Chest-CT
|
| 198 |
+
- SynthRAD
|
| 199 |
+
- PanNuke
|
| 200 |
+
- PublicHealthQA
|
| 201 |
+
- ROCO-QA
|
| 202 |
+
- ChestImagenome
|
| 203 |
+
- Gastrointestinal
|
| 204 |
+
- Skin-Image
|
| 205 |
+
- Ultrasound
|
| 206 |
+
- VinDr-CXR
|
| 207 |
+
- VinDr-Mammo
|
| 208 |
+
|
| 209 |
+
## 3D_Task
|
| 210 |
+
|
| 211 |
+
**1. BraTS-MEN**
|
| 212 |
+
- a. [Preprocess/3D_Task/BraTS-MEN/cut_slides.py](Preprocess/3D_Task/BraTS-MEN/cut_slides.py)
|
| 213 |
+
|
| 214 |
+
**Description**: `cut_slides.py` — This script converts the original `t1n.nii.gz` files from each BraTS-MEN case into 2D T1-weighted MRI slices. We follow the test split defined by the impression-text annotations in [RadGenome-Brain_MRI](https://huggingface.co/datasets/JiayuLei/RadGenome-Brain_MRI). `INPUT_ROOT` should be set to `./BraTS-MEN-Train`.
|
| 215 |
+
|
| 216 |
+
**2. BraTS-PED**
|
| 217 |
+
- a. [Preprocess/3D_Task/BraTS-PED/cut_slides.py](Preprocess/3D_Task/BraTS-PED/cut_slides.py)
|
| 218 |
+
|
| 219 |
+
**Description**: `cut_slides.py` — This script converts the original `t1n.nii.gz` and `t2w.nii.gz` files from each BraTS-PED case into 2D T1-weighted and T2-weighted MRI slices. `INPUT_ROOT` should be set to `./BraTS2023-PED`.
|
| 220 |
+
|
| 221 |
+
**3. CT-RATE**
|
| 222 |
+
- a. [Preprocess/3D_Task/CT-RATE/cut_slides.py](Preprocess/3D_Task/CT-RATE/cut_slides.py)
|
| 223 |
+
- b. [Preprocess/3D_Task/CT-RATE/CT2Xray.py](Preprocess/3D_Task/CT-RATE/CT2Xray.py)
|
| 224 |
+
|
| 225 |
+
**Description**: `cut_slides.py` — This script converts the original 3D CT volumes from each case into 2D CT slices. `CT2Xray.py` — This script converts the original 3D CT volumes into 2D chest X-ray projection images. `INPUT_ROOT` should be set to `./CT_DATA/data_volumes/dataset`.
|
| 226 |
+
|
| 227 |
+
**4. ChirrMRI600**
|
| 228 |
+
- a. [Preprocess/3D_Task/ChirrMRI600/cut_slides_split.py](Preprocess/3D_Task/ChirrMRI600/cut_slides_split.py)
|
| 229 |
+
|
| 230 |
+
**Description**: ChirrMRI600 contains liver MRI cases with different cirrhosis grades and healthy controls. We label healthy cases as 'No liver cirrhosis'. `Cirrhosis_T1_slide_folders.csv` and `Cirrhosis_T2_slide_folders.csv` define the split after adding the healthy cases. `cut_slides_split.py` converts 3D liver MRI volumes into 2D slices and organizes them according to the CSV split. Place the two CSV files in the original data folder and set this folder as the input path.
|
| 231 |
+
|
| 232 |
+
**5. HaN-Seg**
|
| 233 |
+
- a. [Preprocess/3D_Task/HaN-Seg/cut_slides.py](Preprocess/3D_Task/HaN-Seg/cut_slides.py)
|
| 234 |
+
|
| 235 |
+
**Description**: `cut_slides.py` — This script converts the original 3D CT volumes and MRI volumes into 2D slices, the input root shoud be set to `./HaN-Seg/set_1`.
|
| 236 |
+
|
| 237 |
+
**6. MNIST**
|
| 238 |
+
- a. [Preprocess/3D_Task/MNIST/unzip_cut_slides.py](Preprocess/3D_Task/MNIST/unzip_cut_slides.py)
|
| 239 |
+
|
| 240 |
+
**Description**: `unzip_cut_slides.py` — This script extracts the downloaded Zenodo `.npz` files and converts the original 3D `.npy` files into 2D slices. It supports preprocessing for all 3D MedMNIST classification datasets. The input root set to the download `npz` files.
|
| 241 |
+
|
| 242 |
+
**7. MRNet**
|
| 243 |
+
- a. [Preprocess/3D_Task/MRNet/cut_slides.py](Preprocess/3D_Task/MRNet/cut_slides.py)
|
| 244 |
+
|
| 245 |
+
**Description**: `cut_slides.py` — This script converts each 3D knee MRI volume into 2D slices from the `axial`, `coronal`, and `sagittal` views, with 10 slices extracted for each view. We map MRNet labels as follows: `abnormal = 0` is labeled as `healthy`, while cases with `abnormal = 1`, `acl = 0`, and `meniscus = 0` are labeled as `other diseases`.
|
| 246 |
+
|
| 247 |
+
**8. MMDental**
|
| 248 |
+
- a. [Preprocess/3D_Task/MMDental/cut_slides.py](Preprocess/3D_Task/MMDental/cut_slides.py)
|
| 249 |
+
|
| 250 |
+
**Description**: `cut_slides.py` — This script converts the original 3D CBCT volumes from each case into 2D slices from the `axial`, `coronal`, and `sagittal` views, with 20 slices extracted for each view. The input root should be set to `./MMDental`.
|
| 251 |
+
|
| 252 |
+
**9. RadGenomme-Chest-CT**
|
| 253 |
+
- a. [Preprocess/3D_Task/RadGenomme-Chest-CT/cut_slides.py](Preprocess/3D_Task/RadGenomme-Chest-CT/cut_slides.py)
|
| 254 |
+
|
| 255 |
+
**Description**: `cut_slides.py` — This script converts the original 3D CT volumes from each case into 2D CT slices. The input root should be set to `./RadGenomme-Chest-CT/dataset`.
|
| 256 |
+
|
| 257 |
+
**10. SynthRAD**
|
| 258 |
+
- a. [Preprocess/3D_Task/SynthRAD/cut_slides.py](Preprocess/3D_Task/SynthRAD/cut_slides.py)
|
| 259 |
+
|
| 260 |
+
**Description**: `cut_slides.py` — This script converts the `ct.nii.gz` and `mr.nii.gz` files from each SynthRAD2023 Task 1 brain and pelvis case into 2D slices. For brain and pelvis data, set the input roots to `./SynthRAD2023/Task1/brain` and `./SynthRAD2023/Task1/pelvis`, respectively.
|
| 261 |
+
|
| 262 |
+
## Image_Classification
|
| 263 |
+
|
| 264 |
+
**1. MNIST**
|
| 265 |
+
- a. [Preprocess/2D_Task/MNIST/unzip_MNIST.py](Preprocess/2D_Task/MNIST/unzip_MNIST.py)
|
| 266 |
+
|
| 267 |
+
**Description**: `unzip_MNIST.py` — This script processes all 2D MedMNIST classification datasets. It extracts the downloaded Zenodo `.npz` files, organizes images according to the official train/validation/test split, and further separates them into class-specific folders. The input root should be set to the path of the downloaded `.npz` files.
|
| 268 |
+
|
| 269 |
+
**2. PanNuke**
|
| 270 |
+
- a. [Preprocess/2D_Task/PanNuke/Preprocess.py](Preprocess/2D_Task/PanNuke/Preprocess.py)
|
| 271 |
+
|
| 272 |
+
**Description**: `Preprocess.py` — This script merges images from the three `.parquet` files and organizes them into class-specific folders. `PanNuke_split.csv` defines the dataset split and should be placed in the downloaded data folder. The input root should be set to the directory containing the three `.parquet` files.
|
| 273 |
+
|
| 274 |
+
## VQA
|
| 275 |
+
|
| 276 |
+
**1. ROCO_QA**
|
| 277 |
+
- a. [Preprocess/2D_Task/ROCO_QA/unzip.py](Preprocess/2D_Task/ROCO_QA/unzip.py)
|
| 278 |
+
|
| 279 |
+
**Description**: `unzip.py` — This script extracts the five `.parquet` files. The input root should be set to `./ROCO-QA/data`, and `FILE_ORDER` should match the order specified in the script.
|
| 280 |
+
|
| 281 |
+
## Vision_Grounding
|
| 282 |
+
|
| 283 |
+
**1. ChestImagenome**
|
| 284 |
+
- a. [Preprocess/2D_Task/ChestImagenome/Preprocess.py](Preprocess/2D_Task/ChestImagenome/Preprocess.py)
|
| 285 |
+
|
| 286 |
+
**Description**: `Preprocess.py` — This script crops the original images according to the official bounding-box annotations. `gold_crop` is used as the benchmark test set, while `silver_sample_crop` is used as the training set. The input root should be set to `./Chest_imagenome`.
|
| 287 |
+
|
| 288 |
+
**2. Gastrointestinal**
|
| 289 |
+
- a. [Preprocess/2D_Task/Gastrointestinal/Preprocess.py](Preprocess/2D_Task/Gastrointestinal/Preprocess.py)
|
| 290 |
+
|
| 291 |
+
**Description**: `Preprocess.py` — This script converts segmentation masks from `PolypGenMSBench` and `Kvasir-SEG` into bounding boxes and cropped images. The split for `Kvasir-SEG` is defined in `split.csv`, which should be placed in the data folder. `PolypGenMSBench` is automatically downloaded and processed. For `Kvasir-SEG`, set the image root to `./Kvasir-SEG/images` and the annotation file to `./Kvasir-SEG/kvasir_bboxes.json`.
|
| 292 |
+
|
| 293 |
+
**3. PanNuke**
|
| 294 |
+
- a. [Preprocess/2D_Task/PanNuke_VG/Preprocess.py](Preprocess/2D_Task/PanNuke_VG/Preprocess.py)
|
| 295 |
+
|
| 296 |
+
**Description**: `Preprocess.py` — This script merges images from the three `.parquet` files and converts segmentation masks into bounding-box crops. The input root should be set to `./PanNuke/data`.
|
| 297 |
+
|
| 298 |
+
**4. Skin_Image**
|
| 299 |
+
- a. [Preprocess/2D_Task/Skin_Image/Preprocess.py](Preprocess/2D_Task/Skin_Image/Preprocess.py)
|
| 300 |
+
|
| 301 |
+
**Description**: `Preprocess.py` — This script converts segmentation masks from `ISIC2018MSBench` and `UWSkinCancerMSBench` into bounding boxes and cropped images. Both datasets are automatically downloaded and processed.
|
| 302 |
+
|
| 303 |
+
**5. Ultrasound**
|
| 304 |
+
- a. [Preprocess/2D_Task/Ultrasound/Preprocess.py](Preprocess/2D_Task/Ultrasound/Preprocess.py)
|
| 305 |
+
|
| 306 |
+
**Description**: `Preprocess.py` — This script converts segmentation masks from `USforKidneyMSBench` and `UltrasoundNerveMSBench` into bounding boxes and cropped images. Both datasets are automatically downloaded and processed.
|
| 307 |
+
|
| 308 |
+
**6. VindrCXR**
|
| 309 |
+
- a. [Preprocess/2D_Task/VindrCXR/Preprocess.py](Preprocess/2D_Task/VindrCXR/Preprocess.py)
|
| 310 |
+
|
| 311 |
+
**Description**: `Preprocess.py` — This script crops the images and follows the official dataset split. The input root should be set to `./VinDr-CXR`.
|
| 312 |
+
|
| 313 |
+
**7. VindrMammo**
|
| 314 |
+
- a. [Preprocess/2D_Task/VindrMammo/Preprocess.py](Preprocess/2D_Task/VindrMammo/Preprocess.py)
|
| 315 |
+
|
| 316 |
+
**Description**: `Preprocess.py` — This script crops the images and follows the official dataset split. The input root should be set to `./VindrMammo`.
|
| 317 |
+
|
| 318 |
+
## Dataset Arrangement
|
| 319 |
+
|
| 320 |
+
After downloading and preprocessing the datasets, please organize the data according to the folder structure below so that it is compatible with the evaluation code.
|
| 321 |
+
|
| 322 |
+
### MIMIC-CXR Data Setup
|
| 323 |
+
|
| 324 |
+
For the MIMIC-CXR-based datasets, you only need to download the original MIMIC-CXR image files once. The same downloaded files can then be reused across multiple sub-datasets by creating symbolic links.
|
| 325 |
+
|
| 326 |
+
The related datasets include:
|
| 327 |
+
|
| 328 |
+
- `MIMIC-CXR-Report`
|
| 329 |
+
- `MIMIC-CXR-T`
|
| 330 |
+
- `MIMIC-CXR-VQA`
|
| 331 |
+
- `ChestImagrome`
|
| 332 |
+
|
| 333 |
+
After downloading MIMIC-CXR, you can do:
|
| 334 |
+
|
| 335 |
+
```
|
| 336 |
+
cd 2D_Images/(ChestImagrome, MIMIC-CXR-Report, MIMIC-CXR-VQA, MIMIC-CXR-T)
|
| 337 |
+
ln -sfn "../mimic_original/2.0.0/files" "files"
|
| 338 |
+
|
| 339 |
+
for d in p10 p11 p12 p13 p14 p15 p16 p17 p18 p19; do
|
| 340 |
+
ln -sfn "../mimic_original/2.0.0/files/$d" "$d"
|
| 341 |
+
done
|
| 342 |
+
```
|
| 343 |
+
|
| 344 |
+
Organize your data under a single root directory (`DATA_BASEDIR`):
|
| 345 |
+
|
| 346 |
+
```
|
| 347 |
+
DATA_BASEDIR/
|
| 348 |
+
│
|
| 349 |
+
├── 2D_Task/
|
| 350 |
+
│ ├── APTOS/
|
| 351 |
+
│ │ ├── APTOS_test.json
|
| 352 |
+
│ │ ├── APTOS_i2i_test.json
|
| 353 |
+
│ │ └── APTOS_t2i_test.json
|
| 354 |
+
│ ├── BloodMNIST/
|
| 355 |
+
│ │ ├── BloodMNIST_test.json
|
| 356 |
+
│ │ ├── BloodMNIST_i2i_test.json
|
| 357 |
+
│ │ └── BloodMNIST_t2i_test.json
|
| 358 |
+
│ ├── Brain-Tumor-MRI/
|
| 359 |
+
│ │ ├── Brain_Tumor_MRI_test.json
|
| 360 |
+
│ │ ├── Brain_Tumor_MRI_i2i_test.json
|
| 361 |
+
│ │ └── Brain_Tumor_MRI_t2i_test.json
|
| 362 |
+
│ ├── ChestImagrome/
|
| 363 |
+
│ │ └── Chest_imagenome_test.json
|
| 364 |
+
│ ├── ChestMNIST/
|
| 365 |
+
│ │ ├── ChestMNIST_test.json
|
| 366 |
+
│ │ ├── ChestMNIST_i2i_test.json
|
| 367 |
+
│ │ └── ChestMNIST_t2i_test.json
|
| 368 |
+
│ ├── ChexpertPlus/
|
| 369 |
+
│ │ ├── chexpert_plus_cls_new.json
|
| 370 |
+
│ │ ├── chexpert_plus_i2t_new.json
|
| 371 |
+
│ │ └── chexpert_plus_t2i_new.json
|
| 372 |
+
│ ├── DermaMNIST/
|
| 373 |
+
│ │ ├── DermaMNIST_test.json
|
| 374 |
+
│ │ ├── DermaMNIST_i2i_test.json
|
| 375 |
+
│ │ └── DermaMNIST_t2i_test.json
|
| 376 |
+
│ ├── Gastrointestinal/
|
| 377 |
+
│ │ └── gastrointestinal_test.json
|
| 378 |
+
│ ├── ISIC-2019/
|
| 379 |
+
│ │ ├── ISIC_2019_test.json
|
| 380 |
+
│ │ ├── ISIC_2019_i2i_test.json
|
| 381 |
+
│ │ └── ISIC_2019_t2i_test.json
|
| 382 |
+
│ ├── Kvasir/
|
| 383 |
+
│ │ ├── Kvasir_test.json
|
| 384 |
+
│ │ ├── Kvasir_i2i_test.json
|
| 385 |
+
│ │ └── Kvasir_t2i_test.json
|
| 386 |
+
│ ├── lc25000/
|
| 387 |
+
│ │ ├── LC25000_cls_new.json
|
| 388 |
+
│ │ ├── LC25000_i2i_new.json
|
| 389 |
+
│ │ └── LC25000_t2i_new.json
|
| 390 |
+
│ ├── MedPIX/
|
| 391 |
+
│ │ └── MedPix_test.json
|
| 392 |
+
│ ├── MIMIC-CXR-Report/
|
| 393 |
+
│ │ └── MIMIC-CXR_test.json
|
| 394 |
+
│ ├── MIMIC-CXR-T/
|
| 395 |
+
│ │ ├── mimic-cxr-lt-test.json
|
| 396 |
+
│ │ ├── mimic-cxr-lt-i2i-test.json
|
| 397 |
+
│ │ └── mimic-cxr-lt-t2i-test.json
|
| 398 |
+
│ ├── MIMIC-CXR-VQA/
|
| 399 |
+
│ │ └── mimic_test.json
|
| 400 |
+
│ ├── OCTMNIST/
|
| 401 |
+
│ │ ├── OCTMNIST_test.json
|
| 402 |
+
│ │ ├── OCTMNIST_i2i_test.json
|
| 403 |
+
│ │ └── OCTMNIST_t2i_test.json
|
| 404 |
+
│ ├── OmniMedVQA_vqa/
|
| 405 |
+
│ │ └── OmniMedVQA_vqa_new.json
|
| 406 |
+
│ ├── OrganAMNIST/
|
| 407 |
+
│ │ ├── OrganAMNIST_test.json
|
| 408 |
+
│ │ ├── OrganAMNIST_i2i_test.json
|
| 409 |
+
│ │ └── OrganAMNIST_t2i_test.json
|
| 410 |
+
│ ├── OrganCMNIST/
|
| 411 |
+
│ │ ├── OrganCMNIST_test.json
|
| 412 |
+
│ │ ├── OrganCMNIST_i2i_test.json
|
| 413 |
+
│ │ └── OrganCMNIST_t2i_test.json
|
| 414 |
+
│ ├── OrganSMNIST/
|
| 415 |
+
│ │ ├── OrganSMNIST_test.json
|
| 416 |
+
│ │ ├── OrganSMNIST_i2i_test.json
|
| 417 |
+
│ │ └── OrganSMNIST_t2i_test.json
|
| 418 |
+
│ ├── PanNuke/
|
| 419 |
+
│ │ ├── pannuke_i2i_test.json
|
| 420 |
+
│ │ └── pannuke_t2i_test.json
|
| 421 |
+
│ ├── PanNuke_VG/
|
| 422 |
+
│ │ └── PanNuke_test.json
|
| 423 |
+
│ ├── PathMNIST/
|
| 424 |
+
│ │ ├── PathMNIST_test.json
|
| 425 |
+
│ │ ├── PathMNIST_i2i_test.json
|
| 426 |
+
│ │ └── PathMNIST_t2i_test.json
|
| 427 |
+
│ ├── PathVQA/
|
| 428 |
+
│ │ └── Path-VQA_test.json
|
| 429 |
+
│ ├── PMC/
|
| 430 |
+
│ │ └── PMC-VQA_test.json
|
| 431 |
+
│ ├── RadImageNet-VQA/
|
| 432 |
+
│ │ └── RadLmageNet_test.json
|
| 433 |
+
│ ├── REtinal/
|
| 434 |
+
│ │ ├── Retinal_i2i_new.json
|
| 435 |
+
│ │ └── Retinal_t2i_new.json
|
| 436 |
+
│ ├── ROCO-VQA/
|
| 437 |
+
│ │ └── ROCO_test.json
|
| 438 |
+
│ ├── SkinLesion/
|
| 439 |
+
│ │ └── skin_test.json
|
| 440 |
+
│ ├── TissueMNIST/
|
| 441 |
+
│ │ ├── TissueMNIST_test.json
|
| 442 |
+
│ │ ├── TissueMNIST_i2i_test.json
|
| 443 |
+
│ │ └── TissueMNIST_t2i_test.json
|
| 444 |
+
│ ├── UltrasoundNerve/
|
| 445 |
+
│ │ └── Ultrasound_test.json
|
| 446 |
+
│ ├── USData/
|
| 447 |
+
│ │ └── USData_test.json
|
| 448 |
+
│ ├── VindrCXR/
|
| 449 |
+
│ │ └── VinDr-CXR_test.json
|
| 450 |
+
│ ├── VindrMammo/
|
| 451 |
+
│ │ └── VindrMammo_test.json
|
| 452 |
+
│ └── VQA-RAD/
|
| 453 |
+
│ └── VQA_test.json
|
| 454 |
+
│
|
| 455 |
+
├── 2D_Images/
|
| 456 |
+
│ ├── APTOS/train_images/
|
| 457 |
+
│ ├── BloodMNIST/output_images/test/
|
| 458 |
+
│ ├── Brain-Tumor-MRI/datasets/Testing/
|
| 459 |
+
│ ├── ChestImagrome/gold_crop/
|
| 460 |
+
│ ├── ChestMNIST/output_images/test/
|
| 461 |
+
│ ├── chexpert_plus/png/
|
| 462 |
+
│ ├── DermaMNIST/output_images/test/
|
| 463 |
+
│ ├── Gastrointestinal/
|
| 464 |
+
│ ├── ISIC-2019/test/ISIC_2019_Test_Input/
|
| 465 |
+
│ ├── kvasir-dataset/datasets/test/
|
| 466 |
+
│ ├── lc25000/lung_colon_image_set/Test_Set/
|
| 467 |
+
│ ├── MedPIX/
|
| 468 |
+
│ ├── MIMIC-CXR-T/
|
| 469 |
+
│ ├── MIMIC-CXR-Report/files/
|
| 470 |
+
│ ├── MIMIC-CXR-VQA/files/
|
| 471 |
+
│ ├── OCTMNIST/output_images/test/
|
| 472 |
+
│ ├── OmniMedVQA_vqa/test/
|
| 473 |
+
│ ├── OrganAMNIST/output_images/test/
|
| 474 |
+
│ ├── OrganCMNIST/output_images/test/
|
| 475 |
+
│ ├── OrganSMNIST/output_images/test/
|
| 476 |
+
│ ├── PanNuke/test_images/
|
| 477 |
+
│ ├── PanNuke_VG/
|
| 478 |
+
│ ├── PathMNIST/output_images/test/
|
| 479 |
+
│ ├── PathVQA/test_images/
|
| 480 |
+
│ ├── PMC/
|
| 481 |
+
│ ├── RadImageNet-VQA/test/images/
|
| 482 |
+
│ ├── REtinal/Test_Set/Test/
|
| 483 |
+
│ ├── ROCO-VQA/
|
| 484 |
+
│ ├── SkinLesion/
|
| 485 |
+
│ ├── TissueMNIST/output_images/test/
|
| 486 |
+
│ ├── UltrasoundNerve/
|
| 487 |
+
│ ├── USData/
|
| 488 |
+
│ ├── VindrCXR/
|
| 489 |
+
│ ├── VindrMammo/
|
| 490 |
+
│ └── VQA-RAD/test/
|
| 491 |
+
│
|
| 492 |
+
├── 3D_Task/
|
| 493 |
+
│ ├── BraTS2023/
|
| 494 |
+
│ │ ├── bratsped_t1_to_t2_test.json
|
| 495 |
+
│ │ └── bratsped_t2_to_t1_test.json
|
| 496 |
+
│ ├── BraTS_MEN/
|
| 497 |
+
│ │ └── BraTS-MEN-i2t_new.json
|
| 498 |
+
│ ├── ChirrMRI600/
|
| 499 |
+
│ │ ├── Chirr_test.json
|
| 500 |
+
│ │ ├── Chirr_i2i_test.json
|
| 501 |
+
│ │ └── Chirr_t2i_test.json
|
| 502 |
+
│ ├── CirrMRI600_Cross/
|
| 503 |
+
│ │ ├── ChirrMRI_test_t1_to_t2.json
|
| 504 |
+
│ │ └── ChirrMRI_test_t2_to_t1.json
|
| 505 |
+
│ ├── CT_RATE/
|
| 506 |
+
│ │ ├── CT_RATE_test.json
|
| 507 |
+
│ │ ├── CT_RATE_vqa_test.json
|
| 508 |
+
│ │ ├── CT_RATE_t2i_test.json
|
| 509 |
+
│ │ └── CT_RATE_i2i_test.json
|
| 510 |
+
│ ├── HaN-Seg/
|
| 511 |
+
│ │ └── hanseg_mri_to_ct_test.json
|
| 512 |
+
│ ├── M3D/
|
| 513 |
+
│ │ ├── m3d_3dqa_test.json
|
| 514 |
+
│ │ ├── m3d_i2t_test.json
|
| 515 |
+
│ │ └── m3d_t2i_test.json
|
| 516 |
+
│ ├── MMDental/
|
| 517 |
+
│ │ └── MMDental_i2t_test.json
|
| 518 |
+
│ ├── MRNet/
|
| 519 |
+
│ │ ├── MRNet_test.json
|
| 520 |
+
│ │ ├── MRNet_i2i_test.json
|
| 521 |
+
│ │ └── MRNet_t2i_test.json
|
| 522 |
+
│ ├── NoduleMNIST/
|
| 523 |
+
│ │ └── Nodule_test.json
|
| 524 |
+
│ ├── Organ3dMNIST/
|
| 525 |
+
│ │ ├── Organ_test.json
|
| 526 |
+
│ │ ├── Organ_i2i_test.json
|
| 527 |
+
│ │ └── Organ_t2i_test.json
|
| 528 |
+
│ ├── RadGen_CT/
|
| 529 |
+
│ │ ├── RGCC_i2t_test.json
|
| 530 |
+
│ │ └── RGCC_test_VQA.json
|
| 531 |
+
│ ├── SynapseMNIST/
|
| 532 |
+
│ │ └── Synapse_test.json
|
| 533 |
+
│ └── SynthRAD/
|
| 534 |
+
│ ├── brain_ct_to_mri_test.json
|
| 535 |
+
│ ├── brain_mri_to_ct_test.json
|
| 536 |
+
│ ├── pelvis_ct_to_mri_test.json
|
| 537 |
+
│ └── pelvis_mri_to_ct_test.json
|
| 538 |
+
│
|
| 539 |
+
├── 3D_Images/
|
| 540 |
+
│ ├── BraTS2023/
|
| 541 |
+
│ ├── BraTS_MEN/BraTS-MEN-Test/
|
| 542 |
+
│ ├── ChirrMRI600/Cirrhosis_T1_slide/
|
| 543 |
+
│ ├── CirrMRI600_Cross/
|
| 544 |
+
│ ├── CT_RATE/
|
| 545 |
+
│ │ ├── valid_fixed_sliced/
|
| 546 |
+
│ │ └── valid_fixed_drr/
|
| 547 |
+
│ ├── HaN-Seg/
|
| 548 |
+
│ ├── M3D/ct_quizze/
|
| 549 |
+
│ ├── MMDental/cbct_png/
|
| 550 |
+
│ ├── MRNet/valid_images/
|
| 551 |
+
│ ├── NoduleMNIST/test_slides/
|
| 552 |
+
│ ├── Organ3dMNIST/test_slides/
|
| 553 |
+
│ ├── RadGen_CT/valid_preprocessed_sliced/
|
| 554 |
+
│ ├── SynapseMNIST/test_slides/
|
| 555 |
+
│ └── SynthRAD/images/
|
| 556 |
+
│
|
| 557 |
+
└── Text_Task/
|
| 558 |
+
├── ChexpertPlus/
|
| 559 |
+
│ └── chexpert_plus_f2i_new.json
|
| 560 |
+
├── MedicalQARetrieval/
|
| 561 |
+
│ └── MedicalQARetrieval_test.json
|
| 562 |
+
├── MedMCQA/
|
| 563 |
+
│ └── medmcqa_test.json
|
| 564 |
+
├── MIMIC_f2i/
|
| 565 |
+
│ └── MIMIC_f2i_test.json
|
| 566 |
+
├── MMDental/
|
| 567 |
+
│ └── MMDental_t2t_test.json
|
| 568 |
+
├── PublicHealthQA/
|
| 569 |
+
│ └── PublicHealthQA_test.json
|
| 570 |
+
└── PubMedQA/
|
| 571 |
+
└── PubMedQA_test.json
|
| 572 |
+
```
|
| 573 |
+
|
| 574 |
+
## Model Weights
|
| 575 |
+
|
| 576 |
+
The model weights for the two MedEmb variants are hosted in separate Hugging Face repositories:
|
| 577 |
+
|
| 578 |
+
- **MedEmb-2B:** [Model weights](https://huggingface.co/MEDHEB/MedEmb-2B)
|
| 579 |
+
- **MedEmb-8B:** [Model weights](https://huggingface.co/MEDHEB/MedEmb-8B)
|
| 580 |
+
|
| 581 |
+
Please refer to the corresponding model repository for usage instructions, model configuration, and licensing information.
|
| 582 |
+
|
| 583 |
+
## License
|
| 584 |
+
|
| 585 |
+
This benchmark aggregates data drawn from many publicly available medical
|
| 586 |
+
imaging and text datasets. **Each sub-dataset retains the license and
|
| 587 |
+
data-use terms of its original source.** Users are responsible for
|
| 588 |
+
complying with the upstream license of any sub-dataset they use.
|