Commit ·
36a42ec
0
Parent(s):
Duplicate from dek924/PerX2CT
Browse files- .gitattributes +35 -0
- PerX2CT.ckpt +3 -0
- PerX2CT_global.ckpt +3 -0
- README.md +159 -0
.gitattributes
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
PerX2CT.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e659cdcaf44ade7f75d71992d44d2b297a16ae837bae55bde7fe38b06d1f2fff
|
| 3 |
+
size 603049844
|
PerX2CT_global.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:661293ff250736e7b7c4aed70332eaf57a4a0dc252b5602dc97de6ca0ea75cb5
|
| 3 |
+
size 934769316
|
README.md
ADDED
|
@@ -0,0 +1,159 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
pipeline_tag: image-to-image
|
| 6 |
+
tags:
|
| 7 |
+
- image-to-image
|
| 8 |
+
- medical-imaging
|
| 9 |
+
- ct-reconstruction
|
| 10 |
+
- chest-ct
|
| 11 |
+
- x-ray
|
| 12 |
+
- 3d-reconstruction
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
# PerX2CT — Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays
|
| 17 |
+
|
| 18 |
+
This repository hosts the **pretrained checkpoints** for **PerX2CT**, a framework that reconstructs a 3D CT volume from two orthogonal (biplanar) 2D X-ray images, using a perspective-projection-based feature sampling scheme.
|
| 19 |
+
|
| 20 |
+
- **Paper:** Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays (ICASSP 2023, Best Student Paper Award) — https://arxiv.org/abs/2303.05297
|
| 21 |
+
- **Code:** https://github.com/dek924/PerX2CT
|
| 22 |
+
- **Authors:** Daeun Kyung, Kyungmin Jo, Jaegul Choo, Joonseok Lee, Edward Choi
|
| 23 |
+
|
| 24 |
+
> ⚠️ **These are weights only.** The `.ckpt` files cannot be used on their own. You must clone the code from the GitHub repository above to load and run them (see [Usage](#usage)).
|
| 25 |
+
|
| 26 |
+
---
|
| 27 |
+
|
| 28 |
+
## Checkpoints
|
| 29 |
+
|
| 30 |
+
| File | Paired config (in the GitHub repo) | Notes |
|
| 31 |
+
| --- | --- | --- |
|
| 32 |
+
| `PerX2CT.ckpt` | `configs/PerX2CT.yaml` | Base PerX2CT model (full-frame reconstruction). |
|
| 33 |
+
| `PerX2CT_global.ckpt` | `configs/PerX2CT_global_w_zoomin.yaml` | PerX2CT-global model, trained with zoom-in; supports zoom-in (sub-region) reconstruction. |
|
| 34 |
+
|
| 35 |
+
Each checkpoint must be paired with its matching config; using the wrong config will not load correctly.
|
| 36 |
+
|
| 37 |
+
---
|
| 38 |
+
|
| 39 |
+
## Usage
|
| 40 |
+
|
| 41 |
+
These weights require the original codebase.
|
| 42 |
+
|
| 43 |
+
```bash
|
| 44 |
+
# 1. Get the code
|
| 45 |
+
git clone https://github.com/dek924/PerX2CT.git
|
| 46 |
+
cd PerX2CT
|
| 47 |
+
|
| 48 |
+
# 2. Environment
|
| 49 |
+
conda create -n perx2ct python=3.8
|
| 50 |
+
conda activate perx2ct
|
| 51 |
+
pip install --upgrade pip
|
| 52 |
+
pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 \
|
| 53 |
+
-f https://download.pytorch.org/whl/torch_stable.html
|
| 54 |
+
pip install -r requirement.txt
|
| 55 |
+
|
| 56 |
+
# 3. Download the checkpoints from this Hugging Face repo, then run inference.
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
**Full-frame evaluation**
|
| 60 |
+
|
| 61 |
+
```bash
|
| 62 |
+
python main_test.py \
|
| 63 |
+
--ckpt_path /path/to/PerX2CT.ckpt \
|
| 64 |
+
--config_path configs/PerX2CT.yaml \
|
| 65 |
+
--save_dir /path/to/output \
|
| 66 |
+
--val_test test
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
**Zoom-in evaluation** (use the `PerX2CT_global.ckpt` + `configs/PerX2CT_global_w_zoomin.yaml`)
|
| 70 |
+
|
| 71 |
+
```bash
|
| 72 |
+
python main_test_zoom.py \
|
| 73 |
+
--ckpt_path /path/to/PerX2CT_global.ckpt \
|
| 74 |
+
--config_path configs/PerX2CT_global_w_zoomin.yaml \
|
| 75 |
+
--save_dir /path/to/output \
|
| 76 |
+
--zoom_size <target_zoomin_patch_size>
|
| 77 |
+
```
|
| 78 |
+
|
| 79 |
+
See the GitHub README for the full set of options and for training instructions.
|
| 80 |
+
|
| 81 |
+
---
|
| 82 |
+
|
| 83 |
+
## Training data
|
| 84 |
+
|
| 85 |
+
The model was trained on the **LIDC-IDRI** chest CT collection, obtained from
|
| 86 |
+
**The Cancer Imaging Archive (TCIA)**. Synthetic biplanar X-rays were generated from the CT volumes
|
| 87 |
+
following the projection pipeline introduced in the X2CT-GAN work.
|
| 88 |
+
|
| 89 |
+
Use of these weights is subject to the data's terms:
|
| 90 |
+
|
| 91 |
+
- LIDC-IDRI is distributed under the **Creative Commons Attribution 3.0 Unported (CC BY 3.0)** license and the **TCIA Data Usage Policy**. You must give proper attribution (see [Citation](#citation)).
|
| 92 |
+
- Per the TCIA terms of use, **you must not attempt to re-identify** the research participants in the underlying data.
|
| 93 |
+
|
| 94 |
+
---
|
| 95 |
+
|
| 96 |
+
## Intended use & limitations
|
| 97 |
+
|
| 98 |
+
- **Intended use:** research and educational purposes only — method reproduction, benchmarking, and academic study.
|
| 99 |
+
- **Not for clinical use:** this model is **not** a medical device and must **not** be used for diagnosis, treatment decisions, or any clinical purpose.
|
| 100 |
+
- **Domain limits:** trained on LIDC-IDRI chest CT with synthetic X-ray inputs. Performance on real clinical X-rays, other anatomy, other scanners/protocols, or other populations is not guaranteed and is likely to degrade under distribution shift.
|
| 101 |
+
|
| 102 |
+
---
|
| 103 |
+
|
| 104 |
+
## License
|
| 105 |
+
|
| 106 |
+
- **Weights (the `.ckpt` files in this repository):** [Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)](https://creativecommons.org/licenses/by-nc/4.0/). Non-commercial use only; attribution required.
|
| 107 |
+
- **Code:** not included here. It lives in the [GitHub repository](https://github.com/dek924/PerX2CT) under that project's terms. Note that the data-preprocessing pipeline there is adapted from X2CT-GAN, which is **GPLv3**, while the model backbone (`taming/`) is **MIT**. The inference code needed to run these weights does not require the GPLv3 preprocessing scripts.
|
| 108 |
+
- **Underlying data (LIDC-IDRI):** CC BY 3.0 + TCIA Data Usage Policy, as described above.
|
| 109 |
+
|
| 110 |
+
---
|
| 111 |
+
|
| 112 |
+
## Citation
|
| 113 |
+
|
| 114 |
+
If you use these weights, please cite the PerX2CT paper and the LIDC-IDRI / TCIA sources.
|
| 115 |
+
|
| 116 |
+
**PerX2CT**
|
| 117 |
+
|
| 118 |
+
```bibtex
|
| 119 |
+
@INPROCEEDINGS{kyung2023perx2ct,
|
| 120 |
+
author={Kyung, Daeun and Jo, Kyungmin and Choo, Jaegul and Lee, Joonseok and Choi, Edward},
|
| 121 |
+
booktitle={ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
|
| 122 |
+
title={Perspective Projection-Based 3d CT Reconstruction from Biplanar X-Rays},
|
| 123 |
+
year={2023},
|
| 124 |
+
pages={1-5},
|
| 125 |
+
doi={10.1109/ICASSP49357.2023.10096296}}
|
| 126 |
+
```
|
| 127 |
+
|
| 128 |
+
**LIDC-IDRI (publication)**
|
| 129 |
+
|
| 130 |
+
```bibtex
|
| 131 |
+
@article{armato2011lidc,
|
| 132 |
+
title={The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans},
|
| 133 |
+
author={Armato III, Samuel G. and McLennan, Geoffrey and Bidaut, Luc and others},
|
| 134 |
+
journal={Medical Physics},
|
| 135 |
+
volume={38}, number={2}, pages={915--931}, year={2011},
|
| 136 |
+
doi={10.1118/1.3528204}}
|
| 137 |
+
```
|
| 138 |
+
|
| 139 |
+
**TCIA**
|
| 140 |
+
|
| 141 |
+
```bibtex
|
| 142 |
+
@article{clark2013tcia,
|
| 143 |
+
title={The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository},
|
| 144 |
+
author={Clark, Kenneth and Vendt, Bruce and Smith, Kirk and others},
|
| 145 |
+
journal={Journal of Digital Imaging},
|
| 146 |
+
volume={26}, number={6}, pages={1045--1057}, year={2013},
|
| 147 |
+
doi={10.1007/s10278-013-9622-7}}
|
| 148 |
+
```
|
| 149 |
+
|
| 150 |
+
> The LIDC-IDRI **data citation / DOI** (separate from the publication above) is provided on the current TCIA collection page — please copy it from https://www.cancerimagingarchive.net/collection/lidc-idri/ to ensure it is up to date.
|
| 151 |
+
|
| 152 |
+
---
|
| 153 |
+
|
| 154 |
+
## Acknowledgements
|
| 155 |
+
|
| 156 |
+
This work builds on:
|
| 157 |
+
|
| 158 |
+
- [CompVis/taming-transformers](https://github.com/CompVis/taming-transformers) (MIT) — VQGAN backbone.
|
| 159 |
+
- [kylekma/X2CT](https://github.com/kylekma/X2CT) (GPLv3) — CT-to-X-ray projection method used for data preprocessing.
|