README.md

---
license: cc-by-nc-4.0
language:
- en
pipeline_tag: image-to-image
tags:
- image-to-image
- medical-imaging
- ct-reconstruction
- chest-ct
- x-ray
- 3d-reconstruction
---


# PerX2CT — Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays

This repository hosts the **pretrained checkpoints** for **PerX2CT**, a framework that reconstructs a 3D CT volume from two orthogonal (biplanar) 2D X-ray images, using a perspective-projection-based feature sampling scheme.

- **Paper:** Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays (ICASSP 2023, Best Student Paper Award) — https://arxiv.org/abs/2303.05297
- **Code:** https://github.com/dek924/PerX2CT
- **Authors:** Daeun Kyung, Kyungmin Jo, Jaegul Choo, Joonseok Lee, Edward Choi

> ⚠️ **These are weights only.** The `.ckpt` files cannot be used on their own. You must clone the code from the GitHub repository above to load and run them (see [Usage](#usage)).

---

## Checkpoints

| File | Paired config (in the GitHub repo) | Notes |
| --- | --- | --- |
| `PerX2CT.ckpt` | `configs/PerX2CT.yaml` | Base PerX2CT model (full-frame reconstruction). |
| `PerX2CT_global.ckpt` | `configs/PerX2CT_global_w_zoomin.yaml` | PerX2CT-global model, trained with zoom-in; supports zoom-in (sub-region) reconstruction. |

Each checkpoint must be paired with its matching config; using the wrong config will not load correctly.

---

## Usage

These weights require the original codebase.

```bash
# 1. Get the code
git clone https://github.com/dek924/PerX2CT.git
cd PerX2CT

# 2. Environment
conda create -n perx2ct python=3.8
conda activate perx2ct
pip install --upgrade pip
pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 \
  -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirement.txt

# 3. Download the checkpoints from this Hugging Face repo, then run inference.
```

**Full-frame evaluation**

```bash
python main_test.py \
  --ckpt_path /path/to/PerX2CT.ckpt \
  --config_path configs/PerX2CT.yaml \
  --save_dir /path/to/output \
  --val_test test
```

**Zoom-in evaluation** (use the `PerX2CT_global.ckpt` + `configs/PerX2CT_global_w_zoomin.yaml`)

```bash
python main_test_zoom.py \
  --ckpt_path /path/to/PerX2CT_global.ckpt \
  --config_path configs/PerX2CT_global_w_zoomin.yaml \
  --save_dir /path/to/output \
  --zoom_size <target_zoomin_patch_size>
```

See the GitHub README for the full set of options and for training instructions.

---

## Training data

The model was trained on the **LIDC-IDRI** chest CT collection, obtained from
**The Cancer Imaging Archive (TCIA)**. Synthetic biplanar X-rays were generated from the CT volumes
following the projection pipeline introduced in the X2CT-GAN work.

Use of these weights is subject to the data's terms:

- LIDC-IDRI is distributed under the **Creative Commons Attribution 3.0 Unported (CC BY 3.0)** license and the **TCIA Data Usage Policy**. You must give proper attribution (see [Citation](#citation)).
- Per the TCIA terms of use, **you must not attempt to re-identify** the research participants in the underlying data.

---

## Intended use & limitations

- **Intended use:** research and educational purposes only — method reproduction, benchmarking, and academic study.
- **Not for clinical use:** this model is **not** a medical device and must **not** be used for diagnosis, treatment decisions, or any clinical purpose.
- **Domain limits:** trained on LIDC-IDRI chest CT with synthetic X-ray inputs. Performance on real clinical X-rays, other anatomy, other scanners/protocols, or other populations is not guaranteed and is likely to degrade under distribution shift.

---

## License

- **Weights (the `.ckpt` files in this repository):** [Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)](https://creativecommons.org/licenses/by-nc/4.0/). Non-commercial use only; attribution required.
- **Code:** not included here. It lives in the [GitHub repository](https://github.com/dek924/PerX2CT) under that project's terms. Note that the data-preprocessing pipeline there is adapted from X2CT-GAN, which is **GPLv3**, while the model backbone (`taming/`) is **MIT**. The inference code needed to run these weights does not require the GPLv3 preprocessing scripts.
- **Underlying data (LIDC-IDRI):** CC BY 3.0 + TCIA Data Usage Policy, as described above.

---

## Citation

If you use these weights, please cite the PerX2CT paper and the LIDC-IDRI / TCIA sources.

**PerX2CT**

```bibtex
@INPROCEEDINGS{kyung2023perx2ct,
  author={Kyung, Daeun and Jo, Kyungmin and Choo, Jaegul and Lee, Joonseok and Choi, Edward},
  booktitle={ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  title={Perspective Projection-Based 3d CT Reconstruction from Biplanar X-Rays},
  year={2023},
  pages={1-5},
  doi={10.1109/ICASSP49357.2023.10096296}}
```

**LIDC-IDRI (publication)**

```bibtex
@article{armato2011lidc,
  title={The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans},
  author={Armato III, Samuel G. and McLennan, Geoffrey and Bidaut, Luc and others},
  journal={Medical Physics},
  volume={38}, number={2}, pages={915--931}, year={2011},
  doi={10.1118/1.3528204}}
```

**TCIA**

```bibtex
@article{clark2013tcia,
  title={The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository},
  author={Clark, Kenneth and Vendt, Bruce and Smith, Kirk and others},
  journal={Journal of Digital Imaging},
  volume={26}, number={6}, pages={1045--1057}, year={2013},
  doi={10.1007/s10278-013-9622-7}}
```

> The LIDC-IDRI **data citation / DOI** (separate from the publication above) is provided on the current TCIA collection page — please copy it from https://www.cancerimagingarchive.net/collection/lidc-idri/ to ensure it is up to date.

---

## Acknowledgements

This work builds on:

- [CompVis/taming-transformers](https://github.com/CompVis/taming-transformers) (MIT) — VQGAN backbone.
- [kylekma/X2CT](https://github.com/kylekma/X2CT) (GPLv3) — CT-to-X-ray projection method used for data preprocessing.