--- license: cc-by-nc-4.0 language: - en pipeline_tag: image-to-image tags: - image-to-image - medical-imaging - ct-reconstruction - chest-ct - x-ray - 3d-reconstruction --- # PerX2CT — Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays This repository hosts the **pretrained checkpoints** for **PerX2CT**, a framework that reconstructs a 3D CT volume from two orthogonal (biplanar) 2D X-ray images, using a perspective-projection-based feature sampling scheme. - **Paper:** Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays (ICASSP 2023, Best Student Paper Award) — https://arxiv.org/abs/2303.05297 - **Code:** https://github.com/dek924/PerX2CT - **Authors:** Daeun Kyung, Kyungmin Jo, Jaegul Choo, Joonseok Lee, Edward Choi > ⚠️ **These are weights only.** The `.ckpt` files cannot be used on their own. You must clone the code from the GitHub repository above to load and run them (see [Usage](#usage)). --- ## Checkpoints | File | Paired config (in the GitHub repo) | Notes | | --- | --- | --- | | `PerX2CT.ckpt` | `configs/PerX2CT.yaml` | Base PerX2CT model (full-frame reconstruction). | | `PerX2CT_global.ckpt` | `configs/PerX2CT_global_w_zoomin.yaml` | PerX2CT-global model, trained with zoom-in; supports zoom-in (sub-region) reconstruction. | Each checkpoint must be paired with its matching config; using the wrong config will not load correctly. --- ## Usage These weights require the original codebase. ```bash # 1. Get the code git clone https://github.com/dek924/PerX2CT.git cd PerX2CT # 2. Environment conda create -n perx2ct python=3.8 conda activate perx2ct pip install --upgrade pip pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 \ -f https://download.pytorch.org/whl/torch_stable.html pip install -r requirement.txt # 3. Download the checkpoints from this Hugging Face repo, then run inference. ``` **Full-frame evaluation** ```bash python main_test.py \ --ckpt_path /path/to/PerX2CT.ckpt \ --config_path configs/PerX2CT.yaml \ --save_dir /path/to/output \ --val_test test ``` **Zoom-in evaluation** (use the `PerX2CT_global.ckpt` + `configs/PerX2CT_global_w_zoomin.yaml`) ```bash python main_test_zoom.py \ --ckpt_path /path/to/PerX2CT_global.ckpt \ --config_path configs/PerX2CT_global_w_zoomin.yaml \ --save_dir /path/to/output \ --zoom_size ``` See the GitHub README for the full set of options and for training instructions. --- ## Training data The model was trained on the **LIDC-IDRI** chest CT collection, obtained from **The Cancer Imaging Archive (TCIA)**. Synthetic biplanar X-rays were generated from the CT volumes following the projection pipeline introduced in the X2CT-GAN work. Use of these weights is subject to the data's terms: - LIDC-IDRI is distributed under the **Creative Commons Attribution 3.0 Unported (CC BY 3.0)** license and the **TCIA Data Usage Policy**. You must give proper attribution (see [Citation](#citation)). - Per the TCIA terms of use, **you must not attempt to re-identify** the research participants in the underlying data. --- ## Intended use & limitations - **Intended use:** research and educational purposes only — method reproduction, benchmarking, and academic study. - **Not for clinical use:** this model is **not** a medical device and must **not** be used for diagnosis, treatment decisions, or any clinical purpose. - **Domain limits:** trained on LIDC-IDRI chest CT with synthetic X-ray inputs. Performance on real clinical X-rays, other anatomy, other scanners/protocols, or other populations is not guaranteed and is likely to degrade under distribution shift. --- ## License - **Weights (the `.ckpt` files in this repository):** [Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)](https://creativecommons.org/licenses/by-nc/4.0/). Non-commercial use only; attribution required. - **Code:** not included here. It lives in the [GitHub repository](https://github.com/dek924/PerX2CT) under that project's terms. Note that the data-preprocessing pipeline there is adapted from X2CT-GAN, which is **GPLv3**, while the model backbone (`taming/`) is **MIT**. The inference code needed to run these weights does not require the GPLv3 preprocessing scripts. - **Underlying data (LIDC-IDRI):** CC BY 3.0 + TCIA Data Usage Policy, as described above. --- ## Citation If you use these weights, please cite the PerX2CT paper and the LIDC-IDRI / TCIA sources. **PerX2CT** ```bibtex @INPROCEEDINGS{kyung2023perx2ct, author={Kyung, Daeun and Jo, Kyungmin and Choo, Jaegul and Lee, Joonseok and Choi, Edward}, booktitle={ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, title={Perspective Projection-Based 3d CT Reconstruction from Biplanar X-Rays}, year={2023}, pages={1-5}, doi={10.1109/ICASSP49357.2023.10096296}} ``` **LIDC-IDRI (publication)** ```bibtex @article{armato2011lidc, title={The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans}, author={Armato III, Samuel G. and McLennan, Geoffrey and Bidaut, Luc and others}, journal={Medical Physics}, volume={38}, number={2}, pages={915--931}, year={2011}, doi={10.1118/1.3528204}} ``` **TCIA** ```bibtex @article{clark2013tcia, title={The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository}, author={Clark, Kenneth and Vendt, Bruce and Smith, Kirk and others}, journal={Journal of Digital Imaging}, volume={26}, number={6}, pages={1045--1057}, year={2013}, doi={10.1007/s10278-013-9622-7}} ``` > The LIDC-IDRI **data citation / DOI** (separate from the publication above) is provided on the current TCIA collection page — please copy it from https://www.cancerimagingarchive.net/collection/lidc-idri/ to ensure it is up to date. --- ## Acknowledgements This work builds on: - [CompVis/taming-transformers](https://github.com/CompVis/taming-transformers) (MIT) — VQGAN backbone. - [kylekma/X2CT](https://github.com/kylekma/X2CT) (GPLv3) — CT-to-X-ray projection method used for data preprocessing.