VBoussot/ImpactSynth

TotalSynth: whole-body synthetic CT from MRI and CBCT

Pretrained synthetic CT (sCT) models for MRI and CBCT inputs, from the head to the pelvis, built with KonfAI. This repository holds the TotalSynth models described in TotalSynth: Robust Whole-Body Synthetic CT from MRI and CBCT. They run from the command line, from Python, and inside 3D Slicer through the ImpactSynth and KonfAI extensions, with evaluation and uncertainty estimation included.

🧩 Models

App	Input	Output	Training pairs	Ensemble
MR	MRI	sCT	805 MRI/CT pairs: SynthRAD2023 (brain, pelvis), SynthRAD2025 (head and neck, thorax, abdomen), four prostate cohorts	5
CBCT	CBCT	sCT	929 CBCT/CT pairs: SynthRAD2023 (brain, pelvis), SynthRAD2025 (head and neck, thorax, abdomen)	5
MR_CBCT	MRI or CBCT	sCT	Both sets together, 1734 pairs: one model for both modalities	5
Finetune	MRI (whole-body Dixon)	sCT	The MR model fine-tuned on 45 BIC-MAC whole-body MRI/CT cases	5

All four apps share one network: a 2.5D U-Net++ with a ResNet-34 encoder (26.1 M parameters). Five adjacent axial slices go in, the CT of the central slice comes out. Each app ships five cross-validation checkpoints (CV_0.pt to CV_4.pt) that can be ensembled at inference.

Training pairs come from the public SynthRAD challenges. The planning CT was registered to each MRI or CBCT with IMPACT-Reg, every deformation was reviewed, and 1450 of the 1800 public pairs were kept. The deformation fields are published, so the supervision can be inspected: synthrad2023-impact-registration and synthrad2025-impact-registration. The split files (CrossValidation_0..4.txt, Validation.txt) and the per-fold metrics are in MR_MODEL_INFO.md and CBCT_MODEL_INFO.md.

📊 Performance

Held-out cases, metrics computed inside the body mask against the registered CT (mean ± SD).

Model	Evaluation domain	MAE (HU)	SSIM	PSNR (dB)
MR	MRI/CT, 140 cases	67.49 ± 15.97	0.920 ± 0.025	29.28 ± 2.10
CBCT	CBCT/CT, 150 cases	53.55 ± 17.79	0.939 ± 0.026	32.09 ± 2.93
MR_CBCT	MRI inputs	67.68 ± 16.42	0.920 ± 0.025	29.22 ± 2.18
MR_CBCT	CBCT inputs	54.22 ± 17.54	0.938 ± 0.025	31.89 ± 2.87
MR, direct	BIC-MAC, external, 15 subjects	100.91 ± 18.67	0.968 ± 0.007	26.69 ± 1.40
Finetune	BIC-MAC, external, 15 subjects	62.21 ± 6.19	0.982 ± 0.002	30.45 ± 0.92

MAE by region, in HU:

Model	Brain	Pelvis	Head and neck	Thorax	Abdomen	CSIRO (MRI)
MR	79.7	48.2	80.9	58.5	60.1	76.0
CBCT	39.4	34.8	67.5	57.3	57.2

The evaluation workflow of each app also reports the Dice of TotalSegmentator structures segmented on the sCT and on the CT, a perceptual distance in SAM 2.1 feature space, an IMPACT-based registration term between the sCT and its own input, and the ensemble uncertainty. The BIC-MAC rows show what to expect under domain shift: whole-body Dixon MRI is far from the radiotherapy training data, and fine-tuning on 45 local cases brings the error back to the in-distribution range. Validate on your own data before any clinical use, and fine-tune when the images differ from the training domain.

🚀 Usage

3D Slicer. Install the ImpactSynth extension (or KonfAI for the generic interface) from the Extensions Manager. Load an MRI or a CBCT, choose Synthesis: MR, Synthesis: CBCT or Synthesis: MR/CBCT, pick the checkpoints to ensemble, and click Run. The sCT is loaded over the input. The Evaluation section compares it with a reference CT (MAE map, PSNR, SSIM, Dice) or estimates uncertainty from the inference stack without any reference. Walkthrough: SlicerImpactSynth.

Command line.

pip install impact_synth_konfai
impact-synth-konfai synthesize MR -i input_mr.nii.gz -o output/ --gpu 0 --ensemble 5 --tta 2
impact-synth-konfai eval MR -i input_mr.nii.gz --gt reference_ct.nii.gz -o output/
impact-synth-konfai pipeline CBCT -i cbct.mha --gt ct.mha -o output/ --gpu 0 --ensemble 5 --tta 2 -uncertainty

Generic runner. konfai-apps infer VBoussot/ImpactSynth:MR -i input_mr.nii.gz -o output/ --gpu 0 --ensemble 5 --tta 2. Add --host and --port to run on a konfai-apps-server.

Fine-tune on your own pairs. Each app carries its training configuration (Config.yml), so the released weights can start a local adaptation:

konfai-apps fine-tune VBoussot/ImpactSynth:MR MyCenter_MR -d /data/pairs --models CV_0 --epochs 50 --gpu 0

The dataset directory follows the KonfAI layout of the app (one folder per case with the input and the registered CT). The result is a new local app that Slicer and the CLI run like any other.

⚙️ What an app does

1. A body mask is predicted with ImpactSeg and dilated by five voxels.
2. The input is resampled to 1 × 1 × 3 mm, clipped between its minimum and its 99.5th percentile, and scaled to [-1, 1].
3. Every selected checkpoint synthesises 512 × 512 axial patches, with flipped test-time augmentation (--tta 2 by default).
4. The predictions are averaged, scaled back to HU, masked (air outside the body) and written in the input geometry. With -uncertainty, every sampled prediction is kept in an InferenceStack for the uncertainty workflow.

Evaluation.yml compares an sCT with a reference CT: MAE with a voxel map, PSNR, SSIM, SAM perceptual distance, and the Dice of TotalSegmentator (total-3mm) segmentations of both images. Uncertainty.yml turns an inference stack into a voxel-wise variance map, a segmentation disagreement map (conformity) and their means.

⚡ Performance & VRAM

Benchmarked on a single NVIDIA RTX PRO 5000 (24 GB) with a real whole-body MR (295 × 259 × 219, 2 mm). The batch size is measured on your GPU (a forward of one patch, then of two, then the batch the free VRAM holds).

Free VRAM	Batch (auto)	Peak VRAM	Time / case
8 GB	16	~7.6 GB	—
16 GB	28	~15 GB	—
24 GB	32	~16 GB	~24 s

sCT generation keeps system RAM ~2 GB. A full 5-model ensemble runs in ~82 s on the same card. Override with --patch-size / --batch-size.

🔗 Links & Citation

• 🧠 KonfAI: github.com/fideus-labs/KonfAI
• 🖥️ 3D Slicer: SlicerImpactSynth, SlicerKonfAI
• 📦 PyPI: impact_synth_konfai
• 📄 Paper: arXiv:2609.13838

@article{boussot2026totalsynth,
  title   = {TotalSynth: Robust Whole-Body Synthetic CT from MRI and CBCT},
  author  = {Boussot, Valentin and H{\'e}mon, C{\'e}dric and Barateau, Ana{\"i}s and Lafond, Caroline and Nunes, Jean-Claude and Dillenseger, Jean-Louis},
  journal = {arXiv preprint arXiv:2609.13838},
  year    = {2026}
}

Training methodology: Boussot V. et al., Why Registration Quality Matters: Enhancing sCT Synthesis with IMPACT-Based Registration, arXiv:2510.21358. Framework: Boussot V., Dillenseger J.-L., KonfAI: A Modular and Fully Configurable Framework for Deep Learning in Medical Imaging, arXiv:2508.09823.

The models are released for research. Every clinical use (dose calculation, contour propagation, attenuation correction, adaptive decisions) needs its own acceptance criteria.