CaiYuanhao/X-Gaussian

Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

ECCV 2024

5 CT scenes · X-ray novel view synthesis · Direct 3D CT reconstruction

Paper · Code · Models · Data

Point Cloud Visualization

Training Process Visualization

GIF demonstrations are from the official X-Gaussian repository. The five scenes available in this release are listed below.

Model Description

X-Gaussian represents X-ray attenuation using radiative 3D Gaussians. Introduced in Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis (ECCV 2024), it supports efficient X-ray novel view synthesis. These checkpoints use the current joint 2D/3D implementation, which also directly reconstructs a CT volume from the same Gaussian field while retaining the original Gaussian point attributes.

This repository provides five scene-specific trained models for Chest, Abdomen, Foot, Head, and Pancreas, paired with the X-Gaussian dataset. Each model represents its corresponding scene; reconstructing another scene requires training from that scene's projections and scanner geometry.

Available Checkpoints

All weights are the final iteration 31,000 checkpoints from the five runs reported in the code repository.

Scene	Gaussian checkpoint	Dataset file	Gaussians
Chest	point_cloud.ply	chest_50.pickle	500,000
Abdomen	point_cloud.ply	abdomen_50.pickle	59,756
Foot	point_cloud.ply	foot_50.pickle	77,682
Head	point_cloud.ply	head_50.pickle	116,013
Pancreas	point_cloud.ply	pancreas_50.pickle	244,511

Each scene folder preserves the layout expected by the project's render.py:

<scene>/
├── config.json
├── metrics.json
└── point_cloud/
    └── iteration_31000/
        └── point_cloud.ply

config.json contains the training settings, decoder configuration, and original voxel geometry. Its data path is relative to the project root (data/<scene>_50.pickle). The PLY includes global decoder and geometry metadata and retains the original 62 per-point columns. It can be used independently for 3D volume export; 2D rendering also needs the corresponding camera geometry. manifest.json records file sizes and SHA-256 checksums.

Training and Evaluation

Each model was trained from 50 X-ray projections at the dataset's original resolution, using training-only FDK initialization, 30,000 base training steps, and 1,000 volume-ray refinement steps. Training jointly uses projection loss, 3D total variation, and volume-ray consistency. Reference CT volumes were used only for evaluation.

Performance

X-ray Novel View Synthesis

Results measured on the hosted joint 2D/3D checkpoints, averaged over 50 held-out views per scene:

Scene	2D PSNR (dB)	2D SSIM
Chest	47.21	0.9937
Abdomen	48.16	0.9968
Foot	44.40	0.9869
Head	51.66	0.9979
Pancreas	47.65	0.9923

Each predicted and reference projection is normalized by its own maximum.

Original paper comparisons

The figures reproduce the original paper results; the table above evaluates the hosted joint 2D/3D checkpoints.

Direct 3D CT Reconstruction

Results measured on the five released models after 30,000 training steps and 1,000 refinement steps, without downsampling:

Scene	CT resolution	3D PSNR (dB)	3D SSIM
Chest	128 × 128 × 128	31.94	0.8895
Abdomen	512 × 512 × 463	33.33	0.8917
Foot	256 × 256 × 256	31.88	0.8795
Head	256 × 256 × 128	40.21	0.9732
Pancreas	512 × 512 × 240	22.22	0.8750

3D metrics use a fixed range of 1 on the original voxel grid, without clipping or intensity correction. SSIM averages nonempty reference slices along each axis, then across axes. CT ground truth is used only for evaluation.

Usage

Please refer to the X-Gaussian repository for environment setup and model usage, including 2D projection rendering and direct 3D CT reconstruction.

Reconstruction Previews

Central orthogonal slices from the hosted models. In each image, the top row shows the reference CT, the middle row the reconstruction, and the bottom row the absolute error. Displayed CT intensities use a fixed range of 0–1 and errors use 0–0.1; metrics above use the unmodified arrays.

Chest

Abdomen

Foot

Head

Pancreas

Citation

If you use these models, please cite X-Gaussian and the SAX-NeRF dataset source:

@inproceedings{x_gaussian,
  title={Radiative gaussian splatting for efficient x-ray novel view synthesis},
  author={Yuanhao Cai and Yixun Liang and Jiahao Wang and Angtian Wang and Yulun Zhang and Xiaokang Yang and Zongwei Zhou and Alan Yuille},
  booktitle={ECCV},
  year={2024}
}

@inproceedings{sax_nerf,
  title={Structure-Aware Sparse-View X-ray 3D Reconstruction},
  author={Yuanhao Cai and Jiahao Wang and Alan Yuille and Zongwei Zhou and Angtian Wang},
  booktitle={CVPR},
  year={2024}
}