README.md

---
license: apache-2.0
library_name: konfai
pipeline_tag: image-to-image
tags:
- medical-imaging
- registration
- deformable
- multimodal
- ct
- mri
- cbct
- konfai
---

# IMPACT-Reg — Multimodal Medical Image Registration

Robust **multimodal** (**MR / CT / CBCT**) deformable **registration** presets, built with
[**KonfAI**](https://github.com/vboussot/KonfAI). Alignment is driven by the **IMPACT** semantic
similarity metric — deep features from pretrained segmentation / foundation models (**MIND**,
**TotalSegmentator**, **anatomix**) — so cross-modality pairs align while the deformation
stays smooth and physically plausible.

Each preset is a **self-contained KonfAI app**: on the **fixed** grid it produces the moving image
resampled onto the fixed image (**`MovedImage`**) and the **`DisplacementField`**. Presets can be
**ensembled** (their displacement fields are averaged into one transform).

## 🧩 Presets

| Preset | Pair | Engine | Description |
|:--|:--|:--|:--|
| `Generic_Rigid` | any | elastix | Rigid alignment (mutual information, multi-resolution) |
| `Generic_Rigid_BSpline` | any | elastix | Rigid, then B-spline deformable refinement |
| `Elastix_IMPACT_Jacobian` | CT/CBCT | elastix + IMPACT | Early `TS/M730` layers, differentiated through the network (Jacobian) |
| `Elastix_IMPACT_Static` | MR/CT | elastix + IMPACT | A deep `TS/M730` layer + MIND, features extracted once (Static) |
| `ConvexAdam_Coarse` | any | itk-impact (native) | Global coarse coupled-convex initialisation on MIND features |
| `ConvexAdam_Composite` | any | itk-impact (native) | The same coarse pass followed by the Adam refinement, in one app |
| `FireANTs_SyN` | any | FireANTs (native) | Rigid + Affine + deformable — SyN or Greedy via `deformable_method` (Riemannian Adam, GPU) |
| `FireANTs_IMPACT` | any | FireANTs + IMPACT | Rigid + Affine + SyN driven by the IMPACT deep-feature metric (multi-model, GPU) |
| `FireANTs_Anatomix` | any | FireANTs + IMPACT | The anatomix pipeline: anatomix + MIND features registered as feature volumes (Static), each reduced to its main components (GPU) |

Inputs: **Fixed**, **Moving**, and optional **FixedMask** / **MovingMask** (restrict the metric region).

## 🚀 Usage

```bash
pip install impact-reg-konfai
# Register a moving image onto a fixed image (ensemble several presets by listing them):
impact-reg-konfai register ConvexAdam_Composite -f fixed.nii.gz -m moving.nii.gz -o ./Output --gpu 0
```

- **Generic runner (single preset):** `konfai-apps infer VBoussot/ImpactReg:ConvexAdam_Composite -i fixed.nii.gz -i moving.nii.gz -o output/`
- **Interactive:** [**SlicerImpactReg**](https://github.com/vboussot/SlicerImpactReg) — a 3D Slicer extension driving these presets.

> The `ConvexAdam_*` presets depend on [`itk-impact`](https://pypi.org/project/itk-impact/); resolving the app installs it automatically (it reuses your existing PyTorch, CPU or GPU).
>
> The `FireANTs_*` presets depend on [`fireants`](https://pypi.org/project/fireants/) (installed at resolve time; **GPU required**). FireANTs is distributed under the **FireANTs License v1.0** — this app calls its public API without copying its source, and ships its license and citation in each preset's `NOTICE`; please cite FireANTs if you use them. `FireANTs_IMPACT` and `FireANTs_Anatomix` additionally drive the deformable stage with the KonfAI **IMPACT** metric, fetching their feature models from `VBoussot/impact-torchscript-models`.

## ⚡ Performance & VRAM

`ConvexAdam` presets (native, GPU) benchmarked on an **NVIDIA RTX PRO 5000 (24 GB)** with a real
abdomen **MR→CT** pair, **222 × 226 × 124 @ 2 mm** (single pass, no TTA):

| Preset | Stages | Time / case | Peak VRAM |
|:--|:--|:--:|:--:|
| `ConvexAdam_Coarse` | linear + coarse | **≈ 4.6 s** | ~2.1 GB |
| `ConvexAdam_Composite` | linear + coarse + fine | **≈ 5.1 s** | ~2.1 GB |
| `FireANTs_Anatomix` | rigid + affine + SyN on feature volumes | **≈ 75 s** | ~1.5 GB |

ConvexAdam per-stage breakdown: linear pre-align **≈ 4.2 s** (ITK affine, MI) · coarse **≈ 0.4 s** · fine **≈ 0.5 s**.
One-time TorchScript feature-model load **≈ 7 s** (amortised across a batch). Times scale with case size;
`--tta k` multiplies runtime. The `elastix + IMPACT` presets run through elastix and scale differently.

## 🧩 Large volumes

Every preset leaves its patch free (`patch_size: [0, 0, 0]`). A pair that fits the GPU runs as one patch, and
the engine receives exactly what it did before. When the GPU runs out of memory, KonfAI cuts the volume into
the fewest equal patches that fit, registers each, and blends the fields over a 20 % overlap with a cosine
window. A patch size can still be forced with `--patch-size`.

The `elastix + IMPACT` presets rarely get there. IMPACT computes its features on the whole image and, when that
does not fit the GPU, retries on smaller feature patches by itself: the run is slower, and the registration
stays one global registration.

Tiling suits a local, deformable stage (the fine ConvexAdam pass, `FireANTs_SyN`). A global stage
(rigid, affine, the coarse ConvexAdam initialisation) is then estimated per patch, which is no longer one
global transform: for those, prefer a GPU the volume fits on, or a coarser input.

## 🔗 Links & Citation

- 🧠 **KonfAI:** [github.com/vboussot/KonfAI](https://github.com/vboussot/KonfAI)
- 📦 **PyPI:** [impact_reg_konfai](https://pypi.org/project/impact_reg_konfai/)
- 🩻 **Slicer:** [SlicerImpactReg](https://github.com/vboussot/SlicerImpactReg)
- 📄 **Paper:** KonfAI — [arXiv:2508.09823](https://arxiv.org/abs/2508.09823)
- 🧬 **anatomix** (used by `FireANTs_Anatomix`): [github.com/neel-dey/anatomix](https://github.com/neel-dey/anatomix) — Dey et al., *AnatomiX: Anatomy-Aware Multimodal Registration*, 2024 (MIT). See [`FireANTs_Anatomix/NOTICE`](FireANTs_Anatomix/NOTICE).
- 🔥 **FireANTs** (used by the `FireANTs_*` presets): [github.com/rohitrango/FireANTs](https://github.com/rohitrango/FireANTs) — Jena et al., *FireANTs: Adaptive Riemannian Optimization for Multi-Scale Diffeomorphic Registration*, Nature Communications, 2024. See [`FireANTs_SyN/NOTICE`](FireANTs_SyN/NOTICE) for the full citation and license.
</content>
</invoke>