README.md

---
license: other
license_name: nvidia-open-model-license-agreement
license_link: >-
  https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
tags:
- medical-imaging
- diffusion
arxiv: 2409.11169
pipeline_tag: unconditional-image-generation
---

# NV-Generate-MR-Brain Overview

## Description:
NV-Generate-MR-Brain is a three-dimensional (3D) latent diffusion model designed to generate high-quality synthetic brain magnetic resonance imaging (MRI) images, achieving the highest resolution and best FID scores among comparable models. This model is specialized for brain MRI with support for multiple modalities: T1, Fluid Attenuated Inversion Recovery (FLAIR), T2, and Susceptibility Weighted Imaging (SWI).

Compared to the previous NV-Generate-MR release, the key differences are:
- **Resolution and image size:** Brain images are typically smaller than full-body MR; max dimension will be 512x512x256, resolution will be 0.45x0.45x0.7mm
- **Supported modalities:** T1, FLAIR, T2, and SWI, selected via integer label input
- **Cross-modality synthesis:** (Coming soon) The primary use case is cross-modality synthesis (e.g., T1 → FLAIR, FLAIR → T1), enabling generation of complementary MRI modalities from existing data

The model excels at data augmentation and at generating realistic medical imaging data to supplement datasets limited by privacy concerns or the rarity of certain conditions. It can also significantly enhance the performance of other medical imaging AI models by generating diverse, realistic training data.

This model is ready for commercial use.

## Github Links:
Training and inference code are in:
[https://github.com/NVIDIA-Medtech/NV-Generate-CTMR/tree/main](https://github.com/NVIDIA-Medtech/NV-Generate-CTMR/tree/main).


### License/Terms of Use:
Use of this model is governed by the [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). Additional Information: [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0.txt).

### Acknowledgment:
This project was conducted by NVIDIA in collaboration with the University of Zurich, Istanbul Medipol University, and Forithmus.

### Deployment Geography:
Global

### Use Case:
Medical researchers, AI developers, and healthcare institutions would be expected to use this model for:
- **Cross-modality synthesis:** Generating complementary brain MRI modalities (e.g., synthesizing FLAIR from T1 or T1 from FLAIR)
- **Synthetic training data generation:** Producing synthetic brain MRI images for data augmentation and AI model training
- **Data augmentation for rare conditions:** Supplementing limited datasets where privacy or rarity restricts data availability

It is not a clinically validated medical device and should not be used for clinical diagnostic purposes.

### Release Date:
Huggingface: 03/16/2026 (GTC San Jose 2026) via https://huggingface.co/nvidia/NV-Generate-MR-brain

## Reference(s):

[1] MAISI V2: https://arxiv.org/abs/2508.05772

[2] Guo, Pengfei, et al. "MAISI: Medical AI for Synthetic Imaging." arXiv preprint arXiv:2409.11169. 2024. https://arxiv.org/abs/2409.11169

[3] Rombach, Robin, et al. "High-resolution image synthesis with latent diffusion models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022. https://openaccess.thecvf.com/content/CVPR2022/papers/Rombach_High-Resolution_Image_Synthesis_With_Latent_Diffusion_Models_CVPR_2022_paper.pdf

[4] Lvmin Zhang, Anyi Rao, Maneesh Agrawala; "Adding Conditional Control to Text-to-Image Diffusion Models." Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 3836-3847. https://openaccess.thecvf.com/content/ICCV2023/papers/Zhang_Adding_Conditional_Control_to_Text-to-Image_Diffusion_Models_ICCV_2023_paper.pdf

## Model Architecture:
**Architecture Type:** Diffusion Model<br>
**Network Architecture:** 3D UNet + attention blocks (latent diffusion)<br>
**Task:** Generation (Synthetic MRI Image)<br>
**Number of model parameters:** 240M

## Computational Load (Internal Only: For NVIDIA Models Only)
**Cumulative Compute:** 4.375 x 10^22 <br>
**Estimated Energy and Emissions for Model Training:** 24,085kWh <br>

## Input:
**Input Type(s):** Integer, Array<br>
**Input Format(s):** Integer values, Float arrays<br>
**Input Parameters:** Number of Samples (1D), Modality (1D), Output Size (1D), and Spacing (1D)<br>
**Other Properties Related to Input:** Supports controllable synthetic brain MRI generation with modality selection, customizable output dimensions, and configurable voxel spacing.

### num_output_samples
- **Type:** Integer
- **Description:** Required input indicates the number of synthetic brain MRI images the model will generate

### modality
- **Type:** Integer label
- **Description:** Required input specifying the MRI modality to generate
- **Options:**
  - T1
  - FLAIR
  - T2
  - SWI

### output_size
- **Type:** Array of 3 Integers
- **Description:** Optional specification of x, y, and z dimensions of the brain MRI image
- **Constraints:** Max dimension is 512x512x256

### spacing
- **Type:** Array of 3 Floats
- **Description:** Optional voxel spacing specification
- **Range:** 0.45x0.45x0.7mm

## Output:
**Output Type(s):** Image<br>
**Output Format:** Neuroimaging Informatics Technology Initiative (NIfTI)<br>
**Output Parameters:** Three-Dimensional (3D)<br>
**Other Properties Related to Output:** Synthetic brain MRI images in the specified modality (T1, FLAIR, T2, or SWI). Output dimensions and spacing are configurable within supported ranges.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

## Software Integration:
**Runtime Engine(s):**
* MONAI Core v.1.5

**Supported Hardware Microarchitecture Compatibility:**
* NVIDIA Ampere
* NVIDIA Hopper
* NVIDIA Blackwell

**Supported Operating System(s):**
* Linux

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

## Model Version(s):
0.1 - Initial release version for synthetic brain MRI image generation

## Training, Testing, and Evaluation Datasets:

### Dataset Overview:
The model was trained on the MR-Rate brain MRI dataset covering the supported modalities (T1, FLAIR, T2, and SWI). Data from multiple scanner types were processed to create high-quality 3D MRI volumes with corresponding anatomical annotations. The data processing pipeline ensured consistent voxel spacing, standardized orientations, and validated anatomical segmentations.

## Training Dataset:
**Data Modality:**
* Image (Brain MRI — T1, FLAIR, T2, and SWI)

**Image Training Data Size:**
* Less than a Million Images

**Data Collection Method by dataset:**
* Hybrid: Human, Automatic/Sensors

**Labeling Method by dataset:**
* Hybrid: Human, Automatic/Sensors

**Properties:** Approximately 28,000 MRI scans of various types (T1, FLAIR, T2, and SWI).

## Testing Dataset:
**Data Modality:**
* Image (Brain MRI — T1, FLAIR, T2, and SWI)

**Image Training Data Size:**
* Less than a Million Images

**Data Collection Method by dataset:**
* Hybrid: Human, Automatic/Sensors

**Labeling Method by dataset:**
* Hybrid: Human, Automatic/Sensors

**Properties:** Approximately 8,000 MRI scans of various types (T1, FLAIR, T2, and SWI).

## Evaluation Dataset:
**Data Modality:**
* Image (Brain MRI — T1, FLAIR, T2, and SWI)

**Image Training Data Size:**
* Less than a Million Images

**Data Collection Method by dataset:**
* Hybrid: Human, Automatic/Sensors

**Labeling Method by dataset:**
* Hybrid: Human, Automatic/Sensors

**Properties:** Approximately 4,000 MRI scans of various types (T1, FLAIR, T2, and SWI).

# Inference:
**Acceleration Engine:** PyTorch <br>
**Test Hardware:** <br>
* A100
* H100

## Ethical Considerations:
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. 

For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards.

Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.

Please report model quality, risk, security vulnerabilities or concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).