Update README.md

d75ca7d verified about 20 hours ago

7.03 kB

---
license: cc-by-nc-nd-4.0
pipeline_tag: image-feature-extraction
tags:
- feature-extraction
- MRI
- Neuroimaging
- generative-models
- foundation-models
- vision-transformers
- disease-classification
- brain-age-prediction
extra_gated_prompt: >-
  By requesting access you agree to the license terms and will use
  this model for non-commercial research only.
extra_gated_fields:
  Full name: text
  Affiliation: text
  Country: country
  Date of birth: date_picker
  Name of Principal Investigator (PI): text
  Intended use:
    type: select
    options:
      - Research
      - Education
      - Other
  Short description of intended research use: text
  I agree to use this model for non-commercial purposes only: checkbox
---
# Brain2Vec: A Self-Supervised 3D Foundation Model for Structural Brain MRI

This is the hugging face model card for Brain2Vec, a 3D foundation model pre-trained on 74,425 T1-weighted brain MRI scans from the iSTAGING consortium.  
If you are looking for the preprint, it is located here: [**Brain2Vec-preprint**](https://arxiv.org/abs/2610.28411).  
If you are looking for the github repo, it is located here: [**Brain2Vec-repo**](https://github.com/CBICA/Brain2Vec).  

## Gaining access

We suggest to either apply for access on huggingface and use our model directly (i.e. for fine-tuning, ablation studies, etc.) or use our model on [**NiChart**](https://neuroimgagingchart.com) for simpler tasks.
We want our model to help the neuroimaging community (researchers, students, etc.), in order to gain access, please mention your university/company, PI and the purpose of usage.

Once you gain access, you can use our model using huggingface hub:

```python
from Brain2Vec.models.brain2vec import brain2vec_base_patch8_dec512d8b
from Brain2Vec.models.vit import vit_base_patch8
from Brain2Vec.models.utils import load_brain2vec
# for Brain2Vec
model = load_brain2vec(
    repo_id="cbica/Brain2Vec",
    weights_filename="brain2vec.safetensors",
    model_class=brain2vec_base_patch8_dec512d8b,
    model_kwargs={"img_size": 128},
    device="cuda",
)
model.eval()
# you can review inference.py on github to see how to perform inference correctly. We
# use the CLS token for feature extraction, unlike other foundation models that use pooling.
# for ViT-BA
model = load_brain2vec(
    repo_id="cbica/Brain2Vec",
    weights_filename="vit_ba_finetuned.safetensors",
    model_class=vit_base_patch8,
    model_kwargs={"img_size": 128, "num_classes": 1},
    device="cuda",
)
model.eval()
```

You can install Brain2Vec using pip from our GitHub codebase:
```bash
git clone https://github.com/CBICA/Brain2Vec
cd Brain2Vec/
pip install -e .
```

## Feature extraction

One other way of using Brain2Vec is for feature extraction. We can use the Brain2Vec frozen encoder to produce 768-dimensional embeddings from T1 scans.
This compact numerical representation serves as input for lightweight downstream classifiers, such as SVM or MLP models.


```python
python inference.py brain2vec_inference --model brain2vec_base \
                                    --img_size 128 \
                                    --model_weights "model weights (.pth)" \
                                    --in_dir "path to directory with niftii scans" \
                                    --out_json "name of json file with output embeddings" \
                                    --device cuda
python inference.py vit_inference --model vit_base \
                                --num_classes 1 \
                                --img_size 128 \
                                --mode regression \
                                --model_weights "model weights (.pth)" \
                                --in_dir "input directory with niftii scans" \
                                --out_csv "output csv file with predictions" \
                                --device cuda
```

## Fine-tuning
One way of using Brain2Vec is for encoder fine-tuning. The pre-trained encoder weights can serve as initialization for Vision Transformer (ViT) models applied to downstream tasks like disease classification or brain age regression.

The only thing you need is a csv with MRID and target columns.
We provide all the code needed to pre-train/fine-tune on SLURM environments on [github](https://github.com/CBICA/Brain2Vec).

```python
srun Brain2Vec vit_finetune --img_size 128 \
                         --mode binary \
                         --model vit_base \
                         --label_dict "csv with MRID and target columns" \
                         --target "target column" \
                         --pretrain "pre-trained model weights (.pth)" \
                         --in_dir "input_dir/"
                         --batch_size 32 \
                         --out_dir "output_dir/" \
                         --model_name "model name (.pth)" \
                         --device cuda \
                         --lr 0.001 \
                         --weight_decay 0.05 \
                         --optimizer AdamW \
                         --epochs 200 \
                         --patience 20 \
                         --distributed \
                         --verbose
```


## Pre-training

We trained with two A100 GPUs for 500 epochs with a patience of 20 epochs. For more training details please read the Brain2Vec paper. Using our code,
you can parallelize your training on SLURM cluster environments. Other environments are supported but not tested, so bugs may appear.

```python
srun Brain2Vec brain2vec_pretrain --img_size 128 \
                         --masking_type random \
                         --mask_ratio 0.75 \
                         --model brain2vec_base \
                         --in_dir "input_dir/"
                         --batch_size 32 \
                         --out_dir "output_dir/" \
                         --model_name "model name(.pth)" \
                         --device cuda \
                         --lr 0.001 \
                         --weight_decay 0.05 \
                         --optimizer AdamW \
                         --epochs 500 \
                         --patience 20 \
                         --distributed \
                         --verbose \
```

## Usage on NiChart

You can use Brain2Vec -- for both feature extraction and disease classification/brain age regression -- on [**NiChart**](https://neuroimagingchart.com) by simply dragging and dropping your NIfTI files.
No GPU or installation required.


## Citation

If you use our model, make sure to cite our work:
```
@misc{maggioros2026brain2vecselfsupervised3dfoundation,
      title={Brain2Vec: A Self-Supervised 3D Foundation Model for Structural Brain MRI}, 
      author={Spiros Maggioros and Guray Erus and Gareth Harman and George Aidinis and Alexander Getka and Aristeidis Sotiras and Christos Davatzikos},
      year={2026},
      eprint={2610.28411},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.28411}, 
}
```

	---
	license: cc-by-nc-nd-4.0
	pipeline_tag: image-feature-extraction
	tags:
	- feature-extraction
	- MRI
	- Neuroimaging
	- generative-models
	- foundation-models
	- vision-transformers
	- disease-classification
	- brain-age-prediction
	extra_gated_prompt: >-
	By requesting access you agree to the license terms and will use
	this model for non-commercial research only.
	extra_gated_fields:
	Full name: text
	Affiliation: text
	Country: country
	Date of birth: date_picker
	Name of Principal Investigator (PI): text
	Intended use:
	type: select
	options:
	- Research
	- Education
	- Other
	Short description of intended research use: text
	I agree to use this model for non-commercial purposes only: checkbox
	---

	# Brain2Vec: A Self-Supervised 3D Foundation Model for Structural Brain MRI

	This is the hugging face model card for Brain2Vec, a 3D foundation model pre-trained on 74,425 T1-weighted brain MRI scans from the iSTAGING consortium.
	If you are looking for the preprint, it is located here: [Brain2Vec-preprint](https://arxiv.org/abs/2610.28411).
	If you are looking for the github repo, it is located here: [Brain2Vec-repo](https://github.com/CBICA/Brain2Vec).

	## Gaining access

	We suggest to either apply for access on huggingface and use our model directly (i.e. for fine-tuning, ablation studies, etc.) or use our model on [NiChart](https://neuroimgagingchart.com) for simpler tasks.
	We want our model to help the neuroimaging community (researchers, students, etc.), in order to gain access, please mention your university/company, PI and the purpose of usage.

	Once you gain access, you can use our model using huggingface hub:

	```python
	from Brain2Vec.models.brain2vec import brain2vec_base_patch8_dec512d8b
	from Brain2Vec.models.vit import vit_base_patch8
	from Brain2Vec.models.utils import load_brain2vec

	# for Brain2Vec
	model = load_brain2vec(
	repo_id="cbica/Brain2Vec",
	weights_filename="brain2vec.safetensors",
	model_class=brain2vec_base_patch8_dec512d8b,
	model_kwargs={"img_size": 128},
	device="cuda",
	)
	model.eval()

	# you can review inference.py on github to see how to perform inference correctly. We
	# use the CLS token for feature extraction, unlike other foundation models that use pooling.

	# for ViT-BA
	model = load_brain2vec(
	repo_id="cbica/Brain2Vec",
	weights_filename="vit_ba_finetuned.safetensors",
	model_class=vit_base_patch8,
	model_kwargs={"img_size": 128, "num_classes": 1},
	device="cuda",
	)
	model.eval()
	```

	You can install Brain2Vec using pip from our GitHub codebase:
	```bash
	git clone https://github.com/CBICA/Brain2Vec
	cd Brain2Vec/
	pip install -e .
	```

	## Feature extraction

	One other way of using Brain2Vec is for feature extraction. We can use the Brain2Vec frozen encoder to produce 768-dimensional embeddings from T1 scans.
	This compact numerical representation serves as input for lightweight downstream classifiers, such as SVM or MLP models.


	```python
	python inference.py brain2vec_inference --model brain2vec_base \
	--img_size 128 \
	--model_weights "model weights (.pth)" \
	--in_dir "path to directory with niftii scans" \
	--out_json "name of json file with output embeddings" \
	--device cuda

	python inference.py vit_inference --model vit_base \
	--num_classes 1 \
	--img_size 128 \
	--mode regression \
	--model_weights "model weights (.pth)" \
	--in_dir "input directory with niftii scans" \
	--out_csv "output csv file with predictions" \
	--device cuda
	```

	## Fine-tuning
	One way of using Brain2Vec is for encoder fine-tuning. The pre-trained encoder weights can serve as initialization for Vision Transformer (ViT) models applied to downstream tasks like disease classification or brain age regression.

	The only thing you need is a csv with MRID and target columns.
	We provide all the code needed to pre-train/fine-tune on SLURM environments on [github](https://github.com/CBICA/Brain2Vec).

	```python
	srun Brain2Vec vit_finetune --img_size 128 \
	--mode binary \
	--model vit_base \
	--label_dict "csv with MRID and target columns" \
	--target "target column" \
	--pretrain "pre-trained model weights (.pth)" \
	--in_dir "input_dir/"
	--batch_size 32 \
	--out_dir "output_dir/" \
	--model_name "model name (.pth)" \
	--device cuda \
	--lr 0.001 \
	--weight_decay 0.05 \
	--optimizer AdamW \
	--epochs 200 \
	--patience 20 \
	--distributed \
	--verbose
	```


	## Pre-training

	We trained with two A100 GPUs for 500 epochs with a patience of 20 epochs. For more training details please read the Brain2Vec paper. Using our code,
	you can parallelize your training on SLURM cluster environments. Other environments are supported but not tested, so bugs may appear.

	```python
	srun Brain2Vec brain2vec_pretrain --img_size 128 \
	--masking_type random \
	--mask_ratio 0.75 \
	--model brain2vec_base \
	--in_dir "input_dir/"
	--batch_size 32 \
	--out_dir "output_dir/" \
	--model_name "model name(.pth)" \
	--device cuda \
	--lr 0.001 \
	--weight_decay 0.05 \
	--optimizer AdamW \
	--epochs 500 \
	--patience 20 \
	--distributed \
	--verbose \
	```

	## Usage on NiChart

	You can use Brain2Vec -- for both feature extraction and disease classification/brain age regression -- on [NiChart](https://neuroimagingchart.com) by simply dragging and dropping your NIfTI files.
	No GPU or installation required.


	## Citation

	If you use our model, make sure to cite our work:
	```
	@misc{maggioros2026brain2vecselfsupervised3dfoundation,
	title={Brain2Vec: A Self-Supervised 3D Foundation Model for Structural Brain MRI},
	author={Spiros Maggioros and Guray Erus and Gareth Harman and George Aidinis and Alexander Getka and Aristeidis Sotiras and Christos Davatzikos},
	year={2026},
	eprint={2610.28411},
	archivePrefix={arXiv},
	primaryClass={cs.CV},
	url={https://arxiv.org/abs/2610.28411},
	}
	```