medarc/native-fmri-pretraining

Native-space fMRI pretraining: encoder weights

Encoder weights for the 19 pretraining runs in Scaling fMRI Foundation Models with Native-Space Data from Heterogeneous Public Studies (NeurIPS 2026 Workshop on Foundation Models for the Brain and Body). Paper: arXiv:XXXX.XXXXX (coming soon). Code: https://github.com/SophontAI/native-fmri-pretraining.

Model

Each run folder (<folder>/seed<N>/) holds the encoder of a 4D ViT-L masked autoencoder for fMRI, taken from the final checkpoint (step 200,000) of one pretraining run.

• 24 transformer blocks, width 1024, 16 heads, 303M parameters, stored in fp32 (1.2 GB per file).
• Tokens are patches of 6×6×6 voxels of 2 mm (12 mm per side) × 4 frames (4 s). A 16-frame clip gives 4 tokens per spatial patch.
• Positions enter through rotary embeddings on patch centres in world millimetres, read from each scan's affine.
• Pretrained on native-space fMRI: volumes are rigidly aligned to a 2 mm MNI152NLin2009cAsym grid, without nonlinear normalization. The hcpya-mni runs are the exception. They use the HCP minimally preprocessed volumes, which are nonlinearly normalized to MNI space.
• Only the encoder is released. The decoder and optimizer state are not, so these files cannot resume training or score reconstruction.

Each run folder has model.safetensors, the encoder state dict with the training step in its header, and config.yaml, the model and preproc settings that rebuild it. MANIFEST.tsv maps each folder to its training run and lists the SHA-256 of each weights file. Run names are the original training-run names: b1, b4, b8 and b16 are the first 1, 4, 8 and 16 shard batches, and h1604 is the 1,520 h subset.

Folders

folder	pretraining corpus	preparation	stored BOLD hours	seeds	runs
openneuro-747h/	OpenNeuro subset, 459 studies	native	747	42, 43	2
openneuro-1520h/	OpenNeuro subset, 604 studies	native	1,520	42, 43	2
openneuro-2984h/	OpenNeuro subset, 708 studies	native	2,984	42, 43	2
openneuro-5919h/	OpenNeuro subset, 762 studies	native	5,919	42, 43	2
openneuro-11837h/	OpenNeuro subset, 800 studies	native	11,837	42, 43	2
openneuro-22373h/	OpenNeuro, full corpus, 837 studies	native	22,373	42, 43, 44	3
hcpya-native/	HCP-YA, 880 participants	native	1,482	42, 43, 44	3
hcpya-mni/	HCP-YA, 880 participants	MNI (HCP minimal preprocessing)	1,482	42, 43, 44	3

The OpenNeuro subsets are nested: each is a prefix of the next, up to the paper's full corpus. The 1,520 h subset matches HCP-YA in hours.

Usage

Install the code (Python 3.14):

git clone --recurse-submodules https://github.com/SophontAI/native-fmri-pretraining.git
cd native-fmri-pretraining
pip install -e ./brainmarks -e './fmri-pretraining[eval]' -e './fmri-data-pipeline[eval]'

To load the weights in Python only, pip install -e ./fmri-pretraining is enough.

Download one run:

from huggingface_hub import snapshot_download

root = snapshot_download("medarc/native-fmri-pretraining", allow_patterns="openneuro-22373h/seed42/*")
folder = f"{root}/openneuro-22373h/seed42"

Load it:

import torch

from fmrifm.checkpoint import load_weights
from fmrifm.config import parse_config
from fmrifm.model import build_model

model = build_model(parse_config(["--config", f"{folder}/config.yaml"]))
load_weights(f"{folder}/model.safetensors", model)  # strict; fills model.encoder, the decoder stays untrained
model.eval()
# x [B, 16, N, 216]: N patches of 6^3 voxels over 16 frames; coord [B, N, 3]: patch centres in mm;
# valid [B, N]: False marks padding patches
with torch.inference_mode():
    tokens = model(x, coord, valid=valid, compute_loss=False)  # [B, 4, N, 1024]

Inputs follow the training preprocessing: 1 s frames, each voxel z-scored within its run, then each frame z-scored across the subject's cortical grey-matter voxels, voxels outside that mask set to 0, only patches with more than 10% cortical grey matter kept, and values clipped to ±3. fmri-data-pipeline builds the per-run steps, and the Brainmarks plugin applies the per-frame and patch steps.

In Brainmarks, use the model plugin fmrifm_native or fmrifm_mni with model_kwargs.ckpt_path=<folder>/model.safetensors. The plugin reads the config.yaml beside the weights. The plugin name selects which preparation of the benchmark data to evaluate on, and every folder works with both. fmri-pretraining/scripts/eval_brainmarks.sbatch runs all 14 tasks (seven targets × two preparations) and takes the weights path as its fourth argument. The evaluation data is not redistributed. Build it with fmri-data-pipeline. AABC also needs its per-subject target maps (AABC_TARGETS_ROOT), which are not public.

The plugin needs the Brainmarks fork pinned as the repository's brainmarks/ submodule (Aceticia/Brainmarks@d464007), which the install above provides. Upstream Brainmarks lacks the padding mask the plugin returns.

Training

All 19 runs use one recipe (fmri-pretraining/configs/model.yaml). Only the corpus and the seed differ. Each sample is a 16-frame (16 s) clip from one BOLD run, read within the subject's cortical grey-matter mask. The encoder sees 96 spatial patches over the whole clip (384 tokens), and the decoder reconstructs 32 other spatial patches. Training ran 200,000 AdamW updates with batch size 128, peak learning rate 7.1e-5, 5,000 warmup steps, cosine decay and weight decay 0.05, in bf16 mixed precision on one H100 GPU. Each run took 60-123 H100-hours.

Data

The preprocessed OpenNeuro corpus is released under CC0 1.0; access instructions are in the code repository. The OpenNeuro models were trained on the paper's corpus (837 training studies; the 838th stored accession is validation-only) or on nested subsets of it. That corpus includes 21 studies since withdrawn from OpenNeuro (290 h of the 22,373 h), which are removed from the public release, so retraining on the public corpus will not reproduce these runs exactly.

HCP-YA is not redistributed. Access requires registration with the Human Connectome Project.

Evaluation

The paper reports frozen-encoder results on seven targets: HCP-YA task condition (Task21), NSD COCO category, AABC age bin and sex, and PPMI, ABIDE and ADHD-200 diagnosis. Table 1 covers the hcpya-mni, hcpya-native, openneuro-1520h and openneuro-22373h runs on MNI-normalized benchmark scans; Appendix Table 6 covers all eight folders on both MNI-normalized and native-space benchmark scans. All were scored with Brainmarks, with the encoder under fp16 autocast.

HCP-YA

The hcpya-native and hcpya-mni models were trained on HCP-YA data.

Data were provided in part by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.

Users of the hcpya-* models should follow the WU-Minn HCP Open Access Data Use Terms.

Limitations

• Four OpenNeuro training studies overlap the HCP-YA evaluation. ds003452 and ds003465 share HCP-YA participants, and ds000244 and ds002685 re-run the HCP task battery. The paper's pretraining corpus kept them. Account for this when evaluating openneuro-* models on HCP-YA. The audit covered the HCP task battery only; stimulus overlap with the other benchmarks was not checked.
• The hcpya-* models were pretrained on HCP-YA runs that include the Task21 probe's training participants; only its validation and test participants are held out. Use the Brainmarks splits when evaluating them on HCP-YA.
• Every run trained for a fixed 200,000 updates. Held-out reconstruction loss was still falling at the end of every OpenNeuro run, and the 747 h model shows signs of overfitting.
• The models expect the paper's preparation: a 2 mm grid, 1 s frames and patches from a cortical grey-matter mask. Other preparations are untested.

License

The weights are released under CC BY-NC 4.0. The code is MIT.

Citation

If you use these weights, please cite our paper:

@inproceedings{liu2026scaling,
  title         = {Scaling {fMRI} Foundation Models with Native-Space Data from Heterogeneous Public Studies},
  author        = {Liu, Xujin Chris and Lane, Connor and Muhammad, Hassan and Abraham, Tanishq and Scotti, Paul},
  booktitle     = {NeurIPS 2026 Workshop on Foundation Models for the Brain and Body (BrainBodyFM)},
  year          = {2026},
  eprint        = {XXXX.XXXXX},
  archivePrefix = {arXiv}
}

Contact: contact@sophontai.com