neurovlm/text2brain-mirror

Text2Brain (mirror)

Unmodified mirror of the released Text2Brain model: Ngo, Nguyen, Chen and Sabuncu, Text2Brain: Synthesis of Brain Activation Maps from Free-form Text Query, MICCAI 2021 (arXiv:2109.13814). Code: https://github.com/ngohgia/text2brain (MIT, Copyright (c) 2021 ngo.h.gia, see LICENSE). We are not the authors. It is mirrored so the NeuroVLM reviewer analyses can pin it: the checkpoint link of the paper's repository is dead, and the current link is a Google Drive file.

file	origin	sha256
text2brain_checkpoint.tar.gz	https://drive.usercontent.google.com/download?id=1mBO1frmlSMgXGN-F1WKQKlCO0gIk_MHq&export=download, downloaded 2026-10-13, byte-identical	08e8af9af54b5e38e743be56b47c0c987197bc60dc91e3e56aa7feaf406ec6c8
text2brain.safetensors	state_dict of text2brain_checkpoint.pth in the tar (epoch 1220); every tensor unchanged; optimizer and scheduler state dropped	e1f4dccc5b41d934d0239c5f734a0d05cd61de5ed5ad0162ec2fc5bd2aed43b5
text2brain-6f29389.tar.gz	git archive of https://github.com/ngohgia/text2brain at commit 6f29389658e23b89e69248f3d3421dd2bd79b269	35dbb0e6ea58118727a715bc61ea19c5db70a8a242100d3383af863c452f90b1

The model: fine-tuned SciBERT (allenai/scibert_scivocab_uncased vocabulary and config) -> Linear(768, 64x4x5x4) -> 3D transposed-convolution decoder (32 filters), output 40x48x40 placed into the 46x55x46 MNI152 4 mm grid as upstream predict.py.

Running it with current libraries

The upstream code (Python 3.6, torch 1.5.1, transformers 3.3.1) needs two fixes under transformers 4.x:

1. models/text2brain_model.py: call the encoder with return_dict=False. Otherwise _, embedding = self.encoder(...) unpacks the keys of the returned ModelOutput ("last_hidden_state", "pooler_output") and dropout fails on a string.
2. Drop the checkpoint key encoder.embeddings.position_ids (it equals arange(512); transformers no longer saves this buffer) before the strict load_state_dict. Otherwise the load fails with "Unexpected key(s)" (upstream issue #6).

The upstream code also moves the encoder and the inputs to CUDA whenever a GPU is visible; the NeuroVLM copy keeps them on the model's device. The patched code is in the NeuroVLM repository under scripts/figures/review/core/a1_text_to_brain_baselines/text2brain/.