huzey/brain-synthesizer

NSD Brain Encoding Model (Inference)

Pre-trained model for NSD brain encoding.

from PIL import Image
from transformers import AutoImageProcessor, AutoModel

image_processor = AutoImageProcessor.from_pretrained("facebook/dinov2-large")
image = Image.open("example.jpg").convert("RGB")
pixel_values = image_processor(images=image, return_tensors="pt")["pixel_values"]

## whole brain model, including non visual brain
model = AutoModel.from_pretrained(
    "huzey/brain-synthesizer",
    trust_remote_code=True,
    subject="subj01",  # options: subj01, subj02, subj05, subj07
)
output = model(pixel_values)
print(output.shape)  # torch.Size([1, 327684]), vertices in fsaverage, first half is left hemisphere, second half right hemisphere

Subject options

• subject: any subject ID with matching checkpoint files in the repo (for example, subj01, subj02, subj05, subj07).
• part is fixed to whole_brain: outputs all fsaverage 327684 vertices, first half is left hemisphere, second half is right hemisphere.