Card: list the training data (datasets:, base_model:) and add a Training data section

6d736b4 verified 1 day ago

7.56 kB

license: mit
library_name: braindecode
tags:
  - braindecode
  - pytorch
  - safetensors
  - meg
  - bbl
  - self-supervised
  - foundation-model
datasets:
  - CamCAN

BBL (pretrained, CamCAN)

Weights of the MEG encoder of The Brain's Bitter Lesson (Jayalath et al., ICML 2025), pretrained on CamCAN with the band, phase and amplitude pretext tasks, for braindecode.models.BBL, converted from the authors' checkpoint to braindecode's parameter names. The class is added by braindecode#1553 (braindecode 2.1).

from braindecode.models import BBL

model = BBL.from_pretrained("braindecode/bbl-pretrained", n_outputs=2)
features = model(x, return_features=True)["features"]  # (batch, 528)

The linear head (final_layer, n_outputs=2 in config.json) is not part of the release: it is PyTorch's default init under torch.manual_seed(0). Passing n_outputs to from_pretrained replaces it. Fine-tune or linear-probe before use.

Training data

The data that shaped these weights. The ids are the datasets: field of this card, which braindecode reads into model.card_data.datasets when you call from_pretrained (MOABB class names for MOABB datasets, OpenNeuro / NEMAR accessions, Hub ids otherwise). nm/on ids are datasets on NEMAR (on = NEMAR's copy of the OpenNeuro dataset with the same number). Rows marked private: are data the authors did not release; they are not in datasets:.

`datasets:` id	Dataset	Stage	Modality
`CamCAN`	Cambridge Centre for Ageing and Neuroscience (Cam-CAN) MEG data, Shafto et al. 2014 / Taylor et al. 2017	pretraining	MEG

self-supervised band/phase/amplitude pretext tasks; 583 Cam-CAN subject rows trained (train split; rows with non-zero Adam state)
Sources: https://github.com/neural-processing-lab/the-brains-bitter-lesson/blob/3965a2be11677d9c2b9cecb88109a0af6bf19d1c/README.md; checkpoint hyper_parameters (inspected 2026-11-06, meg-ports-20261106/bbl/weights/CHECKPOINT.md and evidence/parity.log); arXiv:2406.04328 Sec. 4 Datasets.

Intended input

MEG, (batch, n_chans, 125): CamCAN's 306 Elekta MEG* channels in FIF order (the projector is indexed by position; inputs with fewer channels are zero-padded to 306, as the reference collator does). dataset_index defaults to slot 0 (CamCAN).
Preprocessed as in the authors' data/preprocess.py and data/utils.py: band-pass 0.1-125 Hz with 50 and 100 Hz notches, resampled to 250 Hz, 0.5 s windows (125 samples); each channel scaled by the median and interquartile range fitted on its recording, clipped to [-5, 5], and the mean of the first 0.5 s subtracted. forward only does the zero-padding.
subject_index row d * 1024 + s for subject s of dataset slot d. Only CamCAN subject rows are trained; the trained_subjects buffer flags those 583 rows and drives the mean-subject fallback used when no subject_index is given.

Source

Paper: D. Jayalath, G. Landau, B. Shillingford, M. Woolrich, O. Parker Jones. The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning. ICML 2025. arXiv:2406.04328
Code: neural-processing-lab/the-brains-bitter-lesson @ 3965a2be11677d9c2b9cecb88109a0af6bf19d1c (MIT)
Weights: the checkpoint "pretrained with CamCAN" linked from the README of that repository ("Pre-trained checkpoints"), hosted on Google Drive (file id 1cYdKDYoIupjoMfq5cEWgB-W74XhztWEw), downloaded 2026-11-06: 509,226,678 bytes, sha256 94a35ca9bb8ca00941ce355d1b5cd97e4984b689209317a4eb18a3560a8e940b. A PyTorch Lightning 2.5.0.post0 Pretrainer checkpoint (original name camcan_preproc_full_cont-best-epoch=02-val_loss=1.26.ckpt; epoch 2, global step 60507; input_dim=306, model_dim=512, subject_embedding_dim=16, projector_dim=1024, tasks band, phase, amp).

Conversion

convert.py (in this repository) was run with braindecode at bruAristimunha/braindecode@388ac6cb96361c2df3d5b530172ed6662b9e6a65 (branch feat/meg-bbl, PR #1553):

python convert.py bbl.ckpt <out_dir>

It keeps the backbone.* tensors, renames them to braindecode's keys (the old-style weight_g/weight_v weight norm maps to parametrizations.weight.original0/1, which computes the same torch._weight_norm), fills the trained_subjects buffer with the subject rows the optimizer updated (non-zero Adam second moment; the original keeps that set in a Python attribute that is not saved), loads everything into BBL(n_chans=306, n_outputs=2, n_times=125, sfreq=250.0) and calls save_pretrained. Tensor values are copied unchanged.

Dropped (training only): the SSL projector, the band/amplitude/phase pretext classifiers, and the Lightning/optimizer state.

file	sha256
`model.safetensors`	`3f5022d2564762917d5b9d0df35060b8f0dd4f09128cc9cb076d5fd6aed193c3`
`pytorch_model.bin`	`7d0da29ad7d4404917bfa70cf6b4b3c692dcbc10501eec9ec100fb5c785679bb`
`config.json`	`71ba8c6efac12d7595f0723522c090400017a9b41255a8f620847a1313699f7f`
`convert.py`	`c3388e146017e55aa439dfba995ea8e5a06efe3e4d77acd3f97b98112b950581`

Verification

BBL.from_pretrained(<this repo>, strict=True) (loaded tensors equal convert.py applied to the original checkpoint) against the original Backbone with the same weights and inputs, CPU float32, eval mode, 4 windows of 0.5 s: features, logits, loss, input gradients and the gradients of all 55 backbone parameters and of the head are equal (max-abs difference 0.0) with 306 sensors; with 204 sensors (zero-padded to 306) the features are equal. The mean-subject fallback (batch 1) differs by 3.7e-8 in the 16 subject-embedding features (the mean is taken in another order); the encoder part is equal.

License and provenance

license: mit follows the repository that distributes the checkpoint. The code of the-brains-bitter-lesson is MIT (licence, Copyright (c) 2025 Dulhan Jayalath). The weights are distributed by the authors through that repository's README (Google Drive link) without a separate weight licence. They are re-hosted here unmodified in converted form (renamed keys, training-only parts dropped), with attribution to the authors. Authors or rights holders who want this copy changed or removed: please open an issue at https://github.com/braindecode/braindecode/issues and it will be taken down.

Citation

@inproceedings{jayalath2025bbl,
  title     = {{The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning}},
  author    = {Jayalath, Dulhan and Landau, Gilad and Shillingford, Brendan and Woolrich, Mark and Parker Jones, Oiwi},
  booktitle = {Forty-second International Conference on Machine Learning (ICML)},
  year      = {2025},
  organization = {PMLR}
}

@article{aristimunha2025braindecode,
  title   = {Braindecode: a deep learning library for raw electrophysiological data},
  author  = {Aristimunha, Bruno and others},
  journal = {Zenodo},
  year    = {2025},
  doi     = {10.5281/zenodo.17699192},
}