Blame for README.md

---
license: mit
library_name: braindecode
tags:
- braindecode
- pytorch
- safetensors
- meg
- bbl
- self-supervised
- foundation-model
datasets:
- CamCAN
---

# BBL (pretrained, CamCAN)

Weights of the MEG encoder of *The Brain's Bitter Lesson* (Jayalath et al., ICML 2025),
pretrained on CamCAN with the band, phase and amplitude pretext tasks, for
[`braindecode.models.BBL`](https://braindecode.org/stable/generated/braindecode.models.BBL.html),
converted from the authors' checkpoint to braindecode's parameter names. The class is
added by [braindecode#1553](https://github.com/braindecode/braindecode/pull/1553)
(braindecode 2.1).

```python
from braindecode.models import BBL

model = BBL.from_pretrained("braindecode/bbl-pretrained", n_outputs=2)
features = model(x, return_features=True)["features"]  # (batch, 528)
```

The linear head (`final_layer`, `n_outputs=2` in `config.json`) is not part of the
release: it is PyTorch's default init under `torch.manual_seed(0)`. Passing `n_outputs` to
`from_pretrained` replaces it. Fine-tune or linear-probe before use.

<!-- braindecode:training-data -->
## Training data

The data that shaped these weights. The ids are the `datasets:` field of this card,
which braindecode reads into `model.card_data.datasets` when you call
`from_pretrained` (MOABB class names for MOABB datasets, OpenNeuro / NEMAR
accessions, Hub ids otherwise). `nm`/`on` ids are datasets on [NEMAR](https://nemar.org)
(`on` = NEMAR's copy of the OpenNeuro dataset with the same number). Rows marked
`private:` are data the authors did not release; they are not in `datasets:`.

| `datasets:` id | Dataset | Stage | Modality |
|---|---|---|---|
| `CamCAN` | [Cambridge Centre for Ageing and Neuroscience (Cam-CAN) MEG data, Shafto et al. 2014 / Taylor et al. 2017](https://cam-can.mrc-cbu.cam.ac.uk/dataset/) | pretraining | MEG |

- self-supervised band/phase/amplitude pretext tasks; 583 Cam-CAN subject rows trained (train split; rows with non-zero Adam state)
- Sources: https://github.com/neural-processing-lab/the-brains-bitter-lesson/blob/3965a2be11677d9c2b9cecb88109a0af6bf19d1c/README.md; checkpoint hyper_parameters (inspected 2026-11-06, meg-ports-20261106/bbl/weights/CHECKPOINT.md and evidence/parity.log); arXiv:2406.04328 Sec. 4 Datasets.


## Intended input

- MEG, `(batch, n_chans, 125)`: CamCAN's 306 Elekta `MEG*` channels in FIF order (the
  projector is indexed by position; inputs with fewer channels are zero-padded to 306, as
  the reference collator does). `dataset_index` defaults to slot 0 (CamCAN).
- Preprocessed as in the authors' `data/preprocess.py` and `data/utils.py`: band-pass
  0.1-125 Hz with 50 and 100 Hz notches, resampled to 250 Hz, 0.5 s windows (125 samples);
  each channel scaled by the median and interquartile range fitted on its recording,
  clipped to `[-5, 5]`, and the mean of the first 0.5 s subtracted. `forward` only does
  the zero-padding.
- `subject_index` row `d * 1024 + s` for subject `s` of dataset slot `d`. Only CamCAN
  subject rows are trained; the `trained_subjects` buffer flags those 583 rows and drives
  the mean-subject fallback used when no `subject_index` is given.

## Source

- Paper: D. Jayalath, G. Landau, B. Shillingford, M. Woolrich, O. Parker Jones. *The
  Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning.* ICML 2025.
  [arXiv:2406.04328](https://arxiv.org/abs/2406.04328)
- Code: [neural-processing-lab/the-brains-bitter-lesson @ `3965a2be11677d9c2b9cecb88109a0af6bf19d1c`](https://github.com/neural-processing-lab/the-brains-bitter-lesson/tree/3965a2be11677d9c2b9cecb88109a0af6bf19d1c) (MIT)
- Weights: the checkpoint "pretrained with CamCAN" linked from the README of that
  repository ("Pre-trained checkpoints"), hosted on Google Drive (file id
  `1cYdKDYoIupjoMfq5cEWgB-W74XhztWEw`), downloaded 2026-11-06: 509,226,678 bytes, sha256
  `94a35ca9bb8ca00941ce355d1b5cd97e4984b689209317a4eb18a3560a8e940b`. A PyTorch Lightning
  2.5.0.post0 `Pretrainer` checkpoint (original name
  `camcan_preproc_full_cont-best-epoch=02-val_loss=1.26.ckpt`; epoch 2, global step 60507;
  `input_dim=306, model_dim=512, subject_embedding_dim=16, projector_dim=1024`, tasks band,
  phase, amp).

## Conversion

`convert.py` (in this repository) was run with braindecode at
`bruAristimunha/braindecode@388ac6cb96361c2df3d5b530172ed6662b9e6a65` (branch `feat/meg-bbl`, PR #1553):

```
python convert.py bbl.ckpt <out_dir>
```

It keeps the `backbone.*` tensors, renames them to braindecode's keys (the old-style
`weight_g`/`weight_v` weight norm maps to `parametrizations.weight.original0/1`, which
computes the same `torch._weight_norm`), fills the `trained_subjects` buffer with the
subject rows the optimizer updated (non-zero Adam second moment; the original keeps that set
in a Python attribute that is not saved), loads everything into
`BBL(n_chans=306, n_outputs=2, n_times=125, sfreq=250.0)` and calls `save_pretrained`.
Tensor values are copied unchanged.

Dropped (training only): the SSL projector, the band/amplitude/phase pretext classifiers,
and the Lightning/optimizer state.

| file | sha256 |
|---|---|
| `model.safetensors` | `3f5022d2564762917d5b9d0df35060b8f0dd4f09128cc9cb076d5fd6aed193c3` |
| `pytorch_model.bin` | `7d0da29ad7d4404917bfa70cf6b4b3c692dcbc10501eec9ec100fb5c785679bb` |
| `config.json` | `71ba8c6efac12d7595f0723522c090400017a9b41255a8f620847a1313699f7f` |
| `convert.py` | `c3388e146017e55aa439dfba995ea8e5a06efe3e4d77acd3f97b98112b950581` |

## Verification

`BBL.from_pretrained(<this repo>, strict=True)` (loaded tensors equal `convert.py` applied
to the original checkpoint) against the original `Backbone` with the same weights and
inputs, CPU float32, eval mode, 4 windows of 0.5 s: features, logits, loss, input
gradients and the gradients of all 55 backbone parameters and of the head are equal
(max-abs difference 0.0) with 306 sensors; with 204 sensors (zero-padded to 306) the
features are equal. The mean-subject fallback (batch 1) differs by 3.7e-8 in the
16 subject-embedding features (the mean is taken in another order); the encoder part is
equal.

## License and provenance

`license: mit` follows the repository that distributes the checkpoint. The code of
[the-brains-bitter-lesson](https://github.com/neural-processing-lab/the-brains-bitter-lesson)
is MIT ([licence](https://github.com/neural-processing-lab/the-brains-bitter-lesson/blob/main/LICENSE),
Copyright (c) 2025 Dulhan Jayalath). The weights are distributed by the authors through
that repository's README (Google Drive link) without a separate weight licence. They are
re-hosted here unmodified in converted form (renamed keys, training-only parts dropped),
with attribution to the authors. Authors or rights holders who want this copy changed or
removed: please open an issue at https://github.com/braindecode/braindecode/issues and it
will be taken down.

## Citation

```bibtex
@inproceedings{jayalath2025bbl,
  title     = {{The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning}},
  author    = {Jayalath, Dulhan and Landau, Gilad and Shillingford, Brendan and Woolrich, Mark and Parker Jones, Oiwi},
  booktitle = {Forty-second International Conference on Machine Learning (ICML)},
  year      = {2025},
  organization = {PMLR}
}

@article{aristimunha2025braindecode,
  title   = {Braindecode: a deep learning library for raw electrophysiological data},
  author  = {Aristimunha, Bruno and others},
  journal = {Zenodo},
  year    = {2025},
  doi     = {10.5281/zenodo.17699192},
}
```