--- license: cc-by-nd-4.0 language: - en library_name: pytorch tags: - eeg - biosignal - foundation-model - self-supervised - masked-modeling - windowed-alternating-attention - neuroscience datasets: - TUEG - SEED - BOAS - SleepEDFx - BCI-NER - GWD - CHB-MIT - Neonate - PhysioNet-MI - SHU-MI - TUAB - STEW - SEED-VIG - ISRUC - Mumtaz - MentalArithmetic - SEED-V metrics: - roc_auc - f1 - nrmse model-index: - name: S-CEReBrO results: - task: type: time-series-classification name: Seizure Detection dataset: type: CHB-MIT name: CHB-MIT metrics: - type: roc_auc value: 87.45 name: AUROC - task: type: time-series-classification name: Seizure Detection dataset: type: Neonate name: Neonate metrics: - type: roc_auc value: 85.76 name: AUROC - task: type: time-series-classification name: Motor Imagery Classification dataset: type: PhysioNet-MI name: PhysioNet-MI metrics: - type: f1 value: 60.93 name: Weighted F1 - task: type: time-series-classification name: Motor Imagery Classification dataset: type: SHU-MI name: SHU-MI metrics: - type: roc_auc value: 66.48 name: AUROC - task: type: time-series-classification name: Abnormal Classification dataset: type: TUAB name: TUAB metrics: - type: roc_auc value: 89.30 name: AUROC - task: type: time-series-classification name: Mental Workload Classification dataset: type: STEW name: STEW metrics: - type: f1 value: 52.90 name: Weighted F1 - task: type: time-series-regression name: Vigilance Estimation dataset: type: SEED-VIG name: SEED-VIG metrics: - type: nrmse value: 0.93 name: NRMSE - task: type: time-series-classification name: Sleep Stage Classification dataset: type: ISRUC name: ISRUC metrics: - type: f1 value: 78.23 name: Weighted F1 - task: type: time-series-classification name: Mental Disorder Diagnosis dataset: type: Mumtaz name: Mumtaz metrics: - type: roc_auc value: 98.23 name: AUROC - task: type: time-series-classification name: Mental Stress Detection dataset: type: MentalArithmetic name: MentalArithmetic metrics: - type: roc_auc value: 68.97 name: AUROC - task: type: time-series-classification name: Emotion Recognition dataset: type: SEED-V name: SEED-V metrics: - type: f1 value: 28.16 name: Weighted F1 ---

S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring

Github License Paper

**S-CEReBrO** (Streaming CEReBrO) is an **EEG foundation model** designed for **continuous monitoring**. S-CEReBrO addresses the memory overflow bottleneck of continuous monitoring by utilizing a novel **Windowed Alternating Attention mechanism**. This factorizes attention computation into fixed-size spatiotemporal windows, guaranteeing a **constant KV cache memory** as only the active window requires resident attention maps. --- ## ๐Ÿ”’ License & Usage Policy (Weights) **Weights license:** The released model weights are licensed under **Creative Commons Attributionโ€“NoDerivatives 4.0 (CC BY-ND 4.0)**. This section summarizes the practical implications for users. *This is not legal advice; please read the full license text.* ### โœ… You may - **Use** and **redistribute** the **unmodified** S-CEReBrO weights (including in commercial settings) **with proper attribution** to the S-CEReBrO authors. - **Fine-tune / adapt** the weights **for your internal use** (research or production) **without redistributing** the modified weights. - **Publish your code, configs, logs, and papers** describing experiments with S-CEReBrO (please cite the paper). ### ๐Ÿšซ You may not - **Share, host, or redistribute any modified weights** (including LoRA/adapter/delta checkpoints or pruned/quantized variants). Any parameter set that encodes an adaptation is considered a derivative and cannot be shared under CC BY-ND 4.0. - **Imply endorsement** by the S-CEReBrO authors for any derivative or evaluation without our written permission. - **Use the S-CEReBrO name** in a way that suggests your modified model is an official S-CEReBrO release. ### ๐Ÿค How to contribute improvements (PR-gated releases) We welcome community improvements via a **pull-request (PR)** workflow. If you believe your improvements should become an **official S-CEReBrO release**: 1. **Open a PR** in the [BioFoundation repository](https://github.com/pulp-bio/biofoundation) describing the change (architecture/head/training recipe, datasets, preprocessing, compute). 2. Include **reproducibility artifacts**: configs, seeds, scripts, environment details, training/validation logs, and the **evaluation protocol** with exact splits. 3. Provide **comprehensive results** (AUROC/AUPR/BA, FLOPs, memory) vs. the baselines reported in the S-CEReBrO paper. 4. After **maintainer review**, approved changes will be **retrained/validated** and, if accepted, **released by the maintainers** as a new **official S-CEReBrO** checkpoint under **CC BY-ND 4.0**. > Rationale: CC BY-ND protects users from fragmented, lower-quality variants, while still enabling internal fine-tuning and a path for the community to upstream improvements through review. --- ## ๐Ÿ”Ž Model Summary - **Goal:** Realize efficient, generalizable, and continuous EEG monitoring while breaking the memory barrier of Transformer-based architectures. - **Core idea:** **Windowed Alternating Attention** interleaves spatial and temporal attention blocks. This restricts interactions to local neighborhoods in time and space, allowing the model to process long signals while maintaining a constant resident memory. Complexity is $\mathcal{O}(CT)$. - **Pre-training data:** **>25,000 hours** of EEG from **>12,000 subjects**. Includes TUEG, SEED series, BOAS, SleepEDFx, BCI-NER, and GWD. Downstream subjects (e.g. TUAB) are strictly excluded to prevent leakage. - **Downstream tasks:** Seizure detection (CHB-MIT, Neonate), Motor Imagery (PhysioNet-MI, SHU-MI), Abnormal classification (TUAB), Mental Workload (STEW), Vigilance Estimation (SEED-VIG), Sleep Staging (ISRUC), Mental Disorder Diagnosis (Mumtaz), Mental Stress (Mental Arithmetic), and Emotion Recognition (SEED-V). --- ## ๐Ÿš€ Model Variants S-CEReBrO is highly compact, utilizing only 2.4M parameters. This design allows it to achieve state-of-the-art performance with up to 60% fewer parameters than existing foundation models. --- ## ๐Ÿ“Š Results S-CEReBrO achieves state-of-the-art performance on 7 of 11 downstream public benchmarks. - **CHB-MIT (seizure detection):** 87.45 AUROC. - **Neonate (seizure detection):** 85.76 AUROC. - **PhysioNet-MI (motor imagery):** 60.93 Weighted F1. - **SHU-MI (motor imagery):** 66.48 AUROC. - **TUAB (abnormal classification):** 89.30 AUROC. - **STEW (mental workload):** 52.90 Weighted F1. - **SEED-VIG (vigilance estimation):** 0.93 NRMSE. - **ISRUC (sleep staging):** 78.23 Weighted F1. - **Mumtaz (mental disorder diagnosis):** 98.23 AUROC. - **Mental Arithmetic (mental stress detection):** 68.97 AUROC. - **SEED-V (emotion recognition):** 28.16 Weighted F1. **Efficiency & Scaling:** Without low-level optimzations, windowed Alternating Attention can process signals 100x longer than full self-attention (up to 14 hours / 50,000 seconds). It also processes signals 3x longer than low-rank linear attention. Compared to low-rank linear attention on long contexts (15,000 seconds), it requires 55% of the memory while increasing inference throughput by 2.1x. --- ## ๐Ÿง  Intended Use & Limitations **Intended use.** Research on continuous EEG representation learning and classification (clinical diagnostics, BCI, state monitoring) where long context and memory efficiency are necessary. **Limitations.** - **Not a medical device.** Do **not** use for clinical decisions without proper validation & regulatory clearance. - **Unseen topologies:** Zero-shot transfer to **very different/dense** layouts can underperform SOTA despite positive scaling; consider augmenting pre-training montage diversity and spatial encodings. - **Distribution shifts:** Performance varies across cohorts, devices, and label protocols; validate locally and consider domain adaptation. --- ## ๐Ÿ—๏ธ Architecture & Training **Spatiotemporal Tokenization.** The raw EEG recording is tokenized using a sliding window. A shared convolutional encoder projects each patch into a latent embedding. Spatiotemporal topology is preserved via an index-based matrix for temporal order and a continuous spatial embedding mapped via an MLP from 3D electrode coordinates. **Windowed Temporal Attention (WTA).** WTA layers model temporal dependencies independently for each channel. Each query attends only to keys within a local neighborhood, avoiding memory accumulation. **Windowed Spatial Attention (WSA).** WSA layers facilitate cross-channel information exchange at a fixed time index. Attention computation is restricted to a local neighborhood. Layer-specific dilation and shift factors stagger receptive fields to enable distant, physiologically related channels to interact. **Pre-training objectives.** The model is pre-trained using Masked Autoencoding (MAE). The objective minimizes the weighted sum of reconstruction losses for masked and visible patch positions. --- ## ๐Ÿ”ง How to Use S-CEReBrO is supported within the BioFoundation framework. Experiments and setups follow similar pipeline structures: 1. **Install & read data prep**: clone the [BioFoundation repo](https://github.com/pulp-bio/biofoundation). Set up the environment as described there, then open `make_datasets/README.md` for dataset-specific notes. 2. **Point to weights**: set `pretrained_safetensors_path` to the downloaded weights in your experiment YAML. 3. **Preprocess data**: acquire fine-tuning dataset and follow preprocessing protocol to generate `train/test/val.lmdb` files. 4. **Update data module config** 5. **Task settings**: Configure `classification_type`, `model.num_classes`, and task modes inside your YAML config based on downstream objectives. 6. **Launch fine-tuning (Hydra)**: ```bash python -u run_train.py +experiment=s_cerebro_finetune ``` --- ## ๐Ÿ”— Sources - **Code:** https://github.com/pulp-bio/BioFoundation - **Paper:** S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring (arxiv:2607.27913) --- ## ๐Ÿ“œ Citation If you use S-CEReBrO, please cite: ```bibtex @misc{bucagu2026scerebrobreakingmemorybarrier, title={S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring}, author={Glenn Anta Bucagu and Thorir Mar Ingolfsson and Yawei Li and Luca Benini}, year={2026}, eprint={2607.27913}, archivePrefix={arXiv}, primaryClass={cs.LG}, url={https://arxiv.org/abs/2607.27913}, } ``` --- ## ๐Ÿ› ๏ธ Maintenance & Contact - **Issues & support:** please open a GitHub issue in the BioFoundation repository. --- ## ๐Ÿ”— Related Models - **[LUNA](https://huggingface.co/PulpBio/LUNA)** โ€” Transformer-based topology-agnostic EEG foundation model (NeurIPS 2025). Source of the channel-unification cross-attention module that LuMamba reuses. - **[FEMBA](https://huggingface.co/PulpBio/FEMBA)** โ€” Bidirectional Mamba foundation model for EEG. Source of the linear-complexity temporal backbone that LuMamba reuses. - **[TinyMyo](https://huggingface.co/PulpBio/TinyMyo)** โ€” Tiny foundation model for flexible EMG signal processing at the edge. ## ๐Ÿ—’๏ธ Changelog - **v1.0:** Initial release of LuMamba model card with task-specific checkpoints and instructions.