| --- |
| license: cc-by-nd-4.0 |
| language: |
| - en |
| library_name: pytorch |
| tags: |
| - eeg |
| - biosignal |
| - foundation-model |
| - self-supervised |
| - masked-modeling |
| - windowed-alternating-attention |
| - neuroscience |
| datasets: |
| - TUEG |
| - SEED |
| - BOAS |
| - SleepEDFx |
| - BCI-NER |
| - GWD |
| - CHB-MIT |
| - Neonate |
| - PhysioNet-MI |
| - SHU-MI |
| - TUAB |
| - STEW |
| - SEED-VIG |
| - ISRUC |
| - Mumtaz |
| - MentalArithmetic |
| - SEED-V |
| metrics: |
| - roc_auc |
| - f1 |
| - nrmse |
| model-index: |
| - name: S-CEReBrO |
| results: |
| - task: |
| type: time-series-classification |
| name: Seizure Detection |
| dataset: |
| type: CHB-MIT |
| name: CHB-MIT |
| metrics: |
| - type: roc_auc |
| value: 87.45 |
| name: AUROC |
| - task: |
| type: time-series-classification |
| name: Seizure Detection |
| dataset: |
| type: Neonate |
| name: Neonate |
| metrics: |
| - type: roc_auc |
| value: 85.76 |
| name: AUROC |
| - task: |
| type: time-series-classification |
| name: Motor Imagery Classification |
| dataset: |
| type: PhysioNet-MI |
| name: PhysioNet-MI |
| metrics: |
| - type: f1 |
| value: 60.93 |
| name: Weighted F1 |
| - task: |
| type: time-series-classification |
| name: Motor Imagery Classification |
| dataset: |
| type: SHU-MI |
| name: SHU-MI |
| metrics: |
| - type: roc_auc |
| value: 66.48 |
| name: AUROC |
| - task: |
| type: time-series-classification |
| name: Abnormal Classification |
| dataset: |
| type: TUAB |
| name: TUAB |
| metrics: |
| - type: roc_auc |
| value: 89.30 |
| name: AUROC |
| - task: |
| type: time-series-classification |
| name: Mental Workload Classification |
| dataset: |
| type: STEW |
| name: STEW |
| metrics: |
| - type: f1 |
| value: 52.90 |
| name: Weighted F1 |
| - task: |
| type: time-series-regression |
| name: Vigilance Estimation |
| dataset: |
| type: SEED-VIG |
| name: SEED-VIG |
| metrics: |
| - type: nrmse |
| value: 0.93 |
| name: NRMSE |
| - task: |
| type: time-series-classification |
| name: Sleep Stage Classification |
| dataset: |
| type: ISRUC |
| name: ISRUC |
| metrics: |
| - type: f1 |
| value: 78.23 |
| name: Weighted F1 |
| - task: |
| type: time-series-classification |
| name: Mental Disorder Diagnosis |
| dataset: |
| type: Mumtaz |
| name: Mumtaz |
| metrics: |
| - type: roc_auc |
| value: 98.23 |
| name: AUROC |
| - task: |
| type: time-series-classification |
| name: Mental Stress Detection |
| dataset: |
| type: MentalArithmetic |
| name: MentalArithmetic |
| metrics: |
| - type: roc_auc |
| value: 68.97 |
| name: AUROC |
| - task: |
| type: time-series-classification |
| name: Emotion Recognition |
| dataset: |
| type: SEED-V |
| name: SEED-V |
| metrics: |
| - type: f1 |
| value: 28.16 |
| name: Weighted F1 |
| --- |
| <div align="center"> |
| <h1>S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring</h1> |
| </div> |
| <p align="center"> |
| <a href="https://github.com/pulp-bio/biofoundation"> |
| <img src ="https://img.shields.io/github/stars/pulp-bio/biofoundation?color=ccf" alt="Github"> |
| </a> |
| <a href="https://creativecommons.org/licenses/by-nd/4.0/"> |
| <img src="https://img.shields.io/badge/License-CC_BY--ND_4.0-lightgrey.svg" alt="License"> |
| </a> |
| <a href="https://arxiv.org/abs/2607.27913"> |
| <img src="https://img.shields.io/badge/arXiv-2607.27913-b31b1b.svg" alt="Paper"> |
| </a> |
| </p> |
| |
| **S-CEReBrO** (Streaming CEReBrO) is an **EEG foundation model** designed for **continuous monitoring**. |
| S-CEReBrO addresses the memory overflow bottleneck of continuous monitoring by utilizing a novel **Windowed Alternating Attention mechanism**. This factorizes attention computation into fixed-size spatiotemporal windows, guaranteeing a **constant KV cache memory** as only the active window requires resident attention maps. |
|
|
| --- |
|
|
| ## 🔒 License & Usage Policy (Weights) |
|
|
| **Weights license:** The released model weights are licensed under **Creative Commons Attribution–NoDerivatives 4.0 (CC BY-ND 4.0)**. This section summarizes the practical implications for users. *This is not legal advice; please read the full license text.* |
|
|
| ### ✅ You may |
| - **Use** and **redistribute** the **unmodified** S-CEReBrO weights (including in commercial settings) **with proper attribution** to the S-CEReBrO authors. |
| - **Fine-tune / adapt** the weights **for your internal use** (research or production) **without redistributing** the modified weights. |
| - **Publish your code, configs, logs, and papers** describing experiments with S-CEReBrO (please cite the paper). |
|
|
| ### 🚫 You may not |
| - **Share, host, or redistribute any modified weights** (including LoRA/adapter/delta checkpoints or pruned/quantized variants). Any parameter set that encodes an adaptation is considered a derivative and cannot be shared under CC BY-ND 4.0. |
| - **Imply endorsement** by the S-CEReBrO authors for any derivative or evaluation without our written permission. |
| - **Use the S-CEReBrO name** in a way that suggests your modified model is an official S-CEReBrO release. |
|
|
| ### 🤝 How to contribute improvements (PR-gated releases) |
| We welcome community improvements via a **pull-request (PR)** workflow. If you believe your improvements should become an **official S-CEReBrO release**: |
| 1. **Open a PR** in the [BioFoundation repository](https://github.com/pulp-bio/biofoundation) describing the change (architecture/head/training recipe, datasets, preprocessing, compute). |
| 2. Include **reproducibility artifacts**: configs, seeds, scripts, environment details, training/validation logs, and the **evaluation protocol** with exact splits. |
| 3. Provide **comprehensive results** (AUROC/AUPR/BA, FLOPs, memory) vs. the baselines reported in the S-CEReBrO paper. |
| 4. After **maintainer review**, approved changes will be **retrained/validated** and, if accepted, **released by the maintainers** as a new **official S-CEReBrO** checkpoint under **CC BY-ND 4.0**. |
|
|
| > Rationale: CC BY-ND protects users from fragmented, lower-quality variants, while still enabling internal fine-tuning and a path for the community to upstream improvements through review. |
|
|
| --- |
|
|
| ## 🔎 Model Summary |
|
|
| - **Goal:** Realize efficient, generalizable, and continuous EEG monitoring while breaking the memory barrier of Transformer-based architectures. |
| - **Core idea:** **Windowed Alternating Attention** interleaves spatial and temporal attention blocks. This restricts interactions to local neighborhoods in time and space, allowing the model to process long signals while maintaining a constant resident memory. Complexity is $\mathcal{O}(CT)$. |
| - **Pre-training data:** **>25,000 hours** of EEG from **>12,000 subjects**. Includes TUEG, SEED series, BOAS, SleepEDFx, BCI-NER, and GWD. Downstream subjects (e.g. TUAB) are strictly excluded to prevent leakage. |
| - **Downstream tasks:** Seizure detection (CHB-MIT, Neonate), Motor Imagery (PhysioNet-MI, SHU-MI), Abnormal classification (TUAB), Mental Workload (STEW), Vigilance Estimation (SEED-VIG), Sleep Staging (ISRUC), Mental Disorder Diagnosis (Mumtaz), Mental Stress (Mental Arithmetic), and Emotion Recognition (SEED-V). |
|
|
| --- |
|
|
| ## 🚀 Model Variants |
|
|
| S-CEReBrO is highly compact, utilizing only 2.4M parameters. This design allows it to achieve state-of-the-art performance with up to 60% fewer parameters than existing foundation models. |
|
|
| --- |
|
|
| ## 📊 Results |
|
|
| S-CEReBrO achieves state-of-the-art performance on 7 of 11 downstream public benchmarks. |
|
|
| - **CHB-MIT (seizure detection):** 87.45 AUROC. |
| - **Neonate (seizure detection):** 85.76 AUROC. |
| - **PhysioNet-MI (motor imagery):** 60.93 Weighted F1. |
| - **SHU-MI (motor imagery):** 66.48 AUROC. |
| - **TUAB (abnormal classification):** 89.30 AUROC. |
| - **STEW (mental workload):** 52.90 Weighted F1. |
| - **SEED-VIG (vigilance estimation):** 0.93 NRMSE. |
| - **ISRUC (sleep staging):** 78.23 Weighted F1. |
| - **Mumtaz (mental disorder diagnosis):** 98.23 AUROC. |
| - **Mental Arithmetic (mental stress detection):** 68.97 AUROC. |
| - **SEED-V (emotion recognition):** 28.16 Weighted F1. |
|
|
| **Efficiency & Scaling:** Without low-level optimzations, windowed Alternating Attention can process signals 100x longer than full self-attention (up to 14 hours / 50,000 seconds). It also processes signals 3x longer than low-rank linear attention. Compared to low-rank linear attention on long contexts (15,000 seconds), it requires 55% of the memory while increasing inference throughput by 2.1x. |
|
|
| --- |
|
|
| ## 🧠 Intended Use & Limitations |
|
|
| **Intended use.** Research on continuous EEG representation learning and classification (clinical diagnostics, BCI, state monitoring) where long context and memory efficiency are necessary. |
|
|
| **Limitations.** |
| - **Not a medical device.** Do **not** use for clinical decisions without proper validation & regulatory clearance. |
| - **Unseen topologies:** Zero-shot transfer to **very different/dense** layouts can underperform SOTA despite positive scaling; consider augmenting pre-training montage diversity and spatial encodings. |
| - **Distribution shifts:** Performance varies across cohorts, devices, and label protocols; validate locally and consider domain adaptation. |
|
|
| --- |
|
|
| ## 🏗️ Architecture & Training |
|
|
| **Spatiotemporal Tokenization.** The raw EEG recording is tokenized using a sliding window. A shared convolutional encoder projects each patch into a latent embedding. Spatiotemporal topology is preserved via an index-based matrix for temporal order and a continuous spatial embedding mapped via an MLP from 3D electrode coordinates. |
|
|
| **Windowed Temporal Attention (WTA).** WTA layers model temporal dependencies independently for each channel. Each query attends only to keys within a local neighborhood, avoiding memory accumulation. |
|
|
| **Windowed Spatial Attention (WSA).** WSA layers facilitate cross-channel information exchange at a fixed time index. Attention computation is restricted to a local neighborhood. Layer-specific dilation and shift factors stagger receptive fields to enable distant, physiologically related channels to interact. |
|
|
| **Pre-training objectives.** The model is pre-trained using Masked Autoencoding (MAE). The objective minimizes the weighted sum of reconstruction losses for masked and visible patch positions. |
|
|
| --- |
|
|
| ## 🔧 How to Use |
|
|
| S-CEReBrO is supported within the BioFoundation framework. Experiments and setups follow similar pipeline structures: |
|
|
| 1. **Install & read data prep**: clone the [BioFoundation repo](https://github.com/pulp-bio/biofoundation). Set up the environment as described there, then open `make_datasets/README.md` for dataset-specific notes. |
| 2. **Point to weights**: set `pretrained_safetensors_path` to the downloaded weights in your experiment YAML. |
| 3. **Preprocess data**: acquire fine-tuning dataset and follow preprocessing protocol to generate `train/test/val.lmdb` files. |
| 4. **Update data module config** |
| 5. **Task settings**: Configure `classification_type`, `model.num_classes`, and task modes inside your YAML config based on downstream objectives. |
| 6. **Launch fine-tuning (Hydra)**: |
|
|
| ```bash |
| python -u run_train.py +experiment=s_cerebro_finetune |
| ``` |
|
|
| --- |
|
|
| ## 🔗 Sources |
|
|
| - **Code:** https://github.com/pulp-bio/BioFoundation |
| - **Paper:** S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring (arxiv:2607.27913) |
|
|
| --- |
|
|
| ## 📜 Citation |
|
|
| If you use S-CEReBrO, please cite: |
|
|
| ```bibtex |
| @misc{bucagu2026scerebrobreakingmemorybarrier, |
| title={S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring}, |
| author={Glenn Anta Bucagu and Thorir Mar Ingolfsson and Yawei Li and Luca Benini}, |
| year={2026}, |
| eprint={2607.27913}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.LG}, |
| url={https://arxiv.org/abs/2607.27913}, |
| } |
| ``` |
|
|
| --- |
|
|
| ## 🛠️ Maintenance & Contact |
|
|
| - **Issues & support:** please open a GitHub issue in the BioFoundation repository. |
|
|
| --- |
|
|
| ## 🔗 Related Models |
|
|
| - **[LUNA](https://huggingface.co/PulpBio/LUNA)** — Transformer-based topology-agnostic EEG foundation model (NeurIPS 2025). Source of the channel-unification cross-attention module that LuMamba reuses. |
| - **[FEMBA](https://huggingface.co/PulpBio/FEMBA)** — Bidirectional Mamba foundation model for EEG. Source of the linear-complexity temporal backbone that LuMamba reuses. |
| - **[TinyMyo](https://huggingface.co/PulpBio/TinyMyo)** — Tiny foundation model for flexible EMG signal processing at the edge. |
| |
| ## 🗒️ Changelog |
|
|
| - **v1.0:** Initial release of LuMamba model card with task-specific checkpoints and instructions. |
