Update README.md

c6ec190 verified 1 day ago

13 kB

license: cc-by-nd-4.0
language:
  - en
library_name: pytorch
tags:
  - eeg
  - biosignal
  - foundation-model
  - self-supervised
  - masked-modeling
  - windowed-alternating-attention
  - neuroscience
datasets:
  - TUEG
  - SEED
  - BOAS
  - SleepEDFx
  - BCI-NER
  - GWD
  - CHB-MIT
  - Neonate
  - PhysioNet-MI
  - SHU-MI
  - TUAB
  - STEW
  - SEED-VIG
  - ISRUC
  - Mumtaz
  - MentalArithmetic
  - SEED-V
metrics:
  - roc_auc
  - f1
  - nrmse
model-index:
  - name: S-CEReBrO
    results:
      - task:
          type: time-series-classification
          name: Seizure Detection
        dataset:
          type: CHB-MIT
          name: CHB-MIT
        metrics:
          - type: roc_auc
            value: 87.45
            name: AUROC
      - task:
          type: time-series-classification
          name: Seizure Detection
        dataset:
          type: Neonate
          name: Neonate
        metrics:
          - type: roc_auc
            value: 85.76
            name: AUROC
      - task:
          type: time-series-classification
          name: Motor Imagery Classification
        dataset:
          type: PhysioNet-MI
          name: PhysioNet-MI
        metrics:
          - type: f1
            value: 60.93
            name: Weighted F1
      - task:
          type: time-series-classification
          name: Motor Imagery Classification
        dataset:
          type: SHU-MI
          name: SHU-MI
        metrics:
          - type: roc_auc
            value: 66.48
            name: AUROC
      - task:
          type: time-series-classification
          name: Abnormal Classification
        dataset:
          type: TUAB
          name: TUAB
        metrics:
          - type: roc_auc
            value: 89.3
            name: AUROC
      - task:
          type: time-series-classification
          name: Mental Workload Classification
        dataset:
          type: STEW
          name: STEW
        metrics:
          - type: f1
            value: 52.9
            name: Weighted F1
      - task:
          type: time-series-regression
          name: Vigilance Estimation
        dataset:
          type: SEED-VIG
          name: SEED-VIG
        metrics:
          - type: nrmse
            value: 0.93
            name: NRMSE
      - task:
          type: time-series-classification
          name: Sleep Stage Classification
        dataset:
          type: ISRUC
          name: ISRUC
        metrics:
          - type: f1
            value: 78.23
            name: Weighted F1
      - task:
          type: time-series-classification
          name: Mental Disorder Diagnosis
        dataset:
          type: Mumtaz
          name: Mumtaz
        metrics:
          - type: roc_auc
            value: 98.23
            name: AUROC
      - task:
          type: time-series-classification
          name: Mental Stress Detection
        dataset:
          type: MentalArithmetic
          name: MentalArithmetic
        metrics:
          - type: roc_auc
            value: 68.97
            name: AUROC
      - task:
          type: time-series-classification
          name: Emotion Recognition
        dataset:
          type: SEED-V
          name: SEED-V
        metrics:
          - type: f1
            value: 28.16
            name: Weighted F1

S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring

S-CEReBrO (Streaming CEReBrO) is an EEG foundation model designed for continuous monitoring. S-CEReBrO addresses the memory overflow bottleneck of continuous monitoring by utilizing a novel Windowed Alternating Attention mechanism. This factorizes attention computation into fixed-size spatiotemporal windows, guaranteeing a constant KV cache memory as only the active window requires resident attention maps.

🔒 License & Usage Policy (Weights)

Weights license: The released model weights are licensed under Creative Commons Attribution–NoDerivatives 4.0 (CC BY-ND 4.0). This section summarizes the practical implications for users. This is not legal advice; please read the full license text.

✅ You may

Use and redistribute the unmodified S-CEReBrO weights (including in commercial settings) with proper attribution to the S-CEReBrO authors.
Fine-tune / adapt the weights for your internal use (research or production) without redistributing the modified weights.
Publish your code, configs, logs, and papers describing experiments with S-CEReBrO (please cite the paper).

🚫 You may not

Share, host, or redistribute any modified weights (including LoRA/adapter/delta checkpoints or pruned/quantized variants). Any parameter set that encodes an adaptation is considered a derivative and cannot be shared under CC BY-ND 4.0.
Imply endorsement by the S-CEReBrO authors for any derivative or evaluation without our written permission.
Use the S-CEReBrO name in a way that suggests your modified model is an official S-CEReBrO release.

🤝 How to contribute improvements (PR-gated releases)

We welcome community improvements via a pull-request (PR) workflow. If you believe your improvements should become an official S-CEReBrO release:

Open a PR in the BioFoundation repository describing the change (architecture/head/training recipe, datasets, preprocessing, compute).
Include reproducibility artifacts: configs, seeds, scripts, environment details, training/validation logs, and the evaluation protocol with exact splits.
Provide comprehensive results (AUROC/AUPR/BA, FLOPs, memory) vs. the baselines reported in the S-CEReBrO paper.
After maintainer review, approved changes will be retrained/validated and, if accepted, released by the maintainers as a new official S-CEReBrO checkpoint under CC BY-ND 4.0.

Rationale: CC BY-ND protects users from fragmented, lower-quality variants, while still enabling internal fine-tuning and a path for the community to upstream improvements through review.

🔎 Model Summary

Goal: Realize efficient, generalizable, and continuous EEG monitoring while breaking the memory barrier of Transformer-based architectures.
Core idea: Windowed Alternating Attention interleaves spatial and temporal attention blocks. This restricts interactions to local neighborhoods in time and space, allowing the model to process long signals while maintaining a constant resident memory. Complexity is $\mathcal{O}(CT)$.
Pre-training data: >25,000 hours of EEG from >12,000 subjects. Includes TUEG, SEED series, BOAS, SleepEDFx, BCI-NER, and GWD. Downstream subjects (e.g. TUAB) are strictly excluded to prevent leakage.
Downstream tasks: Seizure detection (CHB-MIT, Neonate), Motor Imagery (PhysioNet-MI, SHU-MI), Abnormal classification (TUAB), Mental Workload (STEW), Vigilance Estimation (SEED-VIG), Sleep Staging (ISRUC), Mental Disorder Diagnosis (Mumtaz), Mental Stress (Mental Arithmetic), and Emotion Recognition (SEED-V).

🚀 Model Variants

S-CEReBrO is highly compact, utilizing only 2.4M parameters. This design allows it to achieve state-of-the-art performance with up to 60% fewer parameters than existing foundation models.

📊 Results

S-CEReBrO achieves state-of-the-art performance on 7 of 11 downstream public benchmarks.

CHB-MIT (seizure detection): 87.45 AUROC.
Neonate (seizure detection): 85.76 AUROC.
PhysioNet-MI (motor imagery): 60.93 Weighted F1.
SHU-MI (motor imagery): 66.48 AUROC.
TUAB (abnormal classification): 89.30 AUROC.
STEW (mental workload): 52.90 Weighted F1.
SEED-VIG (vigilance estimation): 0.93 NRMSE.
ISRUC (sleep staging): 78.23 Weighted F1.
Mumtaz (mental disorder diagnosis): 98.23 AUROC.
Mental Arithmetic (mental stress detection): 68.97 AUROC.
SEED-V (emotion recognition): 28.16 Weighted F1.

Efficiency & Scaling: Without low-level optimzations, windowed Alternating Attention can process signals 100x longer than full self-attention (up to 14 hours / 50,000 seconds). It also processes signals 3x longer than low-rank linear attention. Compared to low-rank linear attention on long contexts (15,000 seconds), it requires 55% of the memory while increasing inference throughput by 2.1x.

🧠 Intended Use & Limitations

Intended use. Research on continuous EEG representation learning and classification (clinical diagnostics, BCI, state monitoring) where long context and memory efficiency are necessary.

Limitations.

Not a medical device. Do not use for clinical decisions without proper validation & regulatory clearance.
Unseen topologies: Zero-shot transfer to very different/dense layouts can underperform SOTA despite positive scaling; consider augmenting pre-training montage diversity and spatial encodings.
Distribution shifts: Performance varies across cohorts, devices, and label protocols; validate locally and consider domain adaptation.

🏗️ Architecture & Training

Spatiotemporal Tokenization. The raw EEG recording is tokenized using a sliding window. A shared convolutional encoder projects each patch into a latent embedding. Spatiotemporal topology is preserved via an index-based matrix for temporal order and a continuous spatial embedding mapped via an MLP from 3D electrode coordinates.

Windowed Temporal Attention (WTA). WTA layers model temporal dependencies independently for each channel. Each query attends only to keys within a local neighborhood, avoiding memory accumulation.

Windowed Spatial Attention (WSA). WSA layers facilitate cross-channel information exchange at a fixed time index. Attention computation is restricted to a local neighborhood. Layer-specific dilation and shift factors stagger receptive fields to enable distant, physiologically related channels to interact.

Pre-training objectives. The model is pre-trained using Masked Autoencoding (MAE). The objective minimizes the weighted sum of reconstruction losses for masked and visible patch positions.

🔧 How to Use

S-CEReBrO is supported within the BioFoundation framework. Experiments and setups follow similar pipeline structures:

Install & read data prep: clone the BioFoundation repo. Set up the environment as described there, then open make_datasets/README.md for dataset-specific notes.
Point to weights: set pretrained_safetensors_path to the downloaded weights in your experiment YAML.
Preprocess data: acquire fine-tuning dataset and follow preprocessing protocol to generate train/test/val.lmdb files.
Update data module config
Task settings: Configure classification_type, model.num_classes, and task modes inside your YAML config based on downstream objectives.
Launch fine-tuning (Hydra):

python -u run_train.py +experiment=s_cerebro_finetune

🔗 Sources

Code: https://github.com/pulp-bio/BioFoundation
Paper: S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring (arxiv:2607.27913)

📜 Citation

If you use S-CEReBrO, please cite:

@misc{bucagu2026scerebrobreakingmemorybarrier,
      title={S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring}, 
      author={Glenn Anta Bucagu and Thorir Mar Ingolfsson and Yawei Li and Luca Benini},
      year={2026},
      eprint={2607.27913},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2607.27913}, 
}

🛠️ Maintenance & Contact

Issues & support: please open a GitHub issue in the BioFoundation repository.

🔗 Related Models

LUNA — Transformer-based topology-agnostic EEG foundation model (NeurIPS 2025). Source of the channel-unification cross-attention module that LuMamba reuses.
FEMBA — Bidirectional Mamba foundation model for EEG. Source of the linear-complexity temporal backbone that LuMamba reuses.
TinyMyo — Tiny foundation model for flexible EMG signal processing at the edge.

🗒️ Changelog

v1.0: Initial release of LuMamba model card with task-specific checkpoints and instructions.