Autara-Vox: Dual-Stream 4-Center Cortical-Articulatory EEG-to-Speech Model
Autara Research Labs
Autara-Vox is a standalone non-invasive EEG-to-Speech neural architecture and IEEE Transactions manuscript (paper/autara_vox_paper.pdf) that explicitly disentangles:
- Pure Cortical EEG Oscillations (
1β45 Hz) parcellated across 4 neuroanatomical brain centers (Speech/Broca's, Auditory/Wernicke's, Visual/Occipital, Memory/Prefrontal). - Subvocal Jaw & Facial Articulatory EMG (
50β200 Hz) with a leak-free gating switch (dual | pure_eeg | jaw_only). - Mandatory Every-10-Epoch Live Verification Probes paired with Gaussian Random-Noise EEG Controls (
paper/results/*_probe_ep*.json).
Headline Verified Results (14,208 Synchronized OpenNeuro Trials)
- Exp 0 β Verbal Working Memory (
ds003655, Unseen Subjects):63.1%accuracy (+16.8%Macro-F1 over Random-Noise Control). - Exp 1 β Inner / Pronounced+Jaw / Visualized Speech (
ds003626, Unseen Session 3):- Dual-Stream + Memory Transfer:
69.1%3-modality accuracy (73.7%Macro-F1,+42.9%over noise control), beating both Pure EEG (55.3%) and Jaw/EXG (66.6%). - Pronounced+Jaw 4-Word Accuracy:
68.3%(2.7Γchance). - 4-Center Cortical Synergy (Pure EEG Only): All 4 centers (
55.3%) outperform every individual center alone (Visual 45.5%,Auditory 45.1%,Speech 41.0%,Memory 37.6%).
- Dual-Stream + Memory Transfer:
- Exp 2 β Open-Loop Continuous Speech Decoding (
ds004408, Zero Teacher Forcing):- Learned 4-center gate autonomously routed
96.4%β99.7%to the Auditory Center during continuous speech listening. - Best retrieval checkpoint:
Top-5 = 3.66%(+1.10%vs. noise),BLEU-1 = 9.92,ROUGE-L = 11.14; Epoch 30 Pure-EEG:BLEU-1 = 11.48,ROUGE-L = 11.77,PER = 88.74%, diversity18.75%(3.6Γnoise control).
- Learned 4-center gate autonomously routed
Model Weights (Hugging Face)
Trained checkpoints are hosted on Hugging Face at autara-ai/Autara-Vox:
checkpoints/autara_vox_speech_best.ptβ Continuous Speech Decoder (AutaraVoxSpeechDecoder+ BART-base adapter)checkpoints/exp1_inner_mem_transfer_best.ptβ Dual-Stream Inner/Pronounced/Visualized Classifier (Memory-pretrained)checkpoints/exp1_inner_scratch_best.ptβ Dual-Stream Inner/Pronounced/Visualized Classifier (Scratch)checkpoints/memory_pretrained_backbone.ptβ Verbal Working Memory 4-Center Pretrained Backbone
Quick Start
# Install dependencies via uv
uv venv --python 3.12
uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
uv pip install -e . pytest
# 1. Run unit tests
.venv/bin/pytest -v tests/test_autara_vox.py
# 2. Build 4-center OpenNeuro datasets (ds003626, ds004408, ds003655)
.venv/bin/python scripts/download_and_build_datasets.py --workers 8 --ds4408-subs 8 --ds4408-runs 10 --ds3655-subs 12
# 3. Train Exp 0 (Memory) + Exp 1 (Inner/Jaw Speech) with 10-epoch live probes
.venv/bin/python scripts/train_inner_speech.py --epochs-mem 30 --epochs-speech 30
# 4. Train Exp 2 (Continuous Speech Decoder) with 10-epoch live probes
.venv/bin/python scripts/train_speech_decoder.py --epochs 30 --batch-size 32
# 5. Live End-to-End EEG-to-Speech Demo & Publication Figures
.venv/bin/python scripts/demo_thought_to_speech.py
.venv/bin/python scripts/generate_paper_figures.py
Inference Providers NEW
This model isn't deployed by any Inference Provider. π Ask for provider support