Post
37
New write-up: when should an EEG decoder abstain?
Motivated by Jev-style typed decision interfaces; independent evaluation, not affiliated with TypeSafe, AutoTrust, ggml-org or the authors of any Jev paper.
On BETA (70 people, 40-target SSVEP, cue-paced lab trials), one calibrated threshold — accept when the top-class probability is at least 0.8 — accepted very different shares of trials: standard CCA 28.4% (95% interval 22.9–34.2%), EEGNet 16.6% (12.5–21.0%), frozen CBraMod 1.5% (0.8–2.3%), with nothing accepted at all for 38 of the 70 people under CBraMod. A learned reject option did not beat the model's own calibrated confidence, and a certified error level did not reliably hold on new people. These compare methods; they are not deployment error rates.
Article: https://huggingface.co/blog/Twu31/when-should-an-eeg-decoder-abstain
Browse or query the published figures (UI + MCP): Twu31/bci-report-explorer
Collection: Twu31/bci-report-6ac6fc80f1026c08d6cac752
BCI Report is a personal, noncommercial project; aggregate results only.
Motivated by Jev-style typed decision interfaces; independent evaluation, not affiliated with TypeSafe, AutoTrust, ggml-org or the authors of any Jev paper.
On BETA (70 people, 40-target SSVEP, cue-paced lab trials), one calibrated threshold — accept when the top-class probability is at least 0.8 — accepted very different shares of trials: standard CCA 28.4% (95% interval 22.9–34.2%), EEGNet 16.6% (12.5–21.0%), frozen CBraMod 1.5% (0.8–2.3%), with nothing accepted at all for 38 of the 70 people under CBraMod. A learned reject option did not beat the model's own calibrated confidence, and a certified error level did not reliably hold on new people. These compare methods; they are not deployment error rates.
Article: https://huggingface.co/blog/Twu31/when-should-an-eeg-decoder-abstain
Browse or query the published figures (UI + MCP): Twu31/bci-report-explorer
Collection: Twu31/bci-report-6ac6fc80f1026c08d6cac752
BCI Report is a personal, noncommercial project; aggregate results only.