SSp1ash commited on
Commit
6fa5b2e
·
verified ·
1 Parent(s): ba3794e

Improve model card with usage and provenance

Browse files
Files changed (1) hide show
  1. README.md +96 -0
README.md CHANGED
@@ -1,3 +1,99 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ tags:
4
+ - brain-decoding
5
+ - eeg
6
+ - fmri
7
+ - image-reconstruction
8
+ - language-decoding
9
+ - neuroscience
10
+ library_name: pytorch
11
  ---
12
+
13
+ # Eigenbrain: frozen brain-decoding inference bundle
14
+
15
+ This repository contains frozen inference assets for three independent
16
+ brain-decoding demonstrations: EEG classification, neural language decoding,
17
+ and NSD Subject-1 fMRI-to-image reconstruction. It is an inference/replay
18
+ bundle; it does not train a model and does not fit preprocessing transforms.
19
+
20
+ ## Contents
21
+
22
+ | Task | Input | Frozen route | Main output |
23
+ |---|---|---|---|
24
+ | EEG classification | SEED-V eigenmode EEG | temporal/mode-attention encoder + 5-class head | predicted class |
25
+ | Language decoding | Alice BCI observation and prior | PoE + subject layer + QFormer + BGE tokens + Phi-4 LoRA | decoded English text |
26
+ | Image reconstruction | NSD Subject-1 CIFTI-4096 and modes-2000 | anchored PoE + MindEye2 BrainNetwork + diffusion prior + SDXL-unCLIP | reconstructed PNG |
27
+
28
+ The language route uses the public base models `microsoft/Phi-4-mini-instruct`
29
+ and `BAAI/bge-m3`; the files in this repository are the project-specific
30
+ alignment, projector, and LoRA weights. The image route uses a large frozen
31
+ unCLIP checkpoint and requires a CUDA GPU with substantial storage and memory.
32
+
33
+ ## Download
34
+
35
+ Download this repository together with the companion inference code. The
36
+ project code expects the following local layout:
37
+
38
+ ```text
39
+ assets/
40
+ weights/
41
+ weights/alice_lora/
42
+ weights/nsd_sub01/
43
+ models/Phi-4-mini-instruct/
44
+ models/bge-m3/
45
+ ```
46
+
47
+ The companion `download_assets.py` downloads the project files from this
48
+ repository and the two public base models:
49
+
50
+ ```bash
51
+ export EIGEN_HF_REPO=SSp1ash/Eigenbrain
52
+ python download_assets.py
53
+ ```
54
+
55
+ For classification and language only, omit the large NSD image files with
56
+ `python download_assets.py --skip-nsd`.
57
+
58
+ ## Run
59
+
60
+ ```bash
61
+ python run_demo.py --task classification --device cuda
62
+
63
+ python run_demo.py --task language --device cuda \
64
+ --phi-path models/Phi-4-mini-instruct \
65
+ --bge-path models/bge-m3
66
+
67
+ python run_demo.py --task image --device cuda
68
+ ```
69
+
70
+ The image command evaluates the default ten held-out NSD examples and writes
71
+ the selected qualitative reconstructions and image-level metrics under
72
+ `outputs/nsd_sub01/`.
73
+
74
+ ## Reference results
75
+
76
+ The fixed compact replay gives approximately 40% accuracy for EEG
77
+ classification on 10 examples. The language text route gives a mean BGE
78
+ sentence cosine of approximately 0.81 on 5 examples. The NSD image route has
79
+ an archived full-test CLIP 2-way identification score of 0.783; the default
80
+ ten-example smoke test is not a replacement for the full benchmark.
81
+
82
+ These values describe the shipped frozen replay assets and should not be
83
+ interpreted as a new independent held-out evaluation. The language alignment
84
+ checkpoint includes historical oracle-style provenance, and the NSD visual
85
+ demo saves only a small qualitative subset of the evaluated examples.
86
+
87
+ ## Files
88
+
89
+ Custom project files are stored under `assets/` and `weights/`. The largest
90
+ file is `weights/nsd_sub01/unclip6_epoch0_step110000.ckpt` (about 18 GB).
91
+ Public base models are intentionally not duplicated here.
92
+
93
+ ## License and limitations
94
+
95
+ The code and frozen assets are provided for research demonstration and
96
+ reproducibility. Check the licenses and terms of the public Phi-4, BGE-M3,
97
+ MindEye2/SDXL-unCLIP components and the NSD data before redistribution or
98
+ commercial use. The NSD route is within-subject reconstruction for Subject 1,
99
+ not a cross-subject general-purpose image decoder.