Update README.md
Browse files
README.md
CHANGED
|
@@ -34,6 +34,8 @@ The model's predicted embeddings capture 62 dimensions of meaningful variance (a
|
|
| 34 |
|
| 35 |
You will need a 16-channel [OpenBCI Cyton](https://shop.openbci.com/products/the-complete-headset-eeg?variant=44401726128368) device. The model may be compatible with other EEG devices; the autoencoder which forms the front-end of the model performs some work which should enhance cross-device capability, but the pipeline remains untested on non-OpenBCI devices.
|
| 36 |
|
|
|
|
|
|
|
| 37 |
Download the model weights, and this repository. Then create a retrieval index:
|
| 38 |
|
| 39 |
```bash
|
|
@@ -56,10 +58,19 @@ python3 morphism.py decode -f session.eeg \
|
|
| 56 |
|
| 57 |
All subcommands support `--help` for full option details.
|
| 58 |
|
| 59 |
-
You will see a real-time retrieval of documents from your index that the model thinks are relevant to you. Due to the fact that this model is a research preview, the model's output may be somewhat noisy. Relevance tends to vary between individuals with the model performing more strongly on people who are more similar to the patterns represented in the training data.
|
| 60 |
|
| 61 |
## How was this created?
|
| 62 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
This model works in two stages. The first stage is an autoencoder which represents the neural data in a latent space. The second stage is a semantic mapper, which guesses a semantic vector from the neural vector. This relatively simple architecture is surprisingly effective and lays the groundwork for future developments of this technology.
|
| 64 |
|
| 65 |
The underlying dataset represents a large collection of paired neural measurements and text stimuli, collected by Eve Labs over a period of 20 months on approximately forty subjects. Training data was gathered naturalistically; subjects chatted with LLMs while wearing the headset, with the conversation text serving as paired stimuli.
|
|
@@ -70,6 +81,18 @@ We believe that people should be able to understand and use the signals their ow
|
|
| 70 |
|
| 71 |
On a practical level, new models such as this can be evaluated and understood much faster if the weights are released. Given the complexity of the task at hand, we hope that people can report their experiences and help advance the field.
|
| 72 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
## How can I get involved?
|
| 74 |
|
| 75 |
Join the [Discord](TODO), follow [@evelabsai](https://twitter.com/evelabsai) on Twitter, or send over an email to [hello@evelabs.info](mailto:hello@evelabs.info).
|
|
|
|
| 34 |
|
| 35 |
You will need a 16-channel [OpenBCI Cyton](https://shop.openbci.com/products/the-complete-headset-eeg?variant=44401726128368) device. The model may be compatible with other EEG devices; the autoencoder which forms the front-end of the model performs some work which should enhance cross-device capability, but the pipeline remains untested on non-OpenBCI devices.
|
| 36 |
|
| 37 |
+
**Dependencies**: `pip install torch numpy faiss-cpu hnswlib transformers tqdm pyserial paramiko`
|
| 38 |
+
|
| 39 |
Download the model weights, and this repository. Then create a retrieval index:
|
| 40 |
|
| 41 |
```bash
|
|
|
|
| 58 |
|
| 59 |
All subcommands support `--help` for full option details.
|
| 60 |
|
| 61 |
+
You will see a real-time retrieval of documents from your index that the model thinks are relevant to you. Due to the fact that this model is a research preview, the model's output may be somewhat noisy. Relevance tends to vary between individuals with the model performing more strongly on people who are more similar to the patterns represented in the training data. The retrieved text is associative, not transcriptive β the model tracks coarse cognitive state rather than sentence-level content, surfacing thematically related passages from your corpus.
|
| 62 |
|
| 63 |
## How was this created?
|
| 64 |
|
| 65 |
+
```
|
| 66 |
+
raw EEG (16ch, 1kHz)
|
| 67 |
+
β windowed segments (~0.6s)
|
| 68 |
+
β convolutional autoencoder (64-dim bottleneck)
|
| 69 |
+
β semantic VAE (β 1024-dim text embedding space)
|
| 70 |
+
β FAISS nearest-neighbor search
|
| 71 |
+
β matching text from your corpus
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
This model works in two stages. The first stage is an autoencoder which represents the neural data in a latent space. The second stage is a semantic mapper, which guesses a semantic vector from the neural vector. This relatively simple architecture is surprisingly effective and lays the groundwork for future developments of this technology.
|
| 75 |
|
| 76 |
The underlying dataset represents a large collection of paired neural measurements and text stimuli, collected by Eve Labs over a period of 20 months on approximately forty subjects. Training data was gathered naturalistically; subjects chatted with LLMs while wearing the headset, with the conversation text serving as paired stimuli.
|
|
|
|
| 81 |
|
| 82 |
On a practical level, new models such as this can be evaluated and understood much faster if the weights are released. Given the complexity of the task at hand, we hope that people can report their experiences and help advance the field.
|
| 83 |
|
| 84 |
+
## Project structure
|
| 85 |
+
|
| 86 |
+
```
|
| 87 |
+
morphism
|
| 88 |
+
βββ morphism.py # CLI entrypoint
|
| 89 |
+
βββ cyton.py # OpenBCI Cyton+Daisy recording
|
| 90 |
+
βββ embed.py # Text embedding + SQLite storage
|
| 91 |
+
βββ decode.py # EEG β semantic embedding β FAISS search
|
| 92 |
+
βββ eegembed.py # EEG autoencoder streaming
|
| 93 |
+
βββ README.md
|
| 94 |
+
```
|
| 95 |
+
|
| 96 |
## How can I get involved?
|
| 97 |
|
| 98 |
Join the [Discord](TODO), follow [@evelabsai](https://twitter.com/evelabsai) on Twitter, or send over an email to [hello@evelabs.info](mailto:hello@evelabs.info).
|