README.md

---
license: mit
---

# Calcium-Bridged Temporal EEG Decoder (Vibe coded amateur stuff)

At github: https://github.com/anttiluode/CalciumBridgeEEGConstraintViewer/tree/main

![GUI](./pic.png)

This project explores the idea of decoding EEG brain signals by modeling perception as a sequential process. Instead of treating the brain's response as a single event, this system breaks it down into distinct temporal windows, attempting to model the "chain of thought" as a visual concept crystallizes in the mind.

The project consists of two main components:
1.  **A trainer (`pkas_cal_trainer_gemini.py`)** that builds a novel neural network model using the **Alljoined1 dataset**.
2.  **A viewer (`pkas_cal_viewer_gemini2.py`)** that loads a trained model and provides an interactive visualization of its "thought process" on new EEG samples.

## Core Concept: The "Vibecoded" System

The central idea of this project is a system inspired by neuromorphic computing and constraint satisfaction, which we've nicknamed the "vibecoded" system.

Here’s how it works simply:

1.  **Thinking in Moments:** The brain's response to an image (e.g., from 0 to 600ms) is not analyzed all at once. It's sliced into four distinct "thinking moments" or time windows based on known ERP components.

2.  **A Solver for Each Moment:** Each time window is processed by a special `CalciumAttentionModule`. This module's job is to look at the EEG clues in its slice and find the best explanation that satisfies all the "constraints" in the signal.

3.  **The Calcium Bridge:** This is the key. The "hunch" or "focus" (`Calcium` state) from one thinking moment is passed to the next. This creates a causal chain of thought, allowing the model to refine its predictions over time from a general gist to a more specific concept.

## A Note on Data Filtering to Reduce Bias

A key challenge when decoding brain signals from natural images is that certain concepts are omnipresent. For example, `person` appears in a vast number of images in the COCO dataset.

To prevent the model from simply learning to predict these common categories and to create a more focused decoding task, this project intentionally filters the data:

1.  **Selective Training:** The model is only trained to recognize a specific subset of 26 object categories (defined in the code as `TARGET_CATEGORIES`). Common but potentially confounding categories like `person`, `cell phone`, or `book` were deliberately excluded from this training set.

2.  **Clean Visualization:** The viewer script (`pkas_cal_viewer_gemini2.py`) performs an additional, stricter filtering step. It only selects test images that contain **at least one** of the target objects and **zero** of the excluded objects.

This ensures that when you visualize the model's performance, you are seeing its attempt to decode distinct object concepts, rather than it relying on the statistical likelihood of a person being in the scene.

The thought was that a person computer etc are present in the lab where the alljoined data was recorded and may bias the results. 

## Requirements
- Python 3.x
- PyTorch
- `datasets` (from Hugging Face)
- `tkinter` (usually included with Python)
- `matplotlib`
- `pillow`

You can install the main dependencies with pip:
pip install torch datasets matplotlib pillow
code
Code
## Setup and Usage

### 1. Download Data and Model

**Data:**
-   **COCO Images:** Download the 2017 training/validation images from the [COCO Dataset official site](https://cocodataset.org/#download). You will need `train2017.zip` and/or `val2017.zip`. Unzip them into a known directory.
-   **COCO Annotations:** On the same site, download the "2017 Train/Val annotations". You only need the `instances_train2017.json` file.
-   **Alljoined1 EEG Data:** This will be downloaded automatically by the scripts on their first run.

**Pre-trained Model (Recommended):**
- You can download the pre-trained V2 model directly from its [Hugging Face Repository](https://huggingface.co/Aluode/CalciumBridgeEEGConstraintViewer/tree/main). Click on `calcium_bridge_eeg_model_v2.pth` and then click the "download" button.

### 2. Viewing the Results (Using the Pre-trained Model)

1.  Run the V2 viewer script:
    
    python pkas_cal_viewer_gemini2.py
    
2.  In the GUI:
    -   Select the COCO image and annotation paths you downloaded.
    -   Click **"Load V2 Model"** and select the `calcium_bridge_eeg_model_v2.pth` file you downloaded from Hugging Face.
3.  Once the model is loaded, click **"Test Random Sample"** to see the model's analysis of a new brain signal.

### 3. Training Your Own Model (Optional)

1.  Run the V2 training script:


    python pkas_cal_trainer_gemini.py
    
3.  In the GUI, select your COCO image and annotation paths.
4.  Click **"Train Extended Model (V2)"**.
5.  A new file named `calcium_bridge_eeg_model_v2.pth` will be saved with the best-performing model from your training run. You can then load this file into the viewer.

## A Note on Interpretation

This is an exploratory research tool. The model's predictions should **not** be interpreted as literal "mind-reading."

Instead, the results reflect the complex **statistical associations** learned from the multi-subject `Alljoined` dataset. When the model associates a "horse trailer" with "horse," it is because this is a strong, common conceptual link found in the aggregate brain data. The viewer is a window into the "cognitive gestalt" of an "average mind" as represented by the dataset.