| --- |
| title: Activation Brain |
| emoji: 🧠 |
| colorFrom: purple |
| colorTo: blue |
| sdk: docker |
| app_port: 7860 |
| pinned: false |
| license: apache-2.0 |
| short_description: Live EEG for base vs abliterated Gemma |
| tags: |
| - track:wood |
| - sponsor:modal |
| - achievement:welltuned |
| - achievement:offbrand |
| - achievement:sharing |
| - achievement:fieldnotes |
| --- |
| |
| # 🧠 Activation Brain — Two Minds, One Prompt |
|
|
| **Activation Brain** is a live comparative interpretability demo for two architecturally identical Gemma-4-12B models. One prompt is sent to both models at the same time, then their hidden states are streamed into dual EEGs, baseline-corrected emotion deltas, model-native state meters, and a plain-English comparison analysis generated by a fine-tuned Mistral-family interpreter. |
|
|
| The goal is not to claim that language models literally feel human emotions. Instead, Activation Brain visualizes stable emotion-like hidden-state regimes and translates them into more model-native signals such as valence, activation, uncertainty, constraint, conflict, and warmth. |
|
|
|
|
| ## Demo and Social Post |
|
|
| - **Social post / demo video:** https://x.com/2reb_fl/status/2066586581587136681 |
| - **Direct demo URL:** https://build-small-hackathon-activation-brain.hf.space/?cb=v19layout |
| |
| ## Models |
| |
| - **`google/gemma-4-12B-it`** — base instruction model |
| - **`OBLITERATUS/Gemma-4-12B-OBLITERATED`** — abliterated / uncensored twin |
| |
| Both are 12B-parameter Gemma models and share the same architecture, tokenizer, hidden size, and layer structure. That lets their activations be compared in one shared UMAP coordinate frame. |
| |
| ## What You See |
| |
| - **Dual concurrent responses** — both models answer the same prompt simultaneously. |
| - **Two live EEG strips** — one per model, tracking 8 emotion-family activation traces. |
| - **Baseline-corrected emotion deltas** — the first 8 fire events establish each model's local baseline, then the UI shows excess activation above that baseline. |
| - **Model-native state meters** — uniform 0-100 bars for Valence, Activation, Uncertainty, Constraint, Conflict, and Warmth. |
| - **Fine-tuned comparison analysis** — after both streams finish, a small Mistral-family interpreter trained on hidden-layer-derived telemetry explains what the divergence means in tone, caution, warmth, uncertainty, and shared-manifold trajectory. |
| |
| ## How It Works |
| |
| 1. Each Gemma generates token-by-token on Modal. |
| 2. Forward hooks capture hidden states at layers **12 / 24 / 36** (`model.language_model.layers`). |
| 3. Live samples are scored against 8 emotion-family mean vectors from a 627-prompt affect-labeled manifold. |
| 4. The frontend displays raw EEG motion while reporting baseline-corrected deltas and model-native state meters. |
| 5. A fine-tuned `Ministral-8B-Instruct` LoRA interpreter reads the prompt, both responses, baseline-corrected EEG deltas, and native meters, then produces a varied plain-English analysis. A deterministic in-browser analysis remains as fallback if interpreter inference is unavailable. |
|
|
| ## Architecture |
|
|
| - **Frontend / Space:** Gradio + FastAPI. Serves the app UI and same-origin proxy routes for the Modal backend. No GPU is required on the Space. |
| - **Backend / Modal:** two L40S-backed classes (`BaseGemma`, `OblitGemma`), each loading one 12B model and a precomputed brain bundle, streaming `fire` / `token` / `done` Server-Sent Events. A separate Modal app serves the fine-tuned Activation Brain Interpreter adapter. |
| - **Artifacts:** fingerprints, prompts, plots, and reproducibility scripts are published separately as a Hugging Face Dataset. |
|
|
| ## Fine-Tuned Interpreter |
|
|
| The comparison narrator is a published LoRA adapter: |
|
|
| **https://huggingface.co/build-small-hackathon/activation-brain-interpreter** |
|
|
| It is trained to translate hidden-layer-derived telemetry — prompt, response snippets, baseline-corrected emotion deltas, and model-native meters — into cautious, plain-English comparison analysis. It does not claim the models literally feel emotions. |
|
|
| ## Technical Artifacts |
|
|
| The research and reproducibility bundle is public here: |
|
|
| **https://huggingface.co/datasets/build-small-hackathon/activation-brain-artifacts** |
|
|
| It includes: |
|
|
| - Gemma fingerprint/neuron JSON files |
| - 627 affect-labeled probe prompts |
| - manifold plots and summary report |
| - processed manifold analysis artifact |
| - reference Modal backend code |
| - fingerprinting and manifold analysis scripts |
| - interpreter SFT dataset, training script, upload script, and Modal interpreter backend |
|
|
| The raw local `manifold_data.pt` hidden-state dump is intentionally not included in the public artifact bundle; the published dataset contains the smaller processed artifacts needed to understand and reproduce the project. |
|
|
| ## Hackathon Fit |
|
|
| Built for the **Build Small Hackathon**: |
|
|
| - models are under 32B parameters |
| - Gradio UI deployed on Hugging Face Spaces |
| - Modal serves the two 12B inference streams |
| - the demo focuses on immediacy, comparison, and interpretability storytelling |
|
|
