File size: 3,321 Bytes
33ac086 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 | ---
license: mit
library_name: pytorch
tags:
- state-space-model
- sparse-attention
- episodic-memory
- long-context
- interpretability
language:
- en
---
# SynapNet — Episodic Memory Checkpoints
Hybrid **SSM + sparse-attention + episodic-memory** architecture. This repo holds the base architectural checkpoints; the edge-deployment variants live at [`Vineetha00/synapnet-edge`](https://huggingface.co/Vineetha00/synapnet-edge).
📦 **Code:** https://github.com/vineetha00/SynapNet_Exp
🛠️ **Deployment companion:** https://github.com/vineetha00/SynapNet-Edge · 🤗 https://huggingface.co/Vineetha00/synapnet-edge
---
## Checkpoints in this repo
| File | Variant | Size | Notes |
|---|---|---|---|
| [`synapnet_memory_v4.pt`](synapnet_memory_v4.pt) | Track 3 v4 (current best) | 5.7 MB | Trained with salience supervision λ=0.01; near-perfect episodic recall |
| [`synapnet_memory_v2.pt`](synapnet_memory_v2.pt) | Track 3 v2 | 6.8 MB | Earlier episodic-recall checkpoint |
| [`synapnet_memory.pt`](synapnet_memory.pt) | Original episodic memory | 18.2 MB | Initial release of the WriteableMemory variant |
---
## Architecture (Track 3 v4 — recommended)
- `SimpleSSM` (depthwise conv, kernel_size=9) for local temporal dynamics
- `SparseEventAttention` (salience-gated top-K mixing)
- `WriteableMemory` (top-K hidden states written to a fixed-size bank, read via cross-attention)
- Gated fusion (α, β) over the three pathways
- 4 stacked blocks, dim=128, heads=4
---
## Key finding (λ-sweep, 5 seeds, ctx=2048, 32-class recall)
A small dose of salience supervision flips episodic recall from chance to near-perfect:
| λ (salience-supervision strength) | Accuracy | Write hit-rate (target token written?) |
|---|---|---|
| 0.00 | 0.723 ± 0.329 | 0.125 ± 0.109 |
| **0.01** | **0.968 ± 0.022** | **0.993 ± 0.002** |
| 0.10 | 0.970 ± 0.021 | 0.995 ± 0.001 |
| 1.00 | 0.838 ± 0.125 | 0.995 ± 0.002 |
The v4 checkpoint was trained at the sweet-spot λ=0.01.
---
## Loading
```python
import torch
from huggingface_hub import hf_hub_download
# Make sure SynapNet_Exp is on your path:
# git clone https://github.com/vineetha00/SynapNet_Exp
# cd SynapNet_Exp
import sys; sys.path.insert(0, "/path/to/SynapNet_Exp")
from synapnet_memory import SynapEpisodicNet
ckpt_path = hf_hub_download(
repo_id="Vineetha00/synapnet",
filename="synapnet_memory_v4.pt",
)
state = torch.load(ckpt_path, map_location="cpu")
model = SynapEpisodicNet(
dim=128, depth=4, vocab_size=2048,
max_len=2048, num_classes=32, heads=4,
)
model.load_state_dict(state)
model.eval()
```
---
## Training tasks (from the companion code)
- **Track 1** — LRA-style long-range classification (`train_track1_lra.py`)
- **Track 2** — biosignal regression / ECG reconstruction (`train_track2_biosignal.py`)
- **Track 3** — episodic recall (`train_track3_memory_v2/v3/v4.py`)
- **Track 4** — faithful-salience supervision for interpretability (`train_track4_interpret.py`)
---
## License
MIT — see [LICENSE](https://github.com/vineetha00/SynapNet_Exp/blob/main/LICENSE).
## Citation
```bibtex
@article{synapnet_2026,
title={SynapNet: Hybrid SSM + Sparse-Attention + Episodic Memory for Long-Range Sequence Modelling},
author={Vallish Kumar, Vineetha},
year={2026},
}
```
|