---
license: mit
library_name: pytorch
tags:
  - state-space-model
  - sparse-attention
  - episodic-memory
  - long-context
  - interpretability
language:
  - en
---

# SynapNet — Episodic Memory Checkpoints

Hybrid **SSM + sparse-attention + episodic-memory** architecture. This repo holds the base architectural checkpoints; the edge-deployment variants live at [`Vineetha00/synapnet-edge`](https://huggingface.co/Vineetha00/synapnet-edge).

📦 **Code:** https://github.com/vineetha00/SynapNet_Exp
🛠️ **Deployment companion:** https://github.com/vineetha00/SynapNet-Edge · 🤗 https://huggingface.co/Vineetha00/synapnet-edge

---

## Checkpoints in this repo

| File | Variant | Size | Notes |
|---|---|---|---|
| [`synapnet_memory_v4.pt`](synapnet_memory_v4.pt) | Track 3 v4 (current best) | 5.7 MB | Trained with salience supervision λ=0.01; near-perfect episodic recall |
| [`synapnet_memory_v2.pt`](synapnet_memory_v2.pt) | Track 3 v2 | 6.8 MB | Earlier episodic-recall checkpoint |
| [`synapnet_memory.pt`](synapnet_memory.pt) | Original episodic memory | 18.2 MB | Initial release of the WriteableMemory variant |

---

## Architecture (Track 3 v4 — recommended)

- `SimpleSSM` (depthwise conv, kernel_size=9) for local temporal dynamics
- `SparseEventAttention` (salience-gated top-K mixing)
- `WriteableMemory` (top-K hidden states written to a fixed-size bank, read via cross-attention)
- Gated fusion (α, β) over the three pathways
- 4 stacked blocks, dim=128, heads=4

---

## Key finding (λ-sweep, 5 seeds, ctx=2048, 32-class recall)

A small dose of salience supervision flips episodic recall from chance to near-perfect:

| λ (salience-supervision strength) | Accuracy | Write hit-rate (target token written?) |
|---|---|---|
| 0.00 | 0.723 ± 0.329 | 0.125 ± 0.109 |
| **0.01** | **0.968 ± 0.022** | **0.993 ± 0.002** |
| 0.10 | 0.970 ± 0.021 | 0.995 ± 0.001 |
| 1.00 | 0.838 ± 0.125 | 0.995 ± 0.002 |

The v4 checkpoint was trained at the sweet-spot λ=0.01.

---

## Loading

```python
import torch
from huggingface_hub import hf_hub_download

# Make sure SynapNet_Exp is on your path:
#   git clone https://github.com/vineetha00/SynapNet_Exp
#   cd SynapNet_Exp
import sys; sys.path.insert(0, "/path/to/SynapNet_Exp")

from synapnet_memory import SynapEpisodicNet

ckpt_path = hf_hub_download(
    repo_id="Vineetha00/synapnet",
    filename="synapnet_memory_v4.pt",
)
state = torch.load(ckpt_path, map_location="cpu")

model = SynapEpisodicNet(
    dim=128, depth=4, vocab_size=2048,
    max_len=2048, num_classes=32, heads=4,
)
model.load_state_dict(state)
model.eval()
```

---

## Training tasks (from the companion code)

- **Track 1** — LRA-style long-range classification (`train_track1_lra.py`)
- **Track 2** — biosignal regression / ECG reconstruction (`train_track2_biosignal.py`)
- **Track 3** — episodic recall (`train_track3_memory_v2/v3/v4.py`)
- **Track 4** — faithful-salience supervision for interpretability (`train_track4_interpret.py`)

---

## License

MIT — see [LICENSE](https://github.com/vineetha00/SynapNet_Exp/blob/main/LICENSE).

## Citation

```bibtex
@article{synapnet_2026,
  title={SynapNet: Hybrid SSM + Sparse-Attention + Episodic Memory for Long-Range Sequence Modelling},
  author={Vallish Kumar, Vineetha},
  year={2026},
}
```