README.md

metadata

license: mit
library_name: pytorch
tags:
  - state-space-model
  - sparse-attention
  - episodic-memory
  - long-context
  - interpretability
language:
  - en

SynapNet — Episodic Memory Checkpoints

Hybrid SSM + sparse-attention + episodic-memory architecture. This repo holds the base architectural checkpoints; the edge-deployment variants live at Vineetha00/synapnet-edge.

📦 Code: https://github.com/vineetha00/SynapNet_Exp 🛠️ Deployment companion: https://github.com/vineetha00/SynapNet-Edge · 🤗 https://huggingface.co/Vineetha00/synapnet-edge

---

Checkpoints in this repo

File	Variant	Size	Notes
synapnet_memory_v4.pt	Track 3 v4 (current best)	5.7 MB	Trained with salience supervision λ=0.01; near-perfect episodic recall
synapnet_memory_v2.pt	Track 3 v2	6.8 MB	Earlier episodic-recall checkpoint
synapnet_memory.pt	Original episodic memory	18.2 MB	Initial release of the WriteableMemory variant

---

Architecture (Track 3 v4 — recommended)

• SimpleSSM (depthwise conv, kernel_size=9) for local temporal dynamics
• SparseEventAttention (salience-gated top-K mixing)
• WriteableMemory (top-K hidden states written to a fixed-size bank, read via cross-attention)
• Gated fusion (α, β) over the three pathways
• 4 stacked blocks, dim=128, heads=4

---

Key finding (λ-sweep, 5 seeds, ctx=2048, 32-class recall)

A small dose of salience supervision flips episodic recall from chance to near-perfect:

λ (salience-supervision strength)	Accuracy	Write hit-rate (target token written?)
0.00	0.723 ± 0.329	0.125 ± 0.109
0.01	0.968 ± 0.022	0.993 ± 0.002
0.10	0.970 ± 0.021	0.995 ± 0.001
1.00	0.838 ± 0.125	0.995 ± 0.002

The v4 checkpoint was trained at the sweet-spot λ=0.01.

---

Loading

import torch
from huggingface_hub import hf_hub_download

# Make sure SynapNet_Exp is on your path:
#   git clone https://github.com/vineetha00/SynapNet_Exp
#   cd SynapNet_Exp
import sys; sys.path.insert(0, "/path/to/SynapNet_Exp")

from synapnet_memory import SynapEpisodicNet

ckpt_path = hf_hub_download(
    repo_id="Vineetha00/synapnet",
    filename="synapnet_memory_v4.pt",
)
state = torch.load(ckpt_path, map_location="cpu")

model = SynapEpisodicNet(
    dim=128, depth=4, vocab_size=2048,
    max_len=2048, num_classes=32, heads=4,
)
model.load_state_dict(state)
model.eval()

---

Training tasks (from the companion code)

• Track 1 — LRA-style long-range classification (train_track1_lra.py)
• Track 2 — biosignal regression / ECG reconstruction (train_track2_biosignal.py)
• Track 3 — episodic recall (train_track3_memory_v2/v3/v4.py)
• Track 4 — faithful-salience supervision for interpretability (train_track4_interpret.py)

---

License

MIT — see LICENSE.

Citation

@article{synapnet_2026,
  title={SynapNet: Hybrid SSM + Sparse-Attention + Episodic Memory for Long-Range Sequence Modelling},
  author={Vallish Kumar, Vineetha},
  year={2026},
}