Add posthoc-drafters

posthoc-drafters/README.md

@@ -0,0 +1,17 @@
+# Post-hoc drafters for SB (run 1)
+
+A token router (embed2 -> 736 bits) and a draft readout fitted to the **frozen** SB run 1 for 200M tokens. The loss is KL(sg(p) || q) + 0.1 * BCE to SB's real router bits. These are the post-hoc baseline in the paper.
+
+| File | SSM | Val top-1 agreement | Val KL | Accepted at gamma=4 (open / long) |
+|---|---|---|---|---|
+| `posthoc_sb.pt` | - | 0.304 | 2.61 | 0.42 / 0.39 |
+| `posthoc_sb_ssm.pt` | 256 channels | 0.331 | 2.30 | 0.55 / 0.51 |
+
+Router imitation: 93.8% bit accuracy, 78.9% code accuracy per route.
+
+```python
+from ffnmem.infer import load_model
+from drafting.posthoc_model import PosthocDrafter
+target, _ = load_model("sb-run1/model.pt", device="cuda")
+drafter = PosthocDrafter.load("posthoc-drafters/posthoc_sb_ssm.pt", target)
+```

posthoc-drafters/posthoc_sb.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ba6193fd3f17fc3ce63fc38d6fbc7f3dee183f5cbd5de2c8fd3a40b4225e41b
+size 8150477

posthoc-drafters/posthoc_sb_ssm.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d235dc6c1c7ad92a87608d21982a3680f25f092c3ffb95ca82daf2a2cf24d0c8
+size 10221055