Format-blended SFT: chat fluency + preserved benchmark formats
Browse files- README.md +12 -0
- chkpt/quazimoto_sft.pt +2 -2
README.md
CHANGED
|
@@ -101,3 +101,15 @@ Chimera is the synthesis of a controlled series of mixer swaps (Kuramoto → Qua
|
|
| 101 |
note = {Research artifact; custom PyTorch architecture (Quazim0t0 / SpikeWhale family)}
|
| 102 |
}
|
| 103 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
note = {Research artifact; custom PyTorch architecture (Quazim0t0 / SpikeWhale family)}
|
| 102 |
}
|
| 103 |
```
|
| 104 |
+
|
| 105 |
+
## Update: format-blended SFT
|
| 106 |
+
|
| 107 |
+
Short instruction/format SFT on a 60/25/15 blend of HuggingFaceTB/smoltalk,
|
| 108 |
+
GSM8K-train ('#### N' reasoning), and MMLU-style ('Answer: <letter>') examples,
|
| 109 |
+
so chat fluency improves while the benchmark output-formats are preserved.
|
| 110 |
+
Held-out (test-split) before->after:
|
| 111 |
+
|
| 112 |
+
held-out 2.23->1.34, MMLU acc 0.035->0.250, format 0.140->1.000
|
| 113 |
+
|
| 114 |
+
Note: fluency + output-format gains. Benchmark *accuracy* remains near the floor
|
| 115 |
+
for a model this size -- the SFT does not add reasoning ability.
|
chkpt/quazimoto_sft.pt
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:de6e4755b180edfb9acf30dce5749fe39cdead5196c05ca9547f53391297e73c
|
| 3 |
+
size 768972805
|