Note 15M was the Ultra-MC smoke test
Browse files- BENCHMARKS.md +3 -2
BENCHMARKS.md
CHANGED
|
@@ -89,6 +89,7 @@ val_ppl 59.8 at 419M tokens. No full `run_eval.py` on 68k yet.
|
|
| 89 |
|
| 90 |
## 114M sibling
|
| 91 |
|
|
|
|
| 92 |
[Byrne-100M-Ultra-MC](https://huggingface.co/Quazim0t0/Byrne-100M-Ultra-MC)
|
| 93 |
-
|
| 94 |
-
|
|
|
|
| 89 |
|
| 90 |
## 114M sibling
|
| 91 |
|
| 92 |
+
This 15M run was the toy / smoke-test for
|
| 93 |
[Byrne-100M-Ultra-MC](https://huggingface.co/Quazim0t0/Byrne-100M-Ultra-MC)
|
| 94 |
+
(Memory Cache, fractal RoPE, ~2.0B tokens). Different size, data,
|
| 95 |
+
objectives. Not a matched scale-up.
|