Roman Knyazhitskiy commited on
Commit ·
81b6be2
1
Parent(s): 866842e
New README format
Browse files- .gitignore +1 -0
- README.md +3 -1
- assets/kl-vs-size.svg +0 -0
.gitignore
CHANGED
|
@@ -1,2 +1,3 @@
|
|
| 1 |
/readme-generator.toml
|
| 2 |
/kl-results.csv
|
|
|
|
|
|
| 1 |
/readme-generator.toml
|
| 2 |
/kl-results.csv
|
| 3 |
+
/.DS_Store
|
README.md
CHANGED
|
@@ -23,7 +23,9 @@ base_model_relation: quantized
|
|
| 23 |
|
| 24 |

|
| 25 |
|
| 26 |
-
|
|
|
|
|
|
|
| 27 |
|
| 28 |
### Quickstart
|
| 29 |
|
|
|
|
| 23 |
|
| 24 |

|
| 25 |
|
| 26 |
+
Mirai-M is on the size-KL Pareto frontier: we found no checkpoint that is smaller while also having lower KL divergence.
|
| 27 |
+
|
| 28 |
+
*Evaluation data mixture: 45% public agentic, 30% public SFT/long-context, 25% private chat data.*
|
| 29 |
|
| 30 |
### Quickstart
|
| 31 |
|
assets/kl-vs-size.svg
CHANGED
|
|
|
|