Update README.md
Browse files
README.md
CHANGED
|
@@ -29,7 +29,7 @@ library_name: transformers
|
|
| 29 |
|
| 30 |
Trinity-Mini-Base-Pre-Anneal is an Arcee AI 26B MoE model with 3B active parameters. It is the medium-sized model in our new Trinity family, a series of open-weight models for enterprise and tinkerers alike.
|
| 31 |
|
| 32 |
-
This base model is a pre-anneal
|
| 33 |
While this checkpoint was not exposed to the anneal phase mix containing high proportions of math and code content, it has been trained on significant amounts of such data.
|
| 34 |
This checkpoint is not suitable for chatting or general use without further finetuning and should be trained for your specific domain before use.
|
| 35 |
|
|
|
|
| 29 |
|
| 30 |
Trinity-Mini-Base-Pre-Anneal is an Arcee AI 26B MoE model with 3B active parameters. It is the medium-sized model in our new Trinity family, a series of open-weight models for enterprise and tinkerers alike.
|
| 31 |
|
| 32 |
+
This base model is a pre-anneal checkpoint captured at Adam LR: 0.002, Muon LR: 0.001 before starting learning rate decay on a high-quality data mix.
|
| 33 |
While this checkpoint was not exposed to the anneal phase mix containing high proportions of math and code content, it has been trained on significant amounts of such data.
|
| 34 |
This checkpoint is not suitable for chatting or general use without further finetuning and should be trained for your specific domain before use.
|
| 35 |
|