yuxintian commited on
Commit
9ce6978
·
verified ·
1 Parent(s): 3c7e7d9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -33,7 +33,7 @@ These checkpoints are released to support continued pretraining, fine-tuning, an
33
  + **[Warmup-Stable and Merge](https://arxiv.org/abs/2507.17634):** We replace conventional learning-rate decay with weighted checkpoint merging. By eliminating the decay phase, our Base Model is better suited for continual pretraining and dynamic data expansion, while enabling offline exploration of different decay profiles without rerunning costly experiments for each strategy.
34
  + **Scale Seamlessly:** Ling-3.0-tiny-base and Ling-3.0-flash-base share the same training recipe, enabling community to experiment on the Ling-3.0-tiny-base first and then scale validated training strategies to the larger Ling-3.0-flash-base.
35
 
36
- | Model Type | Base (final checkpoint of mid-training) |
37
  | --- | --- |
38
  | Architecture | Hybrid-linear MoE |
39
  | Parameter Scale | Totoal 124B, Activated 5.1B (Non-emb) |
 
33
  + **[Warmup-Stable and Merge](https://arxiv.org/abs/2507.17634):** We replace conventional learning-rate decay with weighted checkpoint merging. By eliminating the decay phase, our Base Model is better suited for continual pretraining and dynamic data expansion, while enabling offline exploration of different decay profiles without rerunning costly experiments for each strategy.
34
  + **Scale Seamlessly:** Ling-3.0-tiny-base and Ling-3.0-flash-base share the same training recipe, enabling community to experiment on the Ling-3.0-tiny-base first and then scale validated training strategies to the larger Ling-3.0-flash-base.
35
 
36
+ | Model Type | Base (final checkpoint of WSM merging) |
37
  | --- | --- |
38
  | Architecture | Hybrid-linear MoE |
39
  | Parameter Scale | Totoal 124B, Activated 5.1B (Non-emb) |