loubnabnl HF Staff commited on
Commit
dee4608
·
verified ·
1 Parent(s): 0ff8759

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -33,7 +33,7 @@ SmolLM3 is a 3B parameter language model designed to push the boundaries of smal
33
 
34
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/61c141342aac764ce1654e43/Zcm_016pWeyFr_uIkT7Ki.png)
35
 
36
- The model is a decoder-only transformer using GQA and NoRope, it was trained on 11.2T tokens with a staged curriculum of web, code, math and reasoning data. Post-training included midtraining on 100B reasoning followed by supervised fine-tuning and alignment via Anchored Preference Optimization.
37
 
38
  ### Key features
39
  - Instruct model optimized for **hybrid reasoning**
 
33
 
34
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/61c141342aac764ce1654e43/Zcm_016pWeyFr_uIkT7Ki.png)
35
 
36
+ The model is a decoder-only transformer using GQA and NoRope, it was pretrained on 11.2T tokens with a staged curriculum of web, code, math and reasoning data. Post-training included midtraining on 140B reasoning tokens followed by supervised fine-tuning and alignment via Anchored Preference Optimization (APO).
37
 
38
  ### Key features
39
  - Instruct model optimized for **hybrid reasoning**