Update README.md

README.md

@@ -2,9 +2,12 @@
 license: apache-2.0
 datasets:
 - HPLT/HPLT3.0
-- allenai/MADLAD-400
 - allenai/olmo-mix-1124
 - HuggingFaceFW/finepdfs
+- HuggingFaceTB/finemath
+- LLM360/MegaMath
+- HuggingFaceTB/stack-edu
+- HuggingFaceFW/finepdfs-edu
 language:
 - nb
 - nn
@@ -22,7 +25,8 @@ tags:
 
 This is a base (not instruction-tuned) large language model, continually pre-trained on Norwegian data starting from the English [OLMo2-13B](https://huggingface.co/allenai/OLMo-2-1124-13B) model.
 
-Our training data mixture included [HPLTv3](https://huggingface.co/datasets/HPLT/HPLT3.0) Bokmål and Nynorsk, FinePDF Bokmål and Nynorsk, MADLAD400 Norwegian, OLMo-Mix, Northern Sami dataset.
+Our training data mixture for the first stage (steps 0-24 000) included [HPLTv3](https://huggingface.co/datasets/HPLT/HPLT3.0) Bokmål and Nynorsk as well as Faroese, Icelandic, Danish, and Swedish, FinePDF Bokmål and Nynorsk as well as Faroese, Icelandic, Danish, and Swedish, OLMo-Mix, Northern Sami dataset.
+For the second stage (steps 24 000-33 000), our training data mixture includes a filtered [HPLTv3](https://huggingface.co/datasets/HPLT/HPLT3.0) Bokmål and Nynorsk as well as Faroese, Icelandic, Danish, and Swedish, FinePDF-Edu Bokmål and Nynorsk as well as Icelandic, Danish, Swedish, and English, FinePDF Faroese, Stack-Edu, MegaMath Web-Pro, FineMath 4+, InfiWebMath 4+, Northern Sami dataset
 The model was trained for 33 000 steps on around 300 billion tokens. Intermediate checkpoints are published here as branches.
 
 Training was conducted as a part of the [HPLT project](https://hplt-project.org/).