Upload README.md with huggingface_hub

README.md

@@ -3,6 +3,7 @@
 datasets:
 - bigscience/xP3
 - mc4
+- Leipzig_corpora_collection
 license: apache-2.0
 language:
 - en
@@ -24,10 +25,29 @@ The vocabulary has been adjusted to contain the top 2000 tokens overall as well
 
 The new vocabulary has been computed using 
   - deu_news_2023_1M for de
-  - eng_news_2023_1M for en
+  - eng_news_2023_1M for en 
+
+ from the [Leipzig language corpora collection](https://wortschatz.uni-leipzig.de/en/download/).
+
+According to the mentioned article the perfarmance degradation can be expected to be small (but no evaluation was done).
+
+
+
+# Language Adaption of mT0
+This model is an adaption of the mT0 variant (see below) for languages en, de based on the methodology in [Load What You Need: Smaller Versions of Multilingual BERT](https://arxiv.org/ftp/arxiv/papers/2010/2010.05609.pdf).
+
+The vocabulary has been adjusted to contain the top 2000 tokens overall as well as 
+  - the top 15000 tokens from en
+  - the top 30000 tokens from de 
+ and 100 special tokens.
+
+The new vocabulary has been computed using 
+  - deu_news_2023_1M for de
+  - eng_news_2023_1M for en 
+
  from the [Leipzig language corpora collection](https://wortschatz.uni-leipzig.de/en/download/).
 
-According to the mentioned article the perfarmance degradation can be expected to be small (but no evaluation was done.
+According to the mentioned article the perfarmance degradation can be expected to be small (but no evaluation was done).
 
 ![xmtf](https://github.com/bigscience-workshop/xmtf/blob/master/xmtf_banner.png?raw=true)