Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -15,24 +15,6 @@ pipeline_tag: text2text-generation
|
|
| 15 |
|
| 16 |
|
| 17 |
|
| 18 |
-
# Language Adaption of mT0
|
| 19 |
-
This model is an adaption of the mT0 variant (see below) for languages en, de based on the methodology in [Load What You Need: Smaller Versions of Multilingual BERT](https://arxiv.org/ftp/arxiv/papers/2010/2010.05609.pdf).
|
| 20 |
-
|
| 21 |
-
The vocabulary has been adjusted to contain the top 2000 tokens overall as well as
|
| 22 |
-
- the top 15000 tokens from en
|
| 23 |
-
- the top 30000 tokens from de
|
| 24 |
-
and 100 special tokens.
|
| 25 |
-
|
| 26 |
-
The new vocabulary has been computed using
|
| 27 |
-
- deu_news_2023_1M for de
|
| 28 |
-
- eng_news_2023_1M for en
|
| 29 |
-
|
| 30 |
-
from the [Leipzig language corpora collection](https://wortschatz.uni-leipzig.de/en/download/).
|
| 31 |
-
|
| 32 |
-
According to the mentioned article the perfarmance degradation can be expected to be small (but no evaluation was done).
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
# Language Adaption of mT0
|
| 37 |
This model is an adaption of the mT0 variant (see below) for languages en, de based on the methodology in [Load What You Need: Smaller Versions of Multilingual BERT](https://arxiv.org/ftp/arxiv/papers/2010/2010.05609.pdf).
|
| 38 |
|
|
|
|
| 15 |
|
| 16 |
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
# Language Adaption of mT0
|
| 19 |
This model is an adaption of the mT0 variant (see below) for languages en, de based on the methodology in [Load What You Need: Smaller Versions of Multilingual BERT](https://arxiv.org/ftp/arxiv/papers/2010/2010.05609.pdf).
|
| 20 |
|