metadata
language:
- ru
- en
tags:
- russian
- fill-mask
- pretraining
- embeddings
license: mit
widget:
- text: Миниатюрная модель для [MASK] разных задач.
This is a very small distilled version of the bert-base-multilingual-cased model for Russian and English.
This model is useful if you want to fine-tune it for a relatively simple Russian task (e.g. NER or sentiment classification), and you care more about speed and size than about accuracy. It is approximately x10 faster than DeepPavlov/rubert-base-cased-sentence.
It was trained on the Yandex translate corpus using MLM loss (partially distilled from bert-base-multilingual-cased) and translation ranking loss.