Create README.md

README.md

@@ -0,0 +1,46 @@
+---
+language: "ru"
+tags:
+- normalization
+- denoising autoencoder
+- russian
+license: mit
+---
+This is a small Russian denoising autoencoder. It can be used for restoring corrupted sentences.
+
+This model was produced by fine-tuning the [rut5-small](https://huggingface.co/cointegrated/rut5-small) model on the task of reconstructing a sentence:
+* restoring word positions (after slightly shuffling them)
+* restoring dropped words and punctuation marks (after dropping some of them randomly)
+* restoring inflection of words (after changing their inflection randomly using [natasha](https://github.com/natasha/natasha) and [pymorphy2](https://github.com/kmike/pymorphy2) packages)
+
+The fine-tuning was performed on a [Leipzig web corpus](https://wortschatz.uni-leipzig.de/en/download/Russian) of Russian sentences. 
+
+The model can be applied as follows:
+```
+# !pip install transformers sentencepiece
+import torch
+from transformers import T5ForConditionalGeneration, T5Tokenizer
+tokenizer = T5Tokenizer.from_pretrained("cointegrated/rut5-small-normalizer")
+model = T5ForConditionalGeneration.from_pretrained("cointegrated/rut5-small-normalizer")
+
+text = 'меня тобой не понимать'
+inputs = tokenizer(text, return_tensors='pt')
+with torch.no_grad():
+    hypotheses = model.generate(
+        **inputs, 
+        do_sample=True, top_p=0.95, 
+        num_return_sequences=5, 
+        repetition_penalty=2.5,
+        max_length=32,
+    )
+for h in hypotheses:
+    print(tokenizer.decode(h, skip_special_tokens=True))
+```
+A possible output is:
+```
+# Мне тебя не понимать.
+# Если бы ты понимаешь меня?
+# Я с тобой не понимаю.
+# Я тебя не понимаю.
+# Я не понимаю о чем ты.
+```