File size: 613 Bytes
87ba23f f6adfcb 87ba23f 3434cd0 a3779aa 3434cd0 dc75713 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | ---
tags:
- model_hub_mixin
- pytorch_model_hub_mixin
datasets:
- IgorVolochay/russian_jokes
language:
- ru
pipeline_tag: text-generation
---
First homewok for NLP course in MSU from VK.
Causal Transformer with Alibi bias and SwiGLU MLP.
Can generate jokes in Russian language. Only the very small "nano" model with only 0.51 million parametrs and 1024 vocabluary size was implemented due to weak computing capabilities.
This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration: |