| --- |
| license: mit |
| language: |
| - ru |
| - en |
| tags: |
| - translation |
| - transformer |
| - from-scratch |
| - pytorch |
| - russian |
| - english |
| datasets: |
| - Helsinki-NLP/opus-100 |
| metrics: |
| - bleu |
| - chrf |
| library_name: pytorch |
| inference: false |
| --- |
| |
| # Russian → English Transformer (from scratch) |
|
|
| A compact **encoder–decoder Transformer trained from scratch** (no pretrained weights) |
| for Russian→English translation. Built as a learning project — the tokenizer, model, |
| training loop, and beam-search decoding are all hand-written. |
|
|
| - **Parameters:** ~11.5M |
| - **Architecture:** 4 encoder + 4 decoder layers, `d_model=256`, 8 heads, `d_ff=1024`, |
| sinusoidal positional encoding, tied input/output embeddings |
| - **Tokenizer:** byte-level BPE, vocab 16,000 (shared RU/EN), included as `tokenizer.json` |
| - **Data:** 200,000 [opus-100](https://huggingface.co/datasets/Helsinki-NLP/opus-100) RU–EN pairs |
| - **Training:** 60 epochs max, early-stopped ~epoch 40 (patience 5), Adam + Noam LR schedule, |
| label smoothing 0.1, batch size 64 |
|
|
| ## Results (held-out test split, 1,951 sentences) |
|
|
| | Decoding | BLEU | chrF | |
| |----------|:----:|:----:| |
| | Greedy | 25.04 | 47.07 | |
| | Beam-5 | **25.91** | **47.85** | |
|
|
| Validation BLEU was 26.96. Note that opus-100 (subtitle-derived) contains some |
| misaligned reference pairs, so these BLEU numbers slightly **underestimate** true quality. |
|
|
| ## Usage |
|
|
| ```python |
| # pip install torch tokenizers huggingface_hub |
| from huggingface_hub import snapshot_download |
| import sys |
| |
| path = snapshot_download("prplguyy/ru-en-transformer") |
| sys.path.insert(0, path) |
| from translator import translate |
| |
| print(translate("Привет, как у тебя дела сегодня?", method="beam")) |
| # -> "Hey, how are you doing today?" |
| ``` |
|
|
| The repo bundles everything needed to run inference on CPU: `model.pt` (weights), |
| `tokenizer.json`, and the model/decoding code (`config.py`, `model.py`, `decoding.py`, |
| `translator.py`). |
|
|
| ## Limitations |
|
|
| Small from-scratch model: strong on everyday conversational sentences, but expect rough |
| edges on rare proper names, idioms, and long or technical text. English→Russian is not |
| supported (trained one direction only). |
|
|
| ## Links |
|
|
| - 🕹️ **Live demo:** https://transformertranslaterussian2english.streamlit.app/ |
| - 💻 **Source / training code:** https://github.com/prplguyy/transformerTranslateRussianEnglish |
|
|
