metadata
license: apache-2.0
language:
- en
- ro
pipeline_tag: translation
tags:
- localtranslate
- bart
- translation
- machine-translation
- english
- romanian
- local
- local-models
- seq2seq
metrics:
- chrf
- bleu
model-index:
- name: localtranslate_EN_RO
results:
- task:
type: translation
name: Translation en to ro
dataset:
name: Flores 200
type: facebook/flores
metrics:
- type: chrf
value: 40.7
name: chrF++ (3-beam)
verified: false
- type: bleu
value: 12.35
name: BLEU (3-beam)
verified: false
LocalTranslate EN -> RO
This is a English -> Romanian translation model trained by Mihai Popa (with the help of Kimi K2.6 Thinking)! It's based on BART, and it's a lot smaller than other models.
Why?
Because it's for my own future LocalTranslate project! Unlike Google Translate, this uses models (LTR files in the future) that you download and put on your device! And yes, it translates locally!
Notes
- Trained in just 22 minutes on Colab T4 GPU. 24M parameters and 92 MB in size!
Model Configurations
| Parameter | Value |
|---|---|
| Tokenizer | BPE |
| Vocabulary Size | 16384 tokens |
| Batch Size | 128 x 1 = 128 |
| Context Window | 128 tokens |
max_position_embeddings |
128 |
encoder_layers |
6 |
decoder_layers |
6 |
encoder_attention_heads |
6 |
decoder_attention_heads |
6 |
encoder_ffn_dim |
768 |
decoder_ffn_dim |
768 |
d_model |
384 |
dropout |
0.1 |
attention_dropout |
0.1 |
activation_function |
"gelu" |
init_std |
0.02 |
scale_embedding |
True |
normalize_before |
True |
add_final_layer_norm |
True |
pad_token_id |
tokenizer.pad_token_id |
bos_token_id |
tokenizer.bos_token_id |
eos_token_id |
tokenizer.eos_token_id |
decoder_start_token_id |
tokenizer.eos_token_id |
forced_eos_token_id |
tokenizer.eos_token_id |
tie_word_embeddings |
True |
Training Configurations
| Hyperparameter | Value |
|---|---|
output_dir |
"./localtranslate_en_ro" |
do_train |
True |
do_eval |
True |
eval_strategy |
"steps" |
eval_steps |
500 |
learning_rate |
3e-4 |
weight_decay |
0.01 |
max_steps |
6000 |
warmup_steps |
1000 |
logging_steps |
50 |
save_steps |
500 |
save_total_limit |
5000 |
fp16 |
True |
gradient_checkpointing |
True |
label_smoothing_factor |
0.1 |
predict_with_generate |
False |
report_to |
"none" |
dataloader_num_workers |
2 |
remove_unused_columns |
False |
Limitations
- Not Perfect: As with any other model, it's not 100% perfect and can generate incorrect translations!
- English-Only: It's for English -> Romanian translation (NOT vice-versa)!
Evaluation Results
| Metric | Score (greedy search on dev split) | Score (beam search, 2 beams on dev split) | Score (3 beams on dev split) | Score (greedy on Flores 200) | Score (3-beam on Flores 200) |
|---|---|---|---|---|---|
| BLEU | 33.74 | 34.67 | 34.83 | 11.31 | 12.35 |
| chrF++ | 61.35 | 62.09 | 62.29 | 39.49 | 40.70 |
Usage
Code is by Gemini 3 Flash/Kimi K2.6 Thinking (then some little modifications by myself):
from transformers import BartForConditionalGeneration, AutoTokenizer
import torch
# 1. Load from your Hugging Face Repo
model_id = "MihaiPopa-1/LocalTranslate_EN_RO"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = BartForConditionalGeneration.from_pretrained(
model_id,
torch_dtype=torch.float32, # Standard for CPU
device_map="cpu" # Forces CPU usage
)
# 2. Translate to Romanian
inputs = tokenizer("At your current usage level, this runtime may last up to 1 hour.", return_tensors="pt")
outputs = model.generate(
**inputs,
max_length=64,
num_beams=2,
early_stopping=True,
forced_eos_token_id=tokenizer.eos_token_id,
)
translation = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(translation)
Data Used
| Dataset | Translation Pairs |
|---|---|
| Europarl 7 | ~399k raw -> ~340k cleaned |
| Tatoeba+ | 16k |
| Total | 343488 |
| Dev Split | 2000 |