pumad/pumadic-en-pl

Pumatic English-Polish Translation Model

A neural machine translation model for English to Polish translation, trained entirely from scratch using the MarianMT architecture.

Model Description

• Model type: Encoder-Decoder (MarianMT architecture)
• Language pair: English → Polish
• Parameters: ~157M
• Training approach: From scratch (randomly initialized weights, custom tokenizer)
• GPU: 4x NVIDIA H200
• Trained by: pumad

Note: This model was not fine-tuned from any existing pre-trained model. Both the model weights and the SentencePiece tokenizer were trained from scratch on the parallel corpus.

Architecture

Component	Configuration
d_model	768
Encoder layers	8
Decoder layers	8
Attention heads	12
FFN dimension	3072
Vocabulary size	32,000
Max position embeddings	512
Activation function	GELU

Training Details

Training Data

The model was trained on high-quality parallel corpora:

• OPUS-100 - Multilingual parallel corpus
• Europarl - European Parliament proceedings
• UN Parallel Corpus (UNPC) - United Nations documents

Training Procedure

• Hardware: 4x NVIDIA H200 GPU (distributed training)
• Framework: Hugging Face Transformers + Accelerate
• Batch size: 512 per GPU (2048 effective)
• Learning rate: 3e-4 with cosine decay
• Warmup: 6% of training steps
• Epochs: 10
• Optimizer: Fused AdamW
• Precision: bf16 mixed precision
• Max sequence length: 128 tokens

Tokenizer

A custom SentencePiece tokenizer (unigram model) was trained on the parallel corpus with:

• 32,000 vocabulary size
• 99.95% character coverage
• Language tag support (>>pl<<)

Data Preprocessing

• Quality filtering: Removed pairs with fewer than 5 words or more than 200 words
• Length ratio filtering: Excluded pairs with extreme length ratios (< 0.5 or > 2.0)
• Deduplication: Removed duplicate source sentences

Usage

Using the Transformers library

from transformers import MarianMTModel, MarianTokenizer

model_name = "pumad/pumatic-en-pl"
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)

text = "Hello, how are you today?"
inputs = tokenizer(text, return_tensors="pt", padding=True)
translated = model.generate(**inputs)
output = tokenizer.decode(translated[0], skip_special_tokens=True)
print(output)

Using the Pipeline API

from transformers import pipeline

translator = pipeline("translation", model="pumad/pumatic-en-pl")
result = translator("The quick brown fox jumps over the lazy dog.")
print(result[0]['translation_text'])

Demo

Try this model live at pumatic.eu

API documentation available at pumatic.eu/docs

Limitations

• Optimized for general-purpose translation; domain-specific terminology may vary in quality
• Maximum input length of ~400 characters per chunk for optimal results
• Best performance on formal/written text; colloquial expressions may be less accurate

License

Apache 2.0

Citation

If you use this model, please cite:

@misc{pumatic-en-pl,
  author = {pumad},
  title = {Pumatic English-Polish Translation Model},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/pumad/pumatic-en-pl}
}