lirou-0/ko-en_transformer

Korean-English Machine Translation Transformer

Model Details

• Model Type: Encoder-Decoder Transformer
• Languages: Korean (source) → English (target)
• Training Data: traintogpb/aihub-koen-translation-integrated-base-1m
• Framework: PyTorch

Architecture

• Embedding Dimension: 256
• Encoder Layers: 4
• Decoder Layers: 4
• Attention Heads: 8
• Feedforward Dimension: 1024
• Total Parameters: 19,677,824
• Model Size: 79.95 MB

Performance

Test Set Results

• chrF Score: 61.96
• BLEU Score: 36.62

Validation Set Results

• Best chrF: 61.46
• Best BLEU: 35.59

Training Details

• Optimizer: AdamW with Transformer learning rate schedule
• Warmup Steps: 10886
• Batch Size: 64
• Label Smoothing: 0.1
• Mixed Precision Training: Yes
• Training Epochs: 15

Citation

Dataset:

@inproceedings{cettolo-etal-2016-wit3,
  title = "The {IWSLT} 2016 Evaluation Campaign",
  author = "Cettolo, Mauro and Niehues, Jan and St{\"u}ker, Sebastian and Bentivogli, Luisa and Federico, Marcello",
  booktitle = "Proceedings of the 13th International Workshop on Spoken Language Translation",
  year = "2016"
}