Model description

mt5-small-finetuned-es-pt

This model is a fine-tuned version of google/mt5-small, specifically adapted to translate short texts from Spanish into Portuguese. The fine-tuning was performed as a test using a very small parallel corpus, resulting in limited performance. It achieves the following results on the evaluation set:

• Loss: 10.2098
• Bleu: 0.0327
• Gen Len: 3.65

Intended uses & limitations

Due to the small size of the training corpus, it has significant limitations and provides very low-quality translations. It is not recommended for any practical translation tasks or production use.

Training and evaluation data

The model was trained and evaluated on a small parallel corpus created from the Universal Declaration of Human Rights, containing approximately 100 aligned sentence pairs in Spanish and Portuguese.

The dataset was split into 80% for training and 20% for evaluation.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
No log	1.0	10	12.5626	0.0373	3.85
No log	2.0	20	11.3537	0.0401	3.85
No log	3.0	30	10.7420	0.0369	3.8
No log	4.0	40	10.3282	0.0327	3.65
No log	5.0	50	10.2098	0.0327	3.65

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.4.1
• Tokenizers 0.21.1