nllb-en-es

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7770

Model description

This model is the first iteration of a larger fine-tuning series I'm developing for a NLP Pipeline that handles translations between English, Spanish, Amharic, and Uzbek. The base model is Facebook's No Language Left Behind, and was chosen for its inclusion of low-resource languages.

Intended uses & limitations

Text to Text Translation between English --> Spanish

The intended use of this model is to handle bidirectional English-centric translation, but so far only one direction for a single pair has been developed.

Training procedure

Trained on Google ColabPRO using an OPUS Books dataset, with High RAM and A100 GPU.

Training hyperparameters

The following hyperparameters were used during training:

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2

More hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
1.8802	1.0	18694	1.7770

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2