translation_1

This model is a fine-tuned version of t5-base on the opus_books dataset. It achieves the following results on the evaluation set:

• Loss: 2.5005
• Bleu: 5.03
• Gen Len: 16.0434

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
3.4771	1.0	875	2.8863	2.2211	16.4083
2.9851	2.0	1750	2.7086	3.2474	16.4271
2.8137	3.0	2625	2.6050	3.8481	16.378
2.6962	4.0	3500	2.5268	4.3032	16.2734
2.6439	5.0	4375	2.4694	4.5354	16.2414
2.5633	6.0	5250	2.4227	4.8672	16.2483
2.5122	7.0	6125	2.4068	5.0916	16.256
2.693	8.0	7000	2.5069	5.0708	16.0449
2.6754	9.0	7875	2.5003	5.0422	16.0389
2.6714	10.0	8750	2.5003	5.0467	16.0357
2.6679	11.0	9625	2.5003	5.0482	16.038
2.6812	12.0	10500	2.5004	5.0456	16.0377
2.6733	13.0	11375	2.5004	5.0456	16.036
2.6802	14.0	12250	2.5005	5.0454	16.0403
2.6652	15.0	13125	2.5005	5.0403	16.0391
2.6718	16.0	14000	2.5005	5.0483	16.0403
2.6756	17.0	14875	2.5005	5.0324	16.04
2.6751	18.0	15750	2.5005	5.03	16.0434
2.6689	19.0	16625	2.5005	5.03	16.0434
2.6687	20.0	17500	2.5005	5.03	16.0434

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.13.3