68555726c11363f53e086754536d00a8

This model is a fine-tuned version of google-t5/t5-small on the Helsinki-NLP/opus_books dataset. It achieves the following results on the evaluation set:

• Loss: 2.5854
• Data Size: 1.0
• Epoch Runtime: 10.5495
• Bleu: 0.3546

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Bleu
No log	0	0	4.5032	0	1.5879	0.0692
No log	1	70	4.4816	0.0078	2.2425	0.0700
No log	2	140	4.3131	0.0156	2.0090	0.0719
No log	3	210	4.1441	0.0312	2.6878	0.0862
No log	4	280	4.0041	0.0625	2.5877	0.0869
No log	5	350	3.8863	0.125	3.1527	0.0873
No log	6	420	3.6893	0.25	4.5614	0.0784
0.6152	7	490	3.5220	0.5	7.0418	0.1073
3.6837	8.0	560	3.3663	1.0	10.8850	0.0846
3.5947	9.0	630	3.2801	1.0	10.7694	0.0891
3.475	10.0	700	3.2189	1.0	10.3918	0.1407
3.4398	11.0	770	3.1681	1.0	11.4925	0.1205
3.3973	12.0	840	3.1214	1.0	10.7400	0.1161
3.3267	13.0	910	3.0848	1.0	10.9519	0.1195
3.2921	14.0	980	3.0437	1.0	10.4212	0.1566
3.241	15.0	1050	3.0125	1.0	11.0945	0.1474
3.211	16.0	1120	2.9879	1.0	10.6209	0.1510
3.1935	17.0	1190	2.9629	1.0	11.1408	0.1270
3.149	18.0	1260	2.9370	1.0	11.3135	0.1382
3.1301	19.0	1330	2.9151	1.0	10.8250	0.2277
3.0878	20.0	1400	2.8955	1.0	11.0756	0.2090
3.0644	21.0	1470	2.8740	1.0	11.4419	0.2166
3.0707	22.0	1540	2.8543	1.0	11.4769	0.2146
3.0192	23.0	1610	2.8377	1.0	10.4927	0.2452
2.9859	24.0	1680	2.8216	1.0	10.8954	0.2454
2.9768	25.0	1750	2.8079	1.0	10.5931	0.2513
2.9549	26.0	1820	2.7927	1.0	10.3818	0.2657
2.9542	27.0	1890	2.7804	1.0	10.5983	0.2818
2.9111	28.0	1960	2.7681	1.0	10.8964	0.2663
2.9148	29.0	2030	2.7577	1.0	10.7036	0.2789
2.8886	30.0	2100	2.7437	1.0	11.0976	0.2809
2.8706	31.0	2170	2.7334	1.0	10.5031	0.2941
2.8605	32.0	2240	2.7215	1.0	10.8753	0.2885
2.8396	33.0	2310	2.7136	1.0	10.7512	0.2872
2.8337	34.0	2380	2.6985	1.0	10.9788	0.3063
2.8054	35.0	2450	2.6883	1.0	10.6171	0.3181
2.8037	36.0	2520	2.6834	1.0	12.0016	0.3142
2.7741	37.0	2590	2.6739	1.0	12.1009	0.3142
2.762	38.0	2660	2.6657	1.0	11.2751	0.3106
2.752	39.0	2730	2.6591	1.0	10.2097	0.3208
2.7483	40.0	2800	2.6489	1.0	11.1446	0.3454
2.7321	41.0	2870	2.6445	1.0	10.4991	0.3394
2.7224	42.0	2940	2.6366	1.0	10.4703	0.3375
2.6932	43.0	3010	2.6267	1.0	10.2642	0.3506
2.7012	44.0	3080	2.6237	1.0	10.6429	0.3334
2.6728	45.0	3150	2.6121	1.0	11.2366	0.3395
2.6742	46.0	3220	2.6072	1.0	10.4530	0.3658
2.6614	47.0	3290	2.6059	1.0	11.8707	0.3640
2.6534	48.0	3360	2.5960	1.0	10.1064	0.3810
2.6341	49.0	3430	2.5891	1.0	10.4508	0.3710
2.6254	50.0	3500	2.5854	1.0	10.5495	0.3546

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.2.0
• Tokenizers 0.22.1