liputan6-seq_bn-rf16

This model is a fine-tuned version of LazarusNLP/IndoNanoT5-base on the id_liputan6 canonical dataset. It achieves the following results on the evaluation set:

• Loss: 0.2554
• Rouge1: 44.408
• Rouge2: 35.788
• Rougel: 40.8449
• Rougelsum: 43.0054
• Gen Len: 62.247

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.9013	1.0	63	0.3600	40.5674	32.5892	37.7471	39.1368	46.887
0.4754	2.0	126	0.2958	43.3031	34.5149	39.7514	41.863	56.767
0.3811	3.0	189	0.2629	43.4511	34.6775	39.9831	42.0606	57.898
0.3317	4.0	252	0.2610	43.9259	35.2198	40.3143	42.5364	57.815
0.299	5.0	315	0.2554	44.3826	35.7034	40.7597	42.9985	58.818

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1