337cf8c74e21b9bc9056dbbd6c8a6e4b

This model is a fine-tuned version of google-bert/bert-large-uncased-whole-word-masking on the fancyzhx/dbpedia_14 dataset. It achieves the following results on the evaluation set:

• Loss: 2.6462
• Data Size: 1.0
• Epoch Runtime: 1629.2923
• Accuracy: 0.0712
• F1 Macro: 0.0095
• Rouge1: 0.0712
• Rouge2: 0.0
• Rougel: 0.0712
• Rougelsum: 0.0712

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Accuracy	F1 Macro	Rouge1	Rouge2	Rougel	Rougelsum
No log	0	0	2.7341	0	53.5921	0.0845	0.0366	0.0845	0.0	0.0844	0.0845
0.1032	1	17500	0.1029	0.0078	68.5890	0.9810	0.9808	0.9810	0.0	0.9809	0.9809
0.0906	2	35000	0.1063	0.0156	78.7501	0.9809	0.9810	0.9810	0.0	0.9810	0.9810
0.0781	3	52500	0.0879	0.0312	103.2231	0.9831	0.9830	0.9831	0.0	0.9831	0.9831
0.0908	4	70000	0.0666	0.0625	152.2387	0.9876	0.9876	0.9876	0.0	0.9876	0.9876
0.0681	5	87500	0.0880	0.125	247.9169	0.9847	0.9847	0.9847	0.0	0.9847	0.9847
0.0906	6	105000	0.0891	0.25	442.5070	0.9859	0.9859	0.9860	0.0	0.9859	0.9859
0.0151	7	122500	2.6462	0.5	834.4264	0.0714	0.0095	0.0714	0.0	0.0714	0.0714
2.6613	8.0	140000	2.6462	1.0	1629.2923	0.0712	0.0095	0.0712	0.0	0.0712	0.0712

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.3.0
• Tokenizers 0.22.1