46c26463bbb2b0137bfdde773eb0e682

This model is a fine-tuned version of openai-community/gpt2-large on the contemmcm/cls_mmlu dataset. It achieves the following results on the evaluation set:

• Loss: 1.4652
• Data Size: 1.0
• Epoch Runtime: 107.5623
• Accuracy: 0.2952
• F1 Macro: 0.2712

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Accuracy	F1 Macro
No log	0	0	1.5314	0	3.9921	0.2400	0.1901
No log	1	438	1.5087	0.0078	5.2868	0.2739	0.2150
No log	2	876	1.4835	0.0156	6.7015	0.2653	0.2060
No log	3	1314	1.4681	0.0312	9.1583	0.2633	0.1794
No log	4	1752	1.3984	0.0625	13.5580	0.2779	0.2211
0.0791	5	2190	1.4042	0.125	20.8593	0.2507	0.1605
0.1818	6	2628	1.3995	0.25	34.1266	0.2646	0.1795
1.3445	7	3066	1.4037	0.5	58.8051	0.2859	0.2783
1.2112	8.0	3504	1.4652	1.0	107.5623	0.2952	0.2712

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.3.0
• Tokenizers 0.22.1