politicalbias-2label-real-model

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7471

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 10
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
2.9828	1.0	39	2.4471
2.2313	2.0	78	2.0422
2.0586	3.0	117	1.8611
1.7542	4.0	156	1.7685
1.549	5.0	195	1.7010
1.4293	6.0	234	1.6650
1.3637	7.0	273	1.6152
1.2406	8.0	312	1.6382
1.1573	9.0	351	1.6230
1.1089	10.0	390	1.6142
1.1462	11.0	429	1.6144
1.0535	12.0	468	1.6303
1.0122	13.0	507	1.6216
0.9419	14.0	546	1.6392
0.8635	15.0	585	1.6257
0.8324	16.0	624	1.6638
0.9237	17.0	663	1.6579
0.8648	18.0	702	1.6813
0.8705	19.0	741	1.6845
0.7731	20.0	780	1.6865
0.7712	21.0	819	1.7009
0.8053	22.0	858	1.7126
0.7734	23.0	897	1.7075
0.7188	24.0	936	1.7203
0.6785	25.0	975	1.7335
0.7539	26.0	1014	1.7470
0.7204	27.0	1053	1.7413
0.7392	28.0	1092	1.7434
0.7806	29.0	1131	1.7458
0.7152	30.0	1170	1.7471

Framework versions

• Transformers 4.46.3
• Pytorch 2.1.2+cu121
• Datasets 2.19.1
• Tokenizers 0.20.3