politicalbias-2label-c05_lib05-model

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9615

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 10
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
2.9496	1.0	39	2.5190
2.1437	2.0	78	2.1535
1.9034	3.0	117	1.9971
1.6806	4.0	156	1.9064
1.4749	5.0	195	1.8746
1.4472	6.0	234	1.8460
1.3154	7.0	273	1.8178
1.2543	8.0	312	1.8026
1.1388	9.0	351	1.8083
1.1692	10.0	390	1.8039
1.1154	11.0	429	1.7799
1.0669	12.0	468	1.8129
1.0117	13.0	507	1.8272
0.9494	14.0	546	1.8080
0.8766	15.0	585	1.8475
0.9277	16.0	624	1.8374
0.9133	17.0	663	1.8668
0.814	18.0	702	1.8835
0.8115	19.0	741	1.8863
0.6828	20.0	780	1.9074
0.7662	21.0	819	1.9201
0.7536	22.0	858	1.9126
0.7635	23.0	897	1.9277
0.7074	24.0	936	1.9442
0.7102	25.0	975	1.9459
0.6578	26.0	1014	1.9417
0.7004	27.0	1053	1.9498
0.6588	28.0	1092	1.9581
0.7162	29.0	1131	1.9591
0.742	30.0	1170	1.9615

Framework versions

• Transformers 4.46.3
• Pytorch 2.1.2+cu121
• Datasets 2.19.1
• Tokenizers 0.20.3