metadata
library_name: transformers
license: mit
base_model: gpt2
tags:
- generated_from_trainer
model-index:
- name: politicalbias-2label-c03_lib07-model
results: []
politicalbias-2label-c03_lib07-model
This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.9355
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 16
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 10
- num_epochs: 30
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 3.5244 | 1.0 | 28 | 2.5668 |
| 2.2692 | 2.0 | 56 | 2.1853 |
| 2.0836 | 3.0 | 84 | 2.0283 |
| 1.7754 | 4.0 | 112 | 1.9467 |
| 1.5982 | 5.0 | 140 | 1.8749 |
| 1.4859 | 6.0 | 168 | 1.8285 |
| 1.3411 | 7.0 | 196 | 1.8039 |
| 1.297 | 8.0 | 224 | 1.8076 |
| 1.2421 | 9.0 | 252 | 1.7796 |
| 1.15 | 10.0 | 280 | 1.7774 |
| 1.1074 | 11.0 | 308 | 1.7635 |
| 1.047 | 12.0 | 336 | 1.8137 |
| 1.0428 | 13.0 | 364 | 1.7921 |
| 0.9604 | 14.0 | 392 | 1.8193 |
| 0.8682 | 15.0 | 420 | 1.8181 |
| 0.8917 | 16.0 | 448 | 1.8343 |
| 0.8894 | 17.0 | 476 | 1.8326 |
| 0.8294 | 18.0 | 504 | 1.8490 |
| 0.7678 | 19.0 | 532 | 1.8554 |
| 0.7505 | 20.0 | 560 | 1.8810 |
| 0.8025 | 21.0 | 588 | 1.8835 |
| 0.7605 | 22.0 | 616 | 1.8865 |
| 0.7012 | 23.0 | 644 | 1.9070 |
| 0.7109 | 24.0 | 672 | 1.9196 |
| 0.7366 | 25.0 | 700 | 1.9230 |
| 0.6955 | 26.0 | 728 | 1.9331 |
| 0.697 | 27.0 | 756 | 1.9366 |
| 0.7307 | 28.0 | 784 | 1.9354 |
| 0.6852 | 29.0 | 812 | 1.9360 |
| 0.7243 | 30.0 | 840 | 1.9355 |
Framework versions
- Transformers 4.46.3
- Pytorch 2.1.2+cu121
- Datasets 2.19.1
- Tokenizers 0.20.3