---
license: mit
tags:
- generated_from_trainer
model-index:
- name: Romance-cleaned-1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Romance-cleaned-1

This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 4.7175

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 256
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1000
- num_epochs: 50
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 0.97  | 29   | 9.9497          |
| No log        | 1.97  | 58   | 9.1816          |
| No log        | 2.97  | 87   | 8.5947          |
| No log        | 3.97  | 116  | 8.2217          |
| No log        | 4.97  | 145  | 7.8354          |
| No log        | 5.97  | 174  | 7.5075          |
| No log        | 6.97  | 203  | 7.2112          |
| No log        | 7.97  | 232  | 6.9077          |
| No log        | 8.97  | 261  | 6.5994          |
| No log        | 9.97  | 290  | 6.3077          |
| No log        | 10.97 | 319  | 6.0416          |
| No log        | 11.97 | 348  | 5.8126          |
| No log        | 12.97 | 377  | 5.6197          |
| No log        | 13.97 | 406  | 5.4789          |
| No log        | 14.97 | 435  | 5.3665          |
| No log        | 15.97 | 464  | 5.2738          |
| No log        | 16.97 | 493  | 5.1942          |
| No log        | 17.97 | 522  | 5.1382          |
| No log        | 18.97 | 551  | 5.0784          |
| No log        | 19.97 | 580  | 5.0347          |
| No log        | 20.97 | 609  | 4.9873          |
| No log        | 21.97 | 638  | 4.9514          |
| No log        | 22.97 | 667  | 4.9112          |
| No log        | 23.97 | 696  | 4.8838          |
| No log        | 24.97 | 725  | 4.8468          |
| No log        | 25.97 | 754  | 4.8221          |
| No log        | 26.97 | 783  | 4.7996          |
| No log        | 27.97 | 812  | 4.7815          |
| No log        | 28.97 | 841  | 4.7606          |
| No log        | 29.97 | 870  | 4.7394          |
| No log        | 30.97 | 899  | 4.7167          |
| No log        | 31.97 | 928  | 4.7140          |
| No log        | 32.97 | 957  | 4.6910          |
| No log        | 33.97 | 986  | 4.6844          |
| No log        | 34.97 | 1015 | 4.6765          |
| No log        | 35.97 | 1044 | 4.6687          |
| No log        | 36.97 | 1073 | 4.6721          |
| No log        | 37.97 | 1102 | 4.6724          |
| No log        | 38.97 | 1131 | 4.6629          |
| No log        | 39.97 | 1160 | 4.6772          |
| No log        | 40.97 | 1189 | 4.6795          |
| No log        | 41.97 | 1218 | 4.6788          |
| No log        | 42.97 | 1247 | 4.6832          |
| No log        | 43.97 | 1276 | 4.6954          |
| No log        | 44.97 | 1305 | 4.7009          |
| No log        | 45.97 | 1334 | 4.7082          |
| No log        | 46.97 | 1363 | 4.7140          |
| No log        | 47.97 | 1392 | 4.7158          |
| No log        | 48.97 | 1421 | 4.7181          |
| No log        | 49.97 | 1450 | 4.7175          |


### Framework versions

- Transformers 4.24.0
- Pytorch 1.12.1+cu113
- Datasets 2.6.1
- Tokenizers 0.13.1