---
license: mit
tags:
- generated_from_keras_callback
model-index:
- name: valve_model
  results: []
---

<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# valve_model

This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 0.4860
- Validation Loss: 6.0810
- Epoch: 99

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 2e-05, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 800, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, '__passive_serialization__': True}, 'warmup_steps': 200, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
- training_precision: mixed_float16

### Training results

| Train Loss | Validation Loss | Epoch |
|:----------:|:---------------:|:-----:|
| 3.1291     | 5.9072          | 0     |
| 3.1205     | 5.9071          | 1     |
| 3.0615     | 5.9070          | 2     |
| 3.1662     | 5.9069          | 3     |
| 3.1011     | 5.9068          | 4     |
| 3.1374     | 5.9066          | 5     |
| 3.1472     | 5.9065          | 6     |
| 3.0926     | 5.9066          | 7     |
| 3.1436     | 5.9065          | 8     |
| 3.1321     | 5.9065          | 9     |
| 3.1027     | 5.9065          | 10    |
| 2.9848     | 5.9068          | 11    |
| 2.9544     | 5.9069          | 12    |
| 3.0212     | 5.9066          | 13    |
| 3.0448     | 5.9066          | 14    |
| 3.0455     | 5.9063          | 15    |
| 3.0294     | 5.9063          | 16    |
| 2.9529     | 5.9058          | 17    |
| 2.8377     | 5.9054          | 18    |
| 2.8682     | 5.9054          | 19    |
| 2.9745     | 5.9050          | 20    |
| 2.9680     | 5.9049          | 21    |
| 2.9270     | 5.9046          | 22    |
| 2.8955     | 5.9039          | 23    |
| 2.9627     | 5.9031          | 24    |
| 2.8304     | 5.9020          | 25    |
| 2.8542     | 5.9009          | 26    |
| 2.8008     | 5.8999          | 27    |
| 2.8067     | 5.8992          | 28    |
| 2.7471     | 5.8987          | 29    |
| 2.7494     | 5.8983          | 30    |
| 2.7467     | 5.8990          | 31    |
| 2.6482     | 5.9001          | 32    |
| 2.7226     | 5.9006          | 33    |
| 2.6202     | 5.9003          | 34    |
| 2.6576     | 5.9005          | 35    |
| 2.6144     | 5.9010          | 36    |
| 2.6040     | 5.9015          | 37    |
| 2.4523     | 5.9022          | 38    |
| 2.4589     | 5.9023          | 39    |
| 2.4796     | 5.9028          | 40    |
| 2.4962     | 5.9027          | 41    |
| 2.4251     | 5.9029          | 42    |
| 2.3685     | 5.9031          | 43    |
| 2.3015     | 5.9034          | 44    |
| 2.3080     | 5.9035          | 45    |
| 2.2066     | 5.9039          | 46    |
| 2.1621     | 5.9061          | 47    |
| 2.1354     | 5.9088          | 48    |
| 2.1527     | 5.9112          | 49    |
| 2.1650     | 5.9115          | 50    |
| 2.1298     | 5.9117          | 51    |
| 2.0993     | 5.9106          | 52    |
| 2.0044     | 5.9099          | 53    |
| 1.9764     | 5.9102          | 54    |
| 1.9662     | 5.9116          | 55    |
| 1.9702     | 5.9145          | 56    |
| 1.9012     | 5.9152          | 57    |
| 1.8061     | 5.9175          | 58    |
| 1.7831     | 5.9211          | 59    |
| 1.8015     | 5.9253          | 60    |
| 1.7642     | 5.9298          | 61    |
| 1.7484     | 5.9328          | 62    |
| 1.5452     | 5.9342          | 63    |
| 1.5996     | 5.9369          | 64    |
| 1.4831     | 5.9396          | 65    |
| 1.4367     | 5.9421          | 66    |
| 1.4981     | 5.9435          | 67    |
| 1.4513     | 5.9475          | 68    |
| 1.3897     | 5.9532          | 69    |
| 1.3108     | 5.9603          | 70    |
| 1.3337     | 5.9664          | 71    |
| 1.2564     | 5.9728          | 72    |
| 1.2671     | 5.9770          | 73    |
| 1.1286     | 5.9814          | 74    |
| 1.1349     | 5.9843          | 75    |
| 1.1645     | 5.9842          | 76    |
| 1.1462     | 5.9806          | 77    |
| 1.1028     | 5.9791          | 78    |
| 0.9843     | 5.9770          | 79    |
| 0.9734     | 5.9768          | 80    |
| 0.9831     | 5.9795          | 81    |
| 1.0021     | 5.9823          | 82    |
| 0.8903     | 5.9826          | 83    |
| 0.8244     | 5.9837          | 84    |
| 0.8597     | 5.9863          | 85    |
| 0.8703     | 5.9907          | 86    |
| 0.7864     | 5.9996          | 87    |
| 0.7394     | 6.0086          | 88    |
| 0.6764     | 6.0188          | 89    |
| 0.7007     | 6.0278          | 90    |
| 0.6247     | 6.0355          | 91    |
| 0.6640     | 6.0430          | 92    |
| 0.6407     | 6.0498          | 93    |
| 0.5903     | 6.0565          | 94    |
| 0.6226     | 6.0614          | 95    |
| 0.5934     | 6.0662          | 96    |
| 0.5140     | 6.0713          | 97    |
| 0.5300     | 6.0766          | 98    |
| 0.4860     | 6.0810          | 99    |


### Framework versions

- Transformers 4.29.0.dev0
- TensorFlow 2.9.1
- Datasets 2.5.1
- Tokenizers 0.13.3