metadata
license: apache-2.0
base_model: t5-small
tags:
- generated_from_keras_callback
model-index:
- name: keyword-masked-model
results: []
keyword-masked-model
This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:
- Train Loss: 0.9545
- Validation Loss: 0.7767
- Train Rouge1: 76.8067
- Train Rouge2: 64.2138
- Train Rougel: 76.1124
- Train Rougelsum: 76.0843
- Epoch: 9
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
- training_precision: float32
Training results
| Train Loss | Validation Loss | Train Rouge1 | Train Rouge2 | Train Rougel | Train Rougelsum | Epoch |
|---|---|---|---|---|---|---|
| 2.7609 | 2.0508 | 69.5404 | 56.1190 | 67.9736 | 67.9553 | 0 |
| 2.0227 | 1.5912 | 71.6288 | 58.5009 | 70.6498 | 70.7363 | 1 |
| 1.6780 | 1.3604 | 70.8191 | 58.1082 | 70.0231 | 69.8913 | 2 |
| 1.4974 | 1.2163 | 69.9943 | 57.3099 | 68.7338 | 68.7593 | 3 |
| 1.3525 | 1.1001 | 70.4850 | 57.7869 | 69.2259 | 69.2106 | 4 |
| 1.2480 | 1.0154 | 71.1204 | 58.1709 | 69.9356 | 69.9324 | 5 |
| 1.1556 | 0.9368 | 72.5520 | 59.4597 | 71.3193 | 71.4219 | 6 |
| 1.0820 | 0.8754 | 74.5839 | 61.6499 | 73.6577 | 73.6458 | 7 |
| 1.0117 | 0.8244 | 75.0935 | 62.3254 | 74.3574 | 74.2253 | 8 |
| 0.9545 | 0.7767 | 76.8067 | 64.2138 | 76.1124 | 76.0843 | 9 |
Framework versions
- Transformers 4.31.0
- TensorFlow 2.12.0
- Datasets 2.13.1
- Tokenizers 0.13.3