Upload README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,126 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- generated_from_keras_callback
|
| 5 |
+
model-index:
|
| 6 |
+
- name: Heem/distilroberta-finetuned-wtner
|
| 7 |
+
results: []
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
<!-- This model card has been generated automatically according to the information Keras had access to. You should
|
| 11 |
+
probably proofread and complete it, then remove this comment. -->
|
| 12 |
+
|
| 13 |
+
# Heem/distilroberta-finetuned-wtner
|
| 14 |
+
|
| 15 |
+
This model is a fine-tuned version of [distilroberta-base](https://huggingface.co/distilroberta-base) on an unknown dataset.
|
| 16 |
+
It achieves the following results on the evaluation set:
|
| 17 |
+
- Train Loss: 0.0055
|
| 18 |
+
- Validation Loss: 0.4521
|
| 19 |
+
- Train Precision: 0.7410
|
| 20 |
+
- Train Recall: 0.8122
|
| 21 |
+
- Train F1: 0.775
|
| 22 |
+
- Train Accuracy: 0.9382
|
| 23 |
+
- Epoch: 69
|
| 24 |
+
|
| 25 |
+
## Model description
|
| 26 |
+
|
| 27 |
+
More information needed
|
| 28 |
+
|
| 29 |
+
## Intended uses & limitations
|
| 30 |
+
|
| 31 |
+
More information needed
|
| 32 |
+
|
| 33 |
+
## Training and evaluation data
|
| 34 |
+
|
| 35 |
+
More information needed
|
| 36 |
+
|
| 37 |
+
## Training procedure
|
| 38 |
+
|
| 39 |
+
### Training hyperparameters
|
| 40 |
+
|
| 41 |
+
The following hyperparameters were used during training:
|
| 42 |
+
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 2030, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
|
| 43 |
+
- training_precision: float32
|
| 44 |
+
|
| 45 |
+
### Training results
|
| 46 |
+
|
| 47 |
+
| Train Loss | Validation Loss | Train Precision | Train Recall | Train F1 | Train Accuracy | Epoch |
|
| 48 |
+
|:----------:|:---------------:|:---------------:|:------------:|:--------:|:--------------:|:-----:|
|
| 49 |
+
| 1.3579 | 0.8909 | 0.0 | 0.0 | 0.0 | 0.7744 | 0 |
|
| 50 |
+
| 0.7332 | 0.6231 | 0.3526 | 0.2926 | 0.3198 | 0.8256 | 1 |
|
| 51 |
+
| 0.5037 | 0.4471 | 0.3927 | 0.3755 | 0.3839 | 0.8575 | 2 |
|
| 52 |
+
| 0.3675 | 0.3776 | 0.484 | 0.5284 | 0.5052 | 0.8855 | 3 |
|
| 53 |
+
| 0.2890 | 0.3519 | 0.5149 | 0.6026 | 0.5553 | 0.9039 | 4 |
|
| 54 |
+
| 0.2367 | 0.3317 | 0.5820 | 0.6507 | 0.6144 | 0.9150 | 5 |
|
| 55 |
+
| 0.1942 | 0.2970 | 0.6220 | 0.6900 | 0.6542 | 0.9237 | 6 |
|
| 56 |
+
| 0.1599 | 0.3040 | 0.6375 | 0.6681 | 0.6525 | 0.9217 | 7 |
|
| 57 |
+
| 0.1281 | 0.3037 | 0.6774 | 0.7336 | 0.7044 | 0.9304 | 8 |
|
| 58 |
+
| 0.1097 | 0.3127 | 0.708 | 0.7729 | 0.7390 | 0.9309 | 9 |
|
| 59 |
+
| 0.0915 | 0.3114 | 0.6836 | 0.7642 | 0.7216 | 0.9290 | 10 |
|
| 60 |
+
| 0.0765 | 0.3190 | 0.7072 | 0.8122 | 0.7561 | 0.9372 | 11 |
|
| 61 |
+
| 0.0665 | 0.3169 | 0.7154 | 0.7904 | 0.7510 | 0.9353 | 12 |
|
| 62 |
+
| 0.0543 | 0.3251 | 0.7059 | 0.7860 | 0.7438 | 0.9329 | 13 |
|
| 63 |
+
| 0.0472 | 0.3307 | 0.7181 | 0.8122 | 0.7623 | 0.9357 | 14 |
|
| 64 |
+
| 0.0427 | 0.3639 | 0.7148 | 0.7991 | 0.7546 | 0.9357 | 15 |
|
| 65 |
+
| 0.0380 | 0.3373 | 0.7373 | 0.8210 | 0.7769 | 0.9377 | 16 |
|
| 66 |
+
| 0.0380 | 0.3422 | 0.7449 | 0.8035 | 0.7731 | 0.9372 | 17 |
|
| 67 |
+
| 0.0304 | 0.3455 | 0.7530 | 0.8122 | 0.7815 | 0.9386 | 18 |
|
| 68 |
+
| 0.0271 | 0.3584 | 0.7294 | 0.8122 | 0.7686 | 0.9377 | 19 |
|
| 69 |
+
| 0.0249 | 0.3661 | 0.7291 | 0.7991 | 0.7625 | 0.9377 | 20 |
|
| 70 |
+
| 0.0205 | 0.3683 | 0.7352 | 0.8122 | 0.7718 | 0.9391 | 21 |
|
| 71 |
+
| 0.0212 | 0.3855 | 0.7331 | 0.8035 | 0.7667 | 0.9382 | 22 |
|
| 72 |
+
| 0.0188 | 0.3814 | 0.7419 | 0.8035 | 0.7715 | 0.9391 | 23 |
|
| 73 |
+
| 0.0189 | 0.3889 | 0.7352 | 0.8122 | 0.7718 | 0.9357 | 24 |
|
| 74 |
+
| 0.0161 | 0.3913 | 0.7379 | 0.7991 | 0.7673 | 0.9382 | 25 |
|
| 75 |
+
| 0.0154 | 0.3872 | 0.7470 | 0.8122 | 0.7782 | 0.9406 | 26 |
|
| 76 |
+
| 0.0144 | 0.3934 | 0.7326 | 0.8253 | 0.7762 | 0.9401 | 27 |
|
| 77 |
+
| 0.0154 | 0.4167 | 0.7255 | 0.8079 | 0.7645 | 0.9343 | 28 |
|
| 78 |
+
| 0.0135 | 0.3976 | 0.7341 | 0.8079 | 0.7692 | 0.9362 | 29 |
|
| 79 |
+
| 0.0119 | 0.4118 | 0.7510 | 0.8297 | 0.7884 | 0.9382 | 30 |
|
| 80 |
+
| 0.0103 | 0.4112 | 0.7323 | 0.8122 | 0.7702 | 0.9372 | 31 |
|
| 81 |
+
| 0.0103 | 0.4172 | 0.7362 | 0.8166 | 0.7743 | 0.9382 | 32 |
|
| 82 |
+
| 0.0111 | 0.4157 | 0.7283 | 0.8079 | 0.7660 | 0.9382 | 33 |
|
| 83 |
+
| 0.0103 | 0.4152 | 0.7262 | 0.7991 | 0.7609 | 0.9372 | 34 |
|
| 84 |
+
| 0.0117 | 0.4090 | 0.7188 | 0.8035 | 0.7588 | 0.9377 | 35 |
|
| 85 |
+
| 0.0098 | 0.4268 | 0.7302 | 0.8035 | 0.7651 | 0.9367 | 36 |
|
| 86 |
+
| 0.0082 | 0.4354 | 0.7233 | 0.7991 | 0.7593 | 0.9362 | 37 |
|
| 87 |
+
| 0.0096 | 0.4298 | 0.7154 | 0.7904 | 0.7510 | 0.9357 | 38 |
|
| 88 |
+
| 0.0093 | 0.4294 | 0.7273 | 0.8035 | 0.7635 | 0.9362 | 39 |
|
| 89 |
+
| 0.0084 | 0.4266 | 0.7298 | 0.7904 | 0.7589 | 0.9348 | 40 |
|
| 90 |
+
| 0.0076 | 0.4230 | 0.7251 | 0.7948 | 0.7583 | 0.9357 | 41 |
|
| 91 |
+
| 0.0068 | 0.4243 | 0.7075 | 0.7817 | 0.7427 | 0.9329 | 42 |
|
| 92 |
+
| 0.0080 | 0.4379 | 0.7137 | 0.7729 | 0.7421 | 0.9338 | 43 |
|
| 93 |
+
| 0.0067 | 0.4361 | 0.7302 | 0.8035 | 0.7651 | 0.9362 | 44 |
|
| 94 |
+
| 0.0066 | 0.4377 | 0.7341 | 0.8079 | 0.7692 | 0.9367 | 45 |
|
| 95 |
+
| 0.0056 | 0.4357 | 0.7222 | 0.7948 | 0.7568 | 0.9362 | 46 |
|
| 96 |
+
| 0.0060 | 0.4393 | 0.7205 | 0.7991 | 0.7578 | 0.9362 | 47 |
|
| 97 |
+
| 0.0060 | 0.4429 | 0.7194 | 0.7948 | 0.7552 | 0.9357 | 48 |
|
| 98 |
+
| 0.0054 | 0.4416 | 0.7312 | 0.8079 | 0.7676 | 0.9367 | 49 |
|
| 99 |
+
| 0.0060 | 0.4413 | 0.7188 | 0.8035 | 0.7588 | 0.9362 | 50 |
|
| 100 |
+
| 0.0058 | 0.4381 | 0.7344 | 0.8210 | 0.7753 | 0.9377 | 51 |
|
| 101 |
+
| 0.0063 | 0.4388 | 0.7309 | 0.7948 | 0.7615 | 0.9377 | 52 |
|
| 102 |
+
| 0.0057 | 0.4402 | 0.7412 | 0.8253 | 0.7810 | 0.9382 | 53 |
|
| 103 |
+
| 0.0052 | 0.4381 | 0.7362 | 0.8166 | 0.7743 | 0.9377 | 54 |
|
| 104 |
+
| 0.0049 | 0.4407 | 0.7362 | 0.8166 | 0.7743 | 0.9377 | 55 |
|
| 105 |
+
| 0.0050 | 0.4394 | 0.7490 | 0.8210 | 0.7833 | 0.9386 | 56 |
|
| 106 |
+
| 0.0047 | 0.4481 | 0.7460 | 0.8210 | 0.7817 | 0.9382 | 57 |
|
| 107 |
+
| 0.0052 | 0.4544 | 0.748 | 0.8166 | 0.7808 | 0.9367 | 58 |
|
| 108 |
+
| 0.0049 | 0.4501 | 0.7430 | 0.8079 | 0.7741 | 0.9362 | 59 |
|
| 109 |
+
| 0.0050 | 0.4504 | 0.744 | 0.8122 | 0.7766 | 0.9367 | 60 |
|
| 110 |
+
| 0.0047 | 0.4517 | 0.7312 | 0.8079 | 0.7676 | 0.9372 | 61 |
|
| 111 |
+
| 0.0049 | 0.4526 | 0.7450 | 0.8166 | 0.7792 | 0.9382 | 62 |
|
| 112 |
+
| 0.0049 | 0.4534 | 0.7490 | 0.8210 | 0.7833 | 0.9386 | 63 |
|
| 113 |
+
| 0.0056 | 0.4543 | 0.748 | 0.8166 | 0.7808 | 0.9386 | 64 |
|
| 114 |
+
| 0.0044 | 0.4522 | 0.7410 | 0.8122 | 0.775 | 0.9382 | 65 |
|
| 115 |
+
| 0.0047 | 0.4522 | 0.7410 | 0.8122 | 0.775 | 0.9382 | 66 |
|
| 116 |
+
| 0.0050 | 0.4521 | 0.7410 | 0.8122 | 0.775 | 0.9382 | 67 |
|
| 117 |
+
| 0.0049 | 0.4521 | 0.7410 | 0.8122 | 0.775 | 0.9382 | 68 |
|
| 118 |
+
| 0.0055 | 0.4521 | 0.7410 | 0.8122 | 0.775 | 0.9382 | 69 |
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
### Framework versions
|
| 122 |
+
|
| 123 |
+
- Transformers 4.20.1
|
| 124 |
+
- TensorFlow 2.9.1
|
| 125 |
+
- Datasets 2.4.0
|
| 126 |
+
- Tokenizers 0.12.1
|