Update README.md
Browse files
README.md
CHANGED
|
@@ -8,7 +8,7 @@ metrics:
|
|
| 8 |
- f1
|
| 9 |
- accuracy
|
| 10 |
model-index:
|
| 11 |
-
- name:
|
| 12 |
results: []
|
| 13 |
---
|
| 14 |
|
|
@@ -17,13 +17,13 @@ should probably proofread and complete it, then remove this comment. -->
|
|
| 17 |
|
| 18 |
# Team-Gryffindor-distilbert-base-finetuned-NER-creditcardcontract-100epoch
|
| 19 |
|
| 20 |
-
This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on
|
| 21 |
It achieves the following results on the evaluation set:
|
| 22 |
-
- Loss: 0.
|
| 23 |
-
- Precision:
|
| 24 |
-
- Recall: 0.
|
| 25 |
-
- F1: 0.
|
| 26 |
-
- Accuracy:
|
| 27 |
|
| 28 |
## Model description
|
| 29 |
|
|
@@ -48,7 +48,7 @@ The following hyperparameters were used during training:
|
|
| 48 |
- seed: 42
|
| 49 |
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
| 50 |
- lr_scheduler_type: linear
|
| 51 |
-
- num_epochs:
|
| 52 |
|
| 53 |
### Training results
|
| 54 |
|
|
@@ -85,273 +85,8 @@ The following hyperparameters were used during training:
|
|
| 85 |
| 0.0107 | 9.71 | 14500 | 0.0444 | 0.6840 | 0.6752 | 0.6796 | 0.9915 |
|
| 86 |
| 0.0118 | 10.05 | 15000 | 0.0457 | 0.7015 | 0.6771 | 0.6891 | 0.9918 |
|
| 87 |
| 0.0102 | 10.38 | 15500 | 0.0500 | 0.7413 | 0.6679 | 0.7027 | 0.9919 |
|
| 88 |
-
| 0.
|
| 89 |
-
|
| 90 |
-
| 0.0105 | 11.39 | 17000 | 0.0469 | 0.6612 | 0.6697 | 0.6655 | 0.9913 |
|
| 91 |
-
| 0.0113 | 11.72 | 17500 | 0.0463 | 0.7266 | 0.6826 | 0.7039 | 0.9922 |
|
| 92 |
-
| 0.0104 | 12.06 | 18000 | 0.0468 | 0.6942 | 0.6789 | 0.6865 | 0.9917 |
|
| 93 |
-
| 0.0108 | 12.39 | 18500 | 0.0468 | 0.7085 | 0.7046 | 0.7065 | 0.9917 |
|
| 94 |
-
| 0.0102 | 12.73 | 19000 | 0.0453 | 0.7391 | 0.6550 | 0.6946 | 0.9918 |
|
| 95 |
-
| 0.0101 | 13.06 | 19500 | 0.0464 | 0.7248 | 0.6716 | 0.6971 | 0.9919 |
|
| 96 |
-
| 0.0111 | 13.4 | 20000 | 0.0476 | 0.7100 | 0.6917 | 0.7007 | 0.9918 |
|
| 97 |
-
| 0.0092 | 13.73 | 20500 | 0.0456 | 0.7013 | 0.6936 | 0.6974 | 0.9919 |
|
| 98 |
-
| 0.0115 | 14.07 | 21000 | 0.0476 | 0.7107 | 0.6807 | 0.6954 | 0.9915 |
|
| 99 |
-
| 0.0105 | 14.4 | 21500 | 0.0490 | 0.7156 | 0.6972 | 0.7063 | 0.9919 |
|
| 100 |
-
| 0.0106 | 14.74 | 22000 | 0.0478 | 0.6613 | 0.6807 | 0.6709 | 0.9913 |
|
| 101 |
-
| 0.0099 | 15.07 | 22500 | 0.0501 | 0.7009 | 0.7138 | 0.7073 | 0.9920 |
|
| 102 |
-
| 0.0105 | 15.41 | 23000 | 0.0477 | 0.7004 | 0.6991 | 0.6997 | 0.9916 |
|
| 103 |
-
| 0.0097 | 15.74 | 23500 | 0.0448 | 0.7175 | 0.7083 | 0.7128 | 0.9918 |
|
| 104 |
-
| 0.0119 | 16.08 | 24000 | 0.0471 | 0.6937 | 0.7229 | 0.7080 | 0.9918 |
|
| 105 |
-
| 0.0101 | 16.41 | 24500 | 0.0465 | 0.7187 | 0.6844 | 0.7011 | 0.9918 |
|
| 106 |
-
| 0.01 | 16.74 | 25000 | 0.0475 | 0.7115 | 0.6697 | 0.6900 | 0.9916 |
|
| 107 |
-
| 0.0099 | 17.08 | 25500 | 0.0478 | 0.7160 | 0.6477 | 0.6802 | 0.9915 |
|
| 108 |
-
| 0.0094 | 17.41 | 26000 | 0.0487 | 0.7255 | 0.6789 | 0.7014 | 0.9919 |
|
| 109 |
-
| 0.0106 | 17.75 | 26500 | 0.0487 | 0.6994 | 0.6404 | 0.6686 | 0.9911 |
|
| 110 |
-
| 0.0104 | 18.08 | 27000 | 0.0509 | 0.7126 | 0.6734 | 0.6925 | 0.9914 |
|
| 111 |
-
| 0.0091 | 18.42 | 27500 | 0.0491 | 0.6910 | 0.6606 | 0.6754 | 0.9911 |
|
| 112 |
-
| 0.0102 | 18.75 | 28000 | 0.0494 | 0.6954 | 0.6954 | 0.6954 | 0.9917 |
|
| 113 |
-
| 0.0107 | 19.09 | 28500 | 0.0475 | 0.7247 | 0.6954 | 0.7097 | 0.9920 |
|
| 114 |
-
| 0.0108 | 19.42 | 29000 | 0.0547 | 0.7284 | 0.6642 | 0.6948 | 0.9914 |
|
| 115 |
-
| 0.0099 | 19.76 | 29500 | 0.0491 | 0.7389 | 0.6697 | 0.7026 | 0.9919 |
|
| 116 |
-
| 0.0097 | 20.09 | 30000 | 0.0504 | 0.7159 | 0.7028 | 0.7093 | 0.9919 |
|
| 117 |
-
| 0.0108 | 20.43 | 30500 | 0.0500 | 0.6939 | 0.6697 | 0.6816 | 0.9918 |
|
| 118 |
-
| 0.0113 | 20.76 | 31000 | 0.0510 | 0.7143 | 0.6789 | 0.6961 | 0.9916 |
|
| 119 |
-
| 0.0081 | 21.1 | 31500 | 0.0538 | 0.7318 | 0.6807 | 0.7053 | 0.9923 |
|
| 120 |
-
| 0.0109 | 21.43 | 32000 | 0.0495 | 0.6909 | 0.6972 | 0.6941 | 0.9917 |
|
| 121 |
-
| 0.0096 | 21.77 | 32500 | 0.0520 | 0.7455 | 0.6771 | 0.7096 | 0.9919 |
|
| 122 |
-
| 0.0104 | 22.1 | 33000 | 0.0513 | 0.7107 | 0.6807 | 0.6954 | 0.9918 |
|
| 123 |
-
| 0.0093 | 22.44 | 33500 | 0.0497 | 0.7205 | 0.6716 | 0.6952 | 0.9917 |
|
| 124 |
-
| 0.011 | 22.77 | 34000 | 0.0513 | 0.7121 | 0.6899 | 0.7008 | 0.9917 |
|
| 125 |
-
| 0.0103 | 23.11 | 34500 | 0.0534 | 0.7455 | 0.6881 | 0.7156 | 0.9921 |
|
| 126 |
-
| 0.0102 | 23.44 | 35000 | 0.0498 | 0.7226 | 0.6642 | 0.6922 | 0.9917 |
|
| 127 |
-
| 0.0105 | 23.78 | 35500 | 0.0486 | 0.6799 | 0.6587 | 0.6692 | 0.9913 |
|
| 128 |
-
| 0.0107 | 24.11 | 36000 | 0.0464 | 0.7107 | 0.6716 | 0.6906 | 0.9912 |
|
| 129 |
-
| 0.0096 | 24.45 | 36500 | 0.0511 | 0.7079 | 0.6716 | 0.6893 | 0.9918 |
|
| 130 |
-
| 0.0098 | 24.78 | 37000 | 0.0472 | 0.7227 | 0.7174 | 0.7201 | 0.9918 |
|
| 131 |
-
| 0.0101 | 25.12 | 37500 | 0.0523 | 0.7101 | 0.6697 | 0.6893 | 0.9917 |
|
| 132 |
-
| 0.0102 | 25.45 | 38000 | 0.0533 | 0.7154 | 0.6917 | 0.7034 | 0.9914 |
|
| 133 |
-
| 0.0103 | 25.79 | 38500 | 0.0464 | 0.7485 | 0.6936 | 0.72 | 0.9922 |
|
| 134 |
-
| 0.0098 | 26.12 | 39000 | 0.0517 | 0.7471 | 0.6991 | 0.7223 | 0.9920 |
|
| 135 |
-
| 0.0104 | 26.46 | 39500 | 0.0504 | 0.7180 | 0.7009 | 0.7094 | 0.9919 |
|
| 136 |
-
| 0.0095 | 26.79 | 40000 | 0.0483 | 0.7394 | 0.7028 | 0.7206 | 0.9919 |
|
| 137 |
-
| 0.0103 | 27.13 | 40500 | 0.0532 | 0.7266 | 0.6972 | 0.7116 | 0.9921 |
|
| 138 |
-
| 0.0094 | 27.46 | 41000 | 0.0486 | 0.7346 | 0.7009 | 0.7174 | 0.9922 |
|
| 139 |
-
| 0.0101 | 27.8 | 41500 | 0.0513 | 0.7047 | 0.6917 | 0.6981 | 0.9920 |
|
| 140 |
-
| 0.0099 | 28.13 | 42000 | 0.0525 | 0.7212 | 0.6550 | 0.6865 | 0.9917 |
|
| 141 |
-
| 0.0106 | 28.47 | 42500 | 0.0472 | 0.6939 | 0.6862 | 0.6900 | 0.9917 |
|
| 142 |
-
| 0.0096 | 28.8 | 43000 | 0.0511 | 0.7056 | 0.6991 | 0.7023 | 0.9916 |
|
| 143 |
-
| 0.01 | 29.14 | 43500 | 0.0533 | 0.7165 | 0.6771 | 0.6962 | 0.9917 |
|
| 144 |
-
| 0.0108 | 29.47 | 44000 | 0.0517 | 0.7253 | 0.6881 | 0.7062 | 0.9918 |
|
| 145 |
-
| 0.0105 | 29.81 | 44500 | 0.0481 | 0.7269 | 0.6936 | 0.7099 | 0.9921 |
|
| 146 |
-
| 0.0099 | 30.14 | 45000 | 0.0533 | 0.7057 | 0.7083 | 0.7070 | 0.9916 |
|
| 147 |
-
| 0.0095 | 30.48 | 45500 | 0.0537 | 0.7178 | 0.6954 | 0.7064 | 0.9917 |
|
| 148 |
-
| 0.0094 | 30.81 | 46000 | 0.0528 | 0.7099 | 0.6826 | 0.6960 | 0.9914 |
|
| 149 |
-
| 0.0106 | 31.15 | 46500 | 0.0553 | 0.7294 | 0.6826 | 0.7052 | 0.9916 |
|
| 150 |
-
| 0.0101 | 31.48 | 47000 | 0.0515 | 0.7339 | 0.6881 | 0.7102 | 0.9917 |
|
| 151 |
-
| 0.0106 | 31.82 | 47500 | 0.0519 | 0.7358 | 0.6899 | 0.7121 | 0.9919 |
|
| 152 |
-
| 0.0101 | 32.15 | 48000 | 0.0500 | 0.6875 | 0.6862 | 0.6869 | 0.9915 |
|
| 153 |
-
| 0.0087 | 32.48 | 48500 | 0.0515 | 0.7232 | 0.7046 | 0.7138 | 0.9921 |
|
| 154 |
-
| 0.0112 | 32.82 | 49000 | 0.0466 | 0.7083 | 0.6862 | 0.6971 | 0.9916 |
|
| 155 |
-
| 0.011 | 33.15 | 49500 | 0.0523 | 0.7304 | 0.6661 | 0.6967 | 0.9918 |
|
| 156 |
-
| 0.0104 | 33.49 | 50000 | 0.0507 | 0.7278 | 0.7064 | 0.7169 | 0.9921 |
|
| 157 |
-
| 0.0097 | 33.82 | 50500 | 0.0515 | 0.7525 | 0.7028 | 0.7268 | 0.9926 |
|
| 158 |
-
| 0.0094 | 34.16 | 51000 | 0.0531 | 0.7283 | 0.6789 | 0.7028 | 0.9916 |
|
| 159 |
-
| 0.01 | 34.49 | 51500 | 0.0520 | 0.7192 | 0.6954 | 0.7071 | 0.9917 |
|
| 160 |
-
| 0.0107 | 34.83 | 52000 | 0.0523 | 0.7481 | 0.7193 | 0.7334 | 0.9921 |
|
| 161 |
-
| 0.01 | 35.16 | 52500 | 0.0514 | 0.7208 | 0.7009 | 0.7107 | 0.9920 |
|
| 162 |
-
| 0.0095 | 35.5 | 53000 | 0.0521 | 0.7384 | 0.6734 | 0.7044 | 0.9917 |
|
| 163 |
-
| 0.01 | 35.83 | 53500 | 0.0504 | 0.7245 | 0.7046 | 0.7144 | 0.9919 |
|
| 164 |
-
| 0.0101 | 36.17 | 54000 | 0.0535 | 0.7456 | 0.7046 | 0.7245 | 0.9923 |
|
| 165 |
-
| 0.0102 | 36.5 | 54500 | 0.0522 | 0.7485 | 0.7101 | 0.7288 | 0.9922 |
|
| 166 |
-
| 0.0098 | 36.84 | 55000 | 0.0531 | 0.7320 | 0.6917 | 0.7113 | 0.9918 |
|
| 167 |
-
| 0.0102 | 37.17 | 55500 | 0.0526 | 0.75 | 0.6936 | 0.7207 | 0.9923 |
|
| 168 |
-
| 0.0098 | 37.51 | 56000 | 0.0551 | 0.7437 | 0.7028 | 0.7226 | 0.9919 |
|
| 169 |
-
| 0.0102 | 37.84 | 56500 | 0.0532 | 0.7529 | 0.7101 | 0.7309 | 0.9919 |
|
| 170 |
-
| 0.0091 | 38.18 | 57000 | 0.0541 | 0.7667 | 0.7174 | 0.7412 | 0.9925 |
|
| 171 |
-
| 0.0113 | 38.51 | 57500 | 0.0524 | 0.7380 | 0.7083 | 0.7228 | 0.9922 |
|
| 172 |
-
| 0.0088 | 38.85 | 58000 | 0.0542 | 0.7285 | 0.7138 | 0.7210 | 0.9920 |
|
| 173 |
-
| 0.0095 | 39.18 | 58500 | 0.0567 | 0.7095 | 0.6991 | 0.7043 | 0.9920 |
|
| 174 |
-
| 0.0107 | 39.52 | 59000 | 0.0525 | 0.7355 | 0.7193 | 0.7273 | 0.9922 |
|
| 175 |
-
| 0.0092 | 39.85 | 59500 | 0.0535 | 0.7359 | 0.6954 | 0.7151 | 0.9921 |
|
| 176 |
-
| 0.0097 | 40.19 | 60000 | 0.0550 | 0.7175 | 0.7083 | 0.7128 | 0.9920 |
|
| 177 |
-
| 0.0096 | 40.52 | 60500 | 0.0519 | 0.7594 | 0.7064 | 0.7319 | 0.9925 |
|
| 178 |
-
| 0.0095 | 40.86 | 61000 | 0.0532 | 0.7297 | 0.7083 | 0.7188 | 0.9922 |
|
| 179 |
-
| 0.0105 | 41.19 | 61500 | 0.0577 | 0.7264 | 0.6771 | 0.7009 | 0.9918 |
|
| 180 |
-
| 0.0097 | 41.53 | 62000 | 0.0551 | 0.7065 | 0.6624 | 0.6837 | 0.9913 |
|
| 181 |
-
| 0.0095 | 41.86 | 62500 | 0.0552 | 0.7102 | 0.6881 | 0.6990 | 0.9918 |
|
| 182 |
-
| 0.0094 | 42.2 | 63000 | 0.0537 | 0.7204 | 0.6807 | 0.7 | 0.9919 |
|
| 183 |
-
| 0.009 | 42.53 | 63500 | 0.0526 | 0.7322 | 0.6972 | 0.7143 | 0.9919 |
|
| 184 |
-
| 0.0111 | 42.87 | 64000 | 0.0533 | 0.7324 | 0.7083 | 0.7201 | 0.9922 |
|
| 185 |
-
| 0.0106 | 43.2 | 64500 | 0.0554 | 0.7170 | 0.6972 | 0.7070 | 0.9918 |
|
| 186 |
-
| 0.0101 | 43.54 | 65000 | 0.0555 | 0.7129 | 0.6881 | 0.7003 | 0.9917 |
|
| 187 |
-
| 0.0104 | 43.87 | 65500 | 0.0528 | 0.7510 | 0.7083 | 0.7290 | 0.9923 |
|
| 188 |
-
| 0.0094 | 44.21 | 66000 | 0.0563 | 0.7402 | 0.6899 | 0.7142 | 0.9921 |
|
| 189 |
-
| 0.0099 | 44.54 | 66500 | 0.0546 | 0.7298 | 0.7138 | 0.7217 | 0.9919 |
|
| 190 |
-
| 0.0101 | 44.88 | 67000 | 0.0527 | 0.7447 | 0.7064 | 0.7250 | 0.9923 |
|
| 191 |
-
| 0.0089 | 45.21 | 67500 | 0.0511 | 0.752 | 0.6899 | 0.7196 | 0.9922 |
|
| 192 |
-
| 0.0102 | 45.55 | 68000 | 0.0547 | 0.6989 | 0.6771 | 0.6878 | 0.9918 |
|
| 193 |
-
| 0.0095 | 45.88 | 68500 | 0.0520 | 0.7385 | 0.7046 | 0.7211 | 0.9921 |
|
| 194 |
-
| 0.0094 | 46.22 | 69000 | 0.0539 | 0.7287 | 0.7046 | 0.7164 | 0.9920 |
|
| 195 |
-
| 0.0102 | 46.55 | 69500 | 0.0521 | 0.7421 | 0.6917 | 0.7160 | 0.9922 |
|
| 196 |
-
| 0.0099 | 46.89 | 70000 | 0.0542 | 0.7417 | 0.6954 | 0.7178 | 0.9919 |
|
| 197 |
-
| 0.0099 | 47.22 | 70500 | 0.0542 | 0.7217 | 0.6899 | 0.7054 | 0.9919 |
|
| 198 |
-
| 0.0082 | 47.56 | 71000 | 0.0551 | 0.7237 | 0.6826 | 0.7025 | 0.9921 |
|
| 199 |
-
| 0.0103 | 47.89 | 71500 | 0.0522 | 0.7331 | 0.6954 | 0.7137 | 0.9919 |
|
| 200 |
-
| 0.0103 | 48.23 | 72000 | 0.0542 | 0.7148 | 0.7174 | 0.7161 | 0.9920 |
|
| 201 |
-
| 0.0083 | 48.56 | 72500 | 0.0542 | 0.7456 | 0.6991 | 0.7216 | 0.9919 |
|
| 202 |
-
| 0.0102 | 48.89 | 73000 | 0.0496 | 0.7451 | 0.7028 | 0.7233 | 0.9921 |
|
| 203 |
-
| 0.0105 | 49.23 | 73500 | 0.0524 | 0.7545 | 0.6936 | 0.7228 | 0.9923 |
|
| 204 |
-
| 0.0097 | 49.56 | 74000 | 0.0518 | 0.7369 | 0.6991 | 0.7175 | 0.9922 |
|
| 205 |
-
| 0.01 | 49.9 | 74500 | 0.0519 | 0.7239 | 0.7119 | 0.7179 | 0.9920 |
|
| 206 |
-
| 0.01 | 50.23 | 75000 | 0.0507 | 0.7238 | 0.6972 | 0.7103 | 0.9919 |
|
| 207 |
-
| 0.0103 | 50.57 | 75500 | 0.0527 | 0.7442 | 0.7046 | 0.7238 | 0.9922 |
|
| 208 |
-
| 0.0092 | 50.9 | 76000 | 0.0540 | 0.7520 | 0.6954 | 0.7226 | 0.9921 |
|
| 209 |
-
| 0.0082 | 51.24 | 76500 | 0.0530 | 0.7419 | 0.7174 | 0.7295 | 0.9923 |
|
| 210 |
-
| 0.0093 | 51.57 | 77000 | 0.0555 | 0.7631 | 0.6972 | 0.7287 | 0.9926 |
|
| 211 |
-
| 0.0103 | 51.91 | 77500 | 0.0536 | 0.7418 | 0.7064 | 0.7237 | 0.9922 |
|
| 212 |
-
| 0.0098 | 52.24 | 78000 | 0.0526 | 0.7505 | 0.7119 | 0.7307 | 0.9925 |
|
| 213 |
-
| 0.0085 | 52.58 | 78500 | 0.0544 | 0.7232 | 0.7046 | 0.7138 | 0.9920 |
|
| 214 |
-
| 0.0107 | 52.91 | 79000 | 0.0551 | 0.7342 | 0.7046 | 0.7191 | 0.9922 |
|
| 215 |
-
| 0.0099 | 53.25 | 79500 | 0.0573 | 0.7666 | 0.6991 | 0.7313 | 0.9923 |
|
| 216 |
-
| 0.0097 | 53.58 | 80000 | 0.0539 | 0.7274 | 0.7101 | 0.7187 | 0.9922 |
|
| 217 |
-
| 0.009 | 53.92 | 80500 | 0.0556 | 0.7645 | 0.7028 | 0.7323 | 0.9922 |
|
| 218 |
-
| 0.0107 | 54.25 | 81000 | 0.0562 | 0.748 | 0.6862 | 0.7158 | 0.9922 |
|
| 219 |
-
| 0.0094 | 54.59 | 81500 | 0.0499 | 0.7337 | 0.7028 | 0.7179 | 0.9922 |
|
| 220 |
-
| 0.0091 | 54.92 | 82000 | 0.0542 | 0.7230 | 0.7138 | 0.7184 | 0.9921 |
|
| 221 |
-
| 0.0089 | 55.26 | 82500 | 0.0551 | 0.7397 | 0.7248 | 0.7322 | 0.9923 |
|
| 222 |
-
| 0.0099 | 55.59 | 83000 | 0.0540 | 0.7495 | 0.6917 | 0.7195 | 0.9920 |
|
| 223 |
-
| 0.0097 | 55.93 | 83500 | 0.0541 | 0.7433 | 0.7174 | 0.7302 | 0.9923 |
|
| 224 |
-
| 0.0093 | 56.26 | 84000 | 0.0559 | 0.7471 | 0.7156 | 0.7310 | 0.9924 |
|
| 225 |
-
| 0.0088 | 56.6 | 84500 | 0.0560 | 0.7378 | 0.7229 | 0.7303 | 0.9925 |
|
| 226 |
-
| 0.0106 | 56.93 | 85000 | 0.0519 | 0.7369 | 0.6991 | 0.7175 | 0.9923 |
|
| 227 |
-
| 0.0099 | 57.27 | 85500 | 0.0557 | 0.7223 | 0.6826 | 0.7019 | 0.9920 |
|
| 228 |
-
| 0.0091 | 57.6 | 86000 | 0.0562 | 0.7380 | 0.7083 | 0.7228 | 0.9923 |
|
| 229 |
-
| 0.0093 | 57.94 | 86500 | 0.0550 | 0.7583 | 0.7138 | 0.7353 | 0.9924 |
|
| 230 |
-
| 0.0092 | 58.27 | 87000 | 0.0541 | 0.75 | 0.7211 | 0.7353 | 0.9923 |
|
| 231 |
-
| 0.0089 | 58.61 | 87500 | 0.0558 | 0.7297 | 0.6936 | 0.7112 | 0.9919 |
|
| 232 |
-
| 0.0105 | 58.94 | 88000 | 0.0551 | 0.7417 | 0.7009 | 0.7208 | 0.9921 |
|
| 233 |
-
| 0.0098 | 59.28 | 88500 | 0.0562 | 0.7407 | 0.6972 | 0.7183 | 0.9919 |
|
| 234 |
-
| 0.009 | 59.61 | 89000 | 0.0551 | 0.7476 | 0.7064 | 0.7264 | 0.9922 |
|
| 235 |
-
| 0.0098 | 59.95 | 89500 | 0.0562 | 0.7641 | 0.6954 | 0.7281 | 0.9924 |
|
| 236 |
-
| 0.0087 | 60.28 | 90000 | 0.0564 | 0.7456 | 0.7046 | 0.7245 | 0.9925 |
|
| 237 |
-
| 0.0099 | 60.62 | 90500 | 0.0569 | 0.7366 | 0.6826 | 0.7086 | 0.9920 |
|
| 238 |
-
| 0.0103 | 60.95 | 91000 | 0.0550 | 0.7563 | 0.7119 | 0.7335 | 0.9926 |
|
| 239 |
-
| 0.0094 | 61.29 | 91500 | 0.0567 | 0.7654 | 0.7064 | 0.7347 | 0.9924 |
|
| 240 |
-
| 0.0092 | 61.62 | 92000 | 0.0562 | 0.7427 | 0.6991 | 0.7202 | 0.9920 |
|
| 241 |
-
| 0.01 | 61.96 | 92500 | 0.0555 | 0.7623 | 0.7119 | 0.7362 | 0.9925 |
|
| 242 |
-
| 0.0089 | 62.29 | 93000 | 0.0570 | 0.7400 | 0.7156 | 0.7276 | 0.9923 |
|
| 243 |
-
| 0.0089 | 62.63 | 93500 | 0.0566 | 0.7486 | 0.7156 | 0.7317 | 0.9923 |
|
| 244 |
-
| 0.0102 | 62.96 | 94000 | 0.0565 | 0.7446 | 0.7009 | 0.7221 | 0.9922 |
|
| 245 |
-
| 0.0096 | 63.3 | 94500 | 0.0571 | 0.7370 | 0.7046 | 0.7205 | 0.9922 |
|
| 246 |
-
| 0.0097 | 63.63 | 95000 | 0.0568 | 0.7432 | 0.7064 | 0.7244 | 0.9923 |
|
| 247 |
-
| 0.0096 | 63.97 | 95500 | 0.0566 | 0.7409 | 0.7083 | 0.7242 | 0.9923 |
|
| 248 |
-
| 0.0088 | 64.3 | 96000 | 0.0576 | 0.7427 | 0.6991 | 0.7202 | 0.9922 |
|
| 249 |
-
| 0.0101 | 64.63 | 96500 | 0.0562 | 0.7340 | 0.7138 | 0.7237 | 0.9922 |
|
| 250 |
-
| 0.0096 | 64.97 | 97000 | 0.0567 | 0.7427 | 0.7046 | 0.7232 | 0.9923 |
|
| 251 |
-
| 0.0096 | 65.3 | 97500 | 0.0557 | 0.7328 | 0.7046 | 0.7184 | 0.9921 |
|
| 252 |
-
| 0.0096 | 65.64 | 98000 | 0.0541 | 0.7416 | 0.6899 | 0.7148 | 0.9920 |
|
| 253 |
-
| 0.0086 | 65.97 | 98500 | 0.0566 | 0.7628 | 0.7083 | 0.7345 | 0.9925 |
|
| 254 |
-
| 0.0085 | 66.31 | 99000 | 0.0588 | 0.7480 | 0.6972 | 0.7217 | 0.9922 |
|
| 255 |
-
| 0.0102 | 66.64 | 99500 | 0.0569 | 0.7437 | 0.7083 | 0.7256 | 0.9922 |
|
| 256 |
-
| 0.0091 | 66.98 | 100000 | 0.0569 | 0.7549 | 0.7064 | 0.7299 | 0.9924 |
|
| 257 |
-
| 0.0097 | 67.31 | 100500 | 0.0593 | 0.7481 | 0.7083 | 0.7276 | 0.9922 |
|
| 258 |
-
| 0.0099 | 67.65 | 101000 | 0.0568 | 0.7357 | 0.7101 | 0.7227 | 0.9921 |
|
| 259 |
-
| 0.0091 | 67.98 | 101500 | 0.0573 | 0.7380 | 0.7028 | 0.7199 | 0.9920 |
|
| 260 |
-
| 0.0099 | 68.32 | 102000 | 0.0580 | 0.7306 | 0.6917 | 0.7107 | 0.9919 |
|
| 261 |
-
| 0.0091 | 68.65 | 102500 | 0.0586 | 0.7534 | 0.7064 | 0.7292 | 0.9924 |
|
| 262 |
-
| 0.0091 | 68.99 | 103000 | 0.0562 | 0.7549 | 0.7119 | 0.7328 | 0.9923 |
|
| 263 |
-
| 0.0086 | 69.32 | 103500 | 0.0573 | 0.7337 | 0.7028 | 0.7179 | 0.9921 |
|
| 264 |
-
| 0.0099 | 69.66 | 104000 | 0.0576 | 0.7456 | 0.7046 | 0.7245 | 0.9921 |
|
| 265 |
-
| 0.0094 | 69.99 | 104500 | 0.0575 | 0.7380 | 0.7028 | 0.7199 | 0.9921 |
|
| 266 |
-
| 0.0093 | 70.33 | 105000 | 0.0585 | 0.7418 | 0.7064 | 0.7237 | 0.9921 |
|
| 267 |
-
| 0.0092 | 70.66 | 105500 | 0.0579 | 0.7624 | 0.7064 | 0.7333 | 0.9924 |
|
| 268 |
-
| 0.0091 | 71.0 | 106000 | 0.0584 | 0.7374 | 0.6954 | 0.7158 | 0.9920 |
|
| 269 |
-
| 0.0087 | 71.33 | 106500 | 0.0562 | 0.7386 | 0.7156 | 0.7269 | 0.9920 |
|
| 270 |
-
| 0.009 | 71.67 | 107000 | 0.0563 | 0.7349 | 0.6917 | 0.7127 | 0.9919 |
|
| 271 |
-
| 0.0106 | 72.0 | 107500 | 0.0560 | 0.7612 | 0.7193 | 0.7396 | 0.9924 |
|
| 272 |
-
| 0.0095 | 72.34 | 108000 | 0.0575 | 0.7414 | 0.7156 | 0.7283 | 0.9922 |
|
| 273 |
-
| 0.0097 | 72.67 | 108500 | 0.0577 | 0.7486 | 0.7156 | 0.7317 | 0.9924 |
|
| 274 |
-
| 0.0089 | 73.01 | 109000 | 0.0572 | 0.7568 | 0.7138 | 0.7347 | 0.9924 |
|
| 275 |
-
| 0.0089 | 73.34 | 109500 | 0.0575 | 0.7647 | 0.7156 | 0.7393 | 0.9924 |
|
| 276 |
-
| 0.0107 | 73.68 | 110000 | 0.0565 | 0.7554 | 0.7083 | 0.7311 | 0.9923 |
|
| 277 |
-
| 0.0089 | 74.01 | 110500 | 0.0576 | 0.7436 | 0.6917 | 0.7167 | 0.9922 |
|
| 278 |
-
| 0.0091 | 74.35 | 111000 | 0.0575 | 0.7466 | 0.7083 | 0.7269 | 0.9922 |
|
| 279 |
-
| 0.0092 | 74.68 | 111500 | 0.0583 | 0.7534 | 0.7119 | 0.7321 | 0.9922 |
|
| 280 |
-
| 0.0102 | 75.02 | 112000 | 0.0551 | 0.7539 | 0.7028 | 0.7274 | 0.9922 |
|
| 281 |
-
| 0.0092 | 75.35 | 112500 | 0.0547 | 0.7442 | 0.7101 | 0.7268 | 0.9922 |
|
| 282 |
-
| 0.009 | 75.69 | 113000 | 0.0566 | 0.7397 | 0.6936 | 0.7159 | 0.9921 |
|
| 283 |
-
| 0.0096 | 76.02 | 113500 | 0.0569 | 0.7442 | 0.7046 | 0.7238 | 0.9922 |
|
| 284 |
-
| 0.0094 | 76.36 | 114000 | 0.0576 | 0.7457 | 0.7101 | 0.7274 | 0.9922 |
|
| 285 |
-
| 0.0089 | 76.69 | 114500 | 0.0577 | 0.7399 | 0.7046 | 0.7218 | 0.9921 |
|
| 286 |
-
| 0.0097 | 77.03 | 115000 | 0.0584 | 0.7529 | 0.7046 | 0.7280 | 0.9924 |
|
| 287 |
-
| 0.0091 | 77.36 | 115500 | 0.0565 | 0.7385 | 0.7046 | 0.7211 | 0.9921 |
|
| 288 |
-
| 0.0092 | 77.7 | 116000 | 0.0582 | 0.7515 | 0.7046 | 0.7273 | 0.9922 |
|
| 289 |
-
| 0.0091 | 78.03 | 116500 | 0.0580 | 0.7665 | 0.7046 | 0.7342 | 0.9923 |
|
| 290 |
-
| 0.0097 | 78.37 | 117000 | 0.0585 | 0.7603 | 0.7101 | 0.7343 | 0.9924 |
|
| 291 |
-
| 0.0088 | 78.7 | 117500 | 0.0582 | 0.7429 | 0.7156 | 0.7290 | 0.9922 |
|
| 292 |
-
| 0.0093 | 79.04 | 118000 | 0.0580 | 0.7380 | 0.7028 | 0.7199 | 0.9921 |
|
| 293 |
-
| 0.0087 | 79.37 | 118500 | 0.0577 | 0.7465 | 0.6917 | 0.7181 | 0.9921 |
|
| 294 |
-
| 0.0093 | 79.71 | 119000 | 0.0580 | 0.7414 | 0.7101 | 0.7254 | 0.9922 |
|
| 295 |
-
| 0.0094 | 80.04 | 119500 | 0.0581 | 0.7324 | 0.6881 | 0.7096 | 0.9921 |
|
| 296 |
-
| 0.0087 | 80.38 | 120000 | 0.0593 | 0.7355 | 0.6991 | 0.7168 | 0.9923 |
|
| 297 |
-
| 0.0098 | 80.71 | 120500 | 0.0592 | 0.7261 | 0.6954 | 0.7104 | 0.9920 |
|
| 298 |
-
| 0.0103 | 81.04 | 121000 | 0.0577 | 0.7384 | 0.6991 | 0.7182 | 0.9922 |
|
| 299 |
-
| 0.0091 | 81.38 | 121500 | 0.0584 | 0.7417 | 0.7009 | 0.7208 | 0.9921 |
|
| 300 |
-
| 0.0097 | 81.71 | 122000 | 0.0587 | 0.7365 | 0.7028 | 0.7192 | 0.9922 |
|
| 301 |
-
| 0.0091 | 82.05 | 122500 | 0.0580 | 0.7368 | 0.6936 | 0.7146 | 0.9921 |
|
| 302 |
-
| 0.0086 | 82.38 | 123000 | 0.0577 | 0.7356 | 0.7046 | 0.7198 | 0.9922 |
|
| 303 |
-
| 0.0099 | 82.72 | 123500 | 0.0579 | 0.7390 | 0.7119 | 0.7252 | 0.9922 |
|
| 304 |
-
| 0.0091 | 83.05 | 124000 | 0.0584 | 0.7578 | 0.7119 | 0.7342 | 0.9925 |
|
| 305 |
-
| 0.009 | 83.39 | 124500 | 0.0585 | 0.7549 | 0.7064 | 0.7299 | 0.9924 |
|
| 306 |
-
| 0.0085 | 83.72 | 125000 | 0.0582 | 0.7529 | 0.7101 | 0.7309 | 0.9922 |
|
| 307 |
-
| 0.0108 | 84.06 | 125500 | 0.0579 | 0.7520 | 0.7064 | 0.7285 | 0.9921 |
|
| 308 |
-
| 0.0085 | 84.39 | 126000 | 0.0583 | 0.7451 | 0.6972 | 0.7204 | 0.9921 |
|
| 309 |
-
| 0.0092 | 84.73 | 126500 | 0.0591 | 0.7367 | 0.6881 | 0.7116 | 0.9920 |
|
| 310 |
-
| 0.0092 | 85.06 | 127000 | 0.0585 | 0.7442 | 0.7046 | 0.7238 | 0.9921 |
|
| 311 |
-
| 0.0098 | 85.4 | 127500 | 0.0598 | 0.7524 | 0.7083 | 0.7297 | 0.9922 |
|
| 312 |
-
| 0.0099 | 85.73 | 128000 | 0.0581 | 0.7486 | 0.7156 | 0.7317 | 0.9923 |
|
| 313 |
-
| 0.0086 | 86.07 | 128500 | 0.0581 | 0.7383 | 0.6936 | 0.7152 | 0.9920 |
|
| 314 |
-
| 0.0094 | 86.4 | 129000 | 0.0584 | 0.7490 | 0.7119 | 0.7300 | 0.9923 |
|
| 315 |
-
| 0.0096 | 86.74 | 129500 | 0.0591 | 0.7466 | 0.7083 | 0.7269 | 0.9924 |
|
| 316 |
-
| 0.0086 | 87.07 | 130000 | 0.0592 | 0.7466 | 0.7083 | 0.7269 | 0.9923 |
|
| 317 |
-
| 0.0089 | 87.41 | 130500 | 0.0584 | 0.7485 | 0.7101 | 0.7288 | 0.9924 |
|
| 318 |
-
| 0.0089 | 87.74 | 131000 | 0.0580 | 0.7447 | 0.7119 | 0.7280 | 0.9922 |
|
| 319 |
-
| 0.0099 | 88.08 | 131500 | 0.0577 | 0.7447 | 0.7064 | 0.7250 | 0.9922 |
|
| 320 |
-
| 0.0095 | 88.41 | 132000 | 0.0576 | 0.7520 | 0.7064 | 0.7285 | 0.9923 |
|
| 321 |
-
| 0.0094 | 88.75 | 132500 | 0.0582 | 0.7471 | 0.7101 | 0.7281 | 0.9922 |
|
| 322 |
-
| 0.0086 | 89.08 | 133000 | 0.0587 | 0.7461 | 0.7064 | 0.7257 | 0.9922 |
|
| 323 |
-
| 0.0085 | 89.42 | 133500 | 0.0594 | 0.7318 | 0.7009 | 0.7160 | 0.9922 |
|
| 324 |
-
| 0.0087 | 89.75 | 134000 | 0.0594 | 0.7305 | 0.6862 | 0.7077 | 0.9920 |
|
| 325 |
-
| 0.0096 | 90.09 | 134500 | 0.0591 | 0.7372 | 0.6844 | 0.7098 | 0.9922 |
|
| 326 |
-
| 0.0094 | 90.42 | 135000 | 0.0587 | 0.7340 | 0.6936 | 0.7132 | 0.9921 |
|
| 327 |
-
| 0.0098 | 90.76 | 135500 | 0.0594 | 0.7407 | 0.6972 | 0.7183 | 0.9922 |
|
| 328 |
-
| 0.0087 | 91.09 | 136000 | 0.0595 | 0.7422 | 0.7028 | 0.7220 | 0.9923 |
|
| 329 |
-
| 0.0095 | 91.43 | 136500 | 0.0596 | 0.7327 | 0.6991 | 0.7155 | 0.9922 |
|
| 330 |
-
| 0.0092 | 91.76 | 137000 | 0.0589 | 0.7342 | 0.7046 | 0.7191 | 0.9922 |
|
| 331 |
-
| 0.009 | 92.1 | 137500 | 0.0595 | 0.7380 | 0.7028 | 0.7199 | 0.9922 |
|
| 332 |
-
| 0.009 | 92.43 | 138000 | 0.0598 | 0.7351 | 0.7028 | 0.7186 | 0.9922 |
|
| 333 |
-
| 0.0097 | 92.77 | 138500 | 0.0593 | 0.7515 | 0.7046 | 0.7273 | 0.9923 |
|
| 334 |
-
| 0.008 | 93.1 | 139000 | 0.0596 | 0.7456 | 0.6991 | 0.7216 | 0.9921 |
|
| 335 |
-
| 0.0097 | 93.44 | 139500 | 0.0597 | 0.75 | 0.7046 | 0.7266 | 0.9922 |
|
| 336 |
-
| 0.0087 | 93.77 | 140000 | 0.0597 | 0.7437 | 0.7028 | 0.7226 | 0.9922 |
|
| 337 |
-
| 0.0084 | 94.11 | 140500 | 0.0598 | 0.7442 | 0.7046 | 0.7238 | 0.9921 |
|
| 338 |
-
| 0.0098 | 94.44 | 141000 | 0.0598 | 0.7437 | 0.7028 | 0.7226 | 0.9922 |
|
| 339 |
-
| 0.0092 | 94.78 | 141500 | 0.0599 | 0.7417 | 0.7009 | 0.7208 | 0.9922 |
|
| 340 |
-
| 0.0094 | 95.11 | 142000 | 0.0599 | 0.7398 | 0.6991 | 0.7189 | 0.9921 |
|
| 341 |
-
| 0.0099 | 95.45 | 142500 | 0.0600 | 0.7422 | 0.6972 | 0.7190 | 0.9921 |
|
| 342 |
-
| 0.0079 | 95.78 | 143000 | 0.0600 | 0.7407 | 0.6972 | 0.7183 | 0.9921 |
|
| 343 |
-
| 0.0097 | 96.12 | 143500 | 0.0599 | 0.7403 | 0.7009 | 0.7201 | 0.9921 |
|
| 344 |
-
| 0.0106 | 96.45 | 144000 | 0.0598 | 0.7427 | 0.7046 | 0.7232 | 0.9922 |
|
| 345 |
-
| 0.0077 | 96.78 | 144500 | 0.0598 | 0.7451 | 0.7028 | 0.7233 | 0.9922 |
|
| 346 |
-
| 0.0089 | 97.12 | 145000 | 0.0598 | 0.7437 | 0.7028 | 0.7226 | 0.9922 |
|
| 347 |
-
| 0.0092 | 97.45 | 145500 | 0.0599 | 0.7422 | 0.7028 | 0.7220 | 0.9922 |
|
| 348 |
-
| 0.009 | 97.79 | 146000 | 0.0599 | 0.7422 | 0.7028 | 0.7220 | 0.9922 |
|
| 349 |
-
| 0.0089 | 98.12 | 146500 | 0.0600 | 0.7442 | 0.7046 | 0.7238 | 0.9922 |
|
| 350 |
-
| 0.0093 | 98.46 | 147000 | 0.0600 | 0.7437 | 0.7028 | 0.7226 | 0.9922 |
|
| 351 |
-
| 0.009 | 98.79 | 147500 | 0.0600 | 0.7403 | 0.7009 | 0.7201 | 0.9922 |
|
| 352 |
-
| 0.0093 | 99.13 | 148000 | 0.0600 | 0.7408 | 0.7028 | 0.7213 | 0.9922 |
|
| 353 |
-
| 0.0087 | 99.46 | 148500 | 0.0600 | 0.7375 | 0.7009 | 0.7187 | 0.9922 |
|
| 354 |
-
| 0.0086 | 99.8 | 149000 | 0.0600 | 0.7375 | 0.7009 | 0.7187 | 0.9922 |
|
| 355 |
|
| 356 |
|
| 357 |
### Framework versions
|
|
|
|
| 8 |
- f1
|
| 9 |
- accuracy
|
| 10 |
model-index:
|
| 11 |
+
- name: Team_Gryffindor_NER
|
| 12 |
results: []
|
| 13 |
---
|
| 14 |
|
|
|
|
| 17 |
|
| 18 |
# Team-Gryffindor-distilbert-base-finetuned-NER-creditcardcontract-100epoch
|
| 19 |
|
| 20 |
+
This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the Credit card agreement dataset.
|
| 21 |
It achieves the following results on the evaluation set:
|
| 22 |
+
- Loss: 0.0470
|
| 23 |
+
- Precision: 0.7319
|
| 24 |
+
- Recall: 0.7064
|
| 25 |
+
- F1: 0.7190
|
| 26 |
+
- Accuracy: 0.9920
|
| 27 |
|
| 28 |
## Model description
|
| 29 |
|
|
|
|
| 48 |
- seed: 42
|
| 49 |
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
| 50 |
- lr_scheduler_type: linear
|
| 51 |
+
- num_epochs: 11
|
| 52 |
|
| 53 |
### Training results
|
| 54 |
|
|
|
|
| 85 |
| 0.0107 | 9.71 | 14500 | 0.0444 | 0.6840 | 0.6752 | 0.6796 | 0.9915 |
|
| 86 |
| 0.0118 | 10.05 | 15000 | 0.0457 | 0.7015 | 0.6771 | 0.6891 | 0.9918 |
|
| 87 |
| 0.0102 | 10.38 | 15500 | 0.0500 | 0.7413 | 0.6679 | 0.7027 | 0.9919 |
|
| 88 |
+
| 0.0107 | 10.72 | 16000 | 0.0470 | 0.7319 | 0.7064 | 0.7190 | 0.9920 |
|
| 89 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
|
| 91 |
|
| 92 |
### Framework versions
|