Hugging Face's logo

amrisaurus
/
pretrained-bert-uncased-90

Transformers
TensorFlow
bert
pretraining
generated_from_keras_callback
Model card Files
xet
Community
amrisaurus commited on
Commit
cdc06bb
·
1 Parent(s): 0c0a246

Upload TFBertForPreTraining

Browse files
Files changed (3) hide show
  1. README.md +97 -3
  2. config.json +1 -2
  3. tf_model.h5 +2 -2
README.md CHANGED
@@ -11,9 +11,11 @@ probably proofread and complete it, then remove this comment. -->
11
 
12
  # pretrained-bert-uncased-90
13
 
14
- This model is a fine-tuned version of [amrisaurus/pretrained-bert-uncased-90](https://huggingface.co/amrisaurus/pretrained-bert-uncased-90) on an unknown dataset.
15
  It achieves the following results on the evaluation set:
16
-
 
 
17
 
18
  ## Model description
19
 
@@ -32,11 +34,103 @@ More information needed
32
  ### Training hyperparameters
33
 
34
  The following hyperparameters were used during training:
35
- - optimizer: None
36
  - training_precision: float32
37
 
38
  ### Training results
39
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
 
41
 
42
  ### Framework versions
 
11
 
12
  # pretrained-bert-uncased-90
13
 
14
+ This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
15
  It achieves the following results on the evaluation set:
16
+ - Train Loss: 5.5801
17
+ - Validation Loss: 13.6573
18
+ - Epoch: 89
19
 
20
  ## Model description
21
 
 
34
  ### Training hyperparameters
35
 
36
  The following hyperparameters were used during training:
37
+ - optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
38
  - training_precision: float32
39
 
40
  ### Training results
41
 
42
+ | Train Loss | Validation Loss | Epoch |
43
+ |:----------:|:---------------:|:-----:|
44
+ | 8.8978 | 9.5686 | 0 |
45
+ | 7.0524 | 9.6480 | 1 |
46
+ | 6.8578 | 10.5054 | 2 |
47
+ | 6.1054 | 10.4137 | 3 |
48
+ | 6.1268 | 10.4515 | 4 |
49
+ | 5.8614 | 10.4313 | 5 |
50
+ | 5.9680 | 10.7224 | 6 |
51
+ | 5.7868 | 11.2948 | 7 |
52
+ | 5.5465 | 10.7112 | 8 |
53
+ | 5.7115 | 10.8543 | 9 |
54
+ | 5.7908 | 11.6466 | 10 |
55
+ | 5.5664 | 11.5085 | 11 |
56
+ | 5.5865 | 11.4894 | 12 |
57
+ | 5.6421 | 11.2182 | 13 |
58
+ | 5.6626 | 11.4446 | 14 |
59
+ | 5.4587 | 11.2814 | 15 |
60
+ | 5.5299 | 11.6601 | 16 |
61
+ | 5.5408 | 12.0485 | 17 |
62
+ | 5.5092 | 11.9469 | 18 |
63
+ | 5.6606 | 12.4353 | 19 |
64
+ | 5.7420 | 12.7461 | 20 |
65
+ | 5.6078 | 12.1650 | 21 |
66
+ | 5.6612 | 12.2811 | 22 |
67
+ | 5.7503 | 12.4086 | 23 |
68
+ | 5.5609 | 12.6149 | 24 |
69
+ | 5.4806 | 12.4447 | 25 |
70
+ | 5.6898 | 12.8078 | 26 |
71
+ | 5.6168 | 12.4649 | 27 |
72
+ | 5.6292 | 12.5851 | 28 |
73
+ | 5.8481 | 12.5146 | 29 |
74
+ | 5.6491 | 12.6358 | 30 |
75
+ | 5.5755 | 12.6996 | 31 |
76
+ | 5.8218 | 12.7957 | 32 |
77
+ | 5.5641 | 13.1650 | 33 |
78
+ | 5.6044 | 12.5065 | 34 |
79
+ | 5.6762 | 12.3722 | 35 |
80
+ | 5.5931 | 12.7162 | 36 |
81
+ | 5.5727 | 12.6179 | 37 |
82
+ | 5.5761 | 12.9479 | 38 |
83
+ | 5.6360 | 13.0610 | 39 |
84
+ | 5.4503 | 13.0441 | 40 |
85
+ | 5.5689 | 13.1673 | 41 |
86
+ | 5.6327 | 13.2184 | 42 |
87
+ | 5.5567 | 12.8114 | 43 |
88
+ | 5.6322 | 13.1793 | 44 |
89
+ | 5.4677 | 13.1324 | 45 |
90
+ | 5.5865 | 13.2891 | 46 |
91
+ | 5.5352 | 13.5036 | 47 |
92
+ | 5.4867 | 13.5010 | 48 |
93
+ | 5.6926 | 13.1743 | 49 |
94
+ | 5.7545 | 13.1689 | 50 |
95
+ | 5.5422 | 13.3362 | 51 |
96
+ | 5.6094 | 13.3983 | 52 |
97
+ | 5.5993 | 13.3638 | 53 |
98
+ | 5.6803 | 13.3884 | 54 |
99
+ | 5.6102 | 12.7277 | 55 |
100
+ | 5.7204 | 13.1669 | 56 |
101
+ | 5.5271 | 13.5684 | 57 |
102
+ | 5.5265 | 13.5086 | 58 |
103
+ | 5.5679 | 13.8641 | 59 |
104
+ | 5.6738 | 13.1735 | 60 |
105
+ | 5.5423 | 13.3285 | 61 |
106
+ | 5.5020 | 13.6262 | 62 |
107
+ | 5.5065 | 13.4765 | 63 |
108
+ | 5.5919 | 13.5598 | 64 |
109
+ | 5.5684 | 13.1651 | 65 |
110
+ | 5.6378 | 13.4781 | 66 |
111
+ | 5.6661 | 13.0726 | 67 |
112
+ | 5.7996 | 13.6267 | 68 |
113
+ | 5.7453 | 13.4608 | 69 |
114
+ | 5.5720 | 13.3663 | 70 |
115
+ | 5.4926 | 13.6905 | 71 |
116
+ | 5.7386 | 13.5941 | 72 |
117
+ | 5.6016 | 13.3110 | 73 |
118
+ | 5.5905 | 14.0529 | 74 |
119
+ | 5.7030 | 13.7322 | 75 |
120
+ | 5.6801 | 13.4712 | 76 |
121
+ | 5.6202 | 13.7954 | 77 |
122
+ | 5.6230 | 13.8177 | 78 |
123
+ | 5.6288 | 13.4887 | 79 |
124
+ | 5.6207 | 13.5817 | 80 |
125
+ | 5.5904 | 13.7643 | 81 |
126
+ | 5.6685 | 14.1648 | 82 |
127
+ | 5.5031 | 14.1816 | 83 |
128
+ | 5.6752 | 13.9170 | 84 |
129
+ | 5.6140 | 13.6953 | 85 |
130
+ | 5.6929 | 13.4916 | 86 |
131
+ | 5.4762 | 13.8740 | 87 |
132
+ | 5.6537 | 13.9725 | 88 |
133
+ | 5.5801 | 13.6573 | 89 |
134
 
135
 
136
  ### Framework versions
config.json CHANGED
@@ -1,7 +1,6 @@
1
  {
2
- "_name_or_path": "amrisaurus/pretrained-bert-uncased-90",
3
  "architectures": [
4
- "BertForSequenceClassification"
5
  ],
6
  "attention_probs_dropout_prob": 0.1,
7
  "classifier_dropout": null,
 
1
  {
 
2
  "architectures": [
3
+ "BertForPreTraining"
4
  ],
5
  "attention_probs_dropout_prob": 0.1,
6
  "classifier_dropout": null,
tf_model.h5 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0c199e6169b69011a77e14265c165b809dcc61f7b30fcab239f1f4e2b99822b6
3
- size 438223192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eab3d761f92c0bbb5f7ccb9a4e9dbd5a55c7bf7ed1bfac51ecb9858f2ad2a368
3
+ size 536063536