Hugging Face's logo

amrisaurus
/
pretrained-m-bert-90

Transformers
TensorFlow
bert
pretraining
generated_from_keras_callback
Model card Files
xet
 Community
amrisaurus commited on
Commit
a881a69
·
1 Parent(s): e1a36c3

Upload TFBertForPreTraining

Browse files
Files changed (3) hide show
  1. README.md +141 -0
  2. config.json +29 -0
  3. tf_model.h5 +3 -0
README.md ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_keras_callback
4
+ model-index:
5
+ - name: pretrained-m-bert-90
6
+ results: []
7
+ ---
8
+
9
+ <!-- This model card has been generated automatically according to the information Keras had access to. You should
10
+ probably proofread and complete it, then remove this comment. -->
11
+
12
+ # pretrained-m-bert-90
13
+
14
+ This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
15
+ It achieves the following results on the evaluation set:
16
+ - Train Loss: 5.7094
17
+ - Validation Loss: 14.5332
18
+ - Epoch: 89
19
+
20
+ ## Model description
21
+
22
+ More information needed
23
+
24
+ ## Intended uses & limitations
25
+
26
+ More information needed
27
+
28
+ ## Training and evaluation data
29
+
30
+ More information needed
31
+
32
+ ## Training procedure
33
+
34
+ ### Training hyperparameters
35
+
36
+ The following hyperparameters were used during training:
37
+ - optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 1e-04, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
38
+ - training_precision: float32
39
+
40
+ ### Training results
41
+
42
+ | Train Loss | Validation Loss | Epoch |
43
+ |:----------:|:---------------:|:-----:|
44
+ | 10.2413 | 10.9668 | 0 |
45
+ | 7.5814 | 10.9638 | 1 |
46
+ | 7.0095 | 11.3733 | 2 |
47
+ | 6.4352 | 11.5989 | 3 |
48
+ | 6.7137 | 11.4072 | 4 |
49
+ | 6.4383 | 11.8287 | 5 |
50
+ | 6.2223 | 12.0344 | 6 |
51
+ | 6.1759 | 11.6900 | 7 |
52
+ | 6.0764 | 11.7144 | 8 |
53
+ | 5.8802 | 12.1089 | 9 |
54
+ | 6.0159 | 12.3456 | 10 |
55
+ | 5.9254 | 12.7065 | 11 |
56
+ | 5.6652 | nan | 12 |
57
+ | 5.8185 | 12.8155 | 13 |
58
+ | 5.9185 | 12.7047 | 14 |
59
+ | 5.8418 | 12.7175 | 15 |
60
+ | 5.9122 | 12.5688 | 16 |
61
+ | 5.9698 | 12.5251 | 17 |
62
+ | 5.8286 | 12.7015 | 18 |
63
+ | 5.8807 | 13.2514 | 19 |
64
+ | 5.8330 | 12.8541 | 20 |
65
+ | 5.6456 | 13.4088 | 21 |
66
+ | 5.7257 | 13.5517 | 22 |
67
+ | 5.8854 | 12.8775 | 23 |
68
+ | 5.6770 | 13.6499 | 24 |
69
+ | 5.6026 | 13.9732 | 25 |
70
+ | 5.6651 | 13.0827 | 26 |
71
+ | 5.8888 | 13.1292 | 27 |
72
+ | 5.8123 | 12.8970 | 28 |
73
+ | 5.7525 | 13.3724 | 29 |
74
+ | 5.9020 | 13.5507 | 30 |
75
+ | 5.8642 | 13.3284 | 31 |
76
+ | 5.9329 | 13.7350 | 32 |
77
+ | 5.7728 | 13.3011 | 33 |
78
+ | 5.8297 | 13.6108 | 34 |
79
+ | 5.8118 | 13.3331 | 35 |
80
+ | 5.7382 | 13.7047 | 36 |
81
+ | 5.8061 | 13.8107 | 37 |
82
+ | 5.8423 | 13.4207 | 38 |
83
+ | 5.8442 | 13.6832 | 39 |
84
+ | 5.7680 | 14.1248 | 40 |
85
+ | 5.7668 | 13.6626 | 41 |
86
+ | 5.7826 | 13.6470 | 42 |
87
+ | 5.7692 | 13.9430 | 43 |
88
+ | 5.5109 | 14.0924 | 44 |
89
+ | 5.7394 | 14.0253 | 45 |
90
+ | 5.8013 | 13.5926 | 46 |
91
+ | 5.7222 | 13.9732 | 47 |
92
+ | 5.7023 | 14.0204 | 48 |
93
+ | 5.8250 | 13.9655 | 49 |
94
+ | 5.6064 | 14.0406 | 50 |
95
+ | 5.7319 | 14.1826 | 51 |
96
+ | 5.6849 | 13.9114 | 52 |
97
+ | 5.8167 | 13.9917 | 53 |
98
+ | 5.7573 | 14.1509 | 54 |
99
+ | 5.6921 | 14.3722 | 55 |
100
+ | 5.7190 | 14.4919 | 56 |
101
+ | 5.8501 | 13.6970 | 57 |
102
+ | 5.7627 | 14.1393 | 58 |
103
+ | 5.8031 | 14.1246 | 59 |
104
+ | 5.7207 | 14.3084 | 60 |
105
+ | 5.7979 | 13.9398 | 61 |
106
+ | 5.7068 | 14.2865 | 62 |
107
+ | 5.7547 | 14.2590 | 63 |
108
+ | 5.8349 | 14.1481 | 64 |
109
+ | 5.7924 | 14.0461 | 65 |
110
+ | 5.8127 | 14.1274 | 66 |
111
+ | 5.7590 | 14.3578 | 67 |
112
+ | 5.8297 | 14.2429 | 68 |
113
+ | 5.7822 | 14.2742 | 69 |
114
+ | 5.7708 | 14.3720 | 70 |
115
+ | 5.6521 | 14.8640 | 71 |
116
+ | 5.7253 | 14.4404 | 72 |
117
+ | 5.8076 | 14.1843 | 73 |
118
+ | 5.7746 | 14.4657 | 74 |
119
+ | 5.8592 | 14.2965 | 75 |
120
+ | 5.6643 | 14.0996 | 76 |
121
+ | 5.7849 | 14.3531 | 77 |
122
+ | 5.7418 | 14.4266 | 78 |
123
+ | 5.7030 | 14.5584 | 79 |
124
+ | 5.8298 | 14.1390 | 80 |
125
+ | 5.9061 | 13.9172 | 81 |
126
+ | 5.6570 | 14.6991 | 82 |
127
+ | 5.7040 | 14.7839 | 83 |
128
+ | 5.8064 | 14.2581 | 84 |
129
+ | 5.6855 | 14.4449 | 85 |
130
+ | 5.7803 | 14.7469 | 86 |
131
+ | 5.7495 | 14.4704 | 87 |
132
+ | 5.7539 | 14.5520 | 88 |
133
+ | 5.7094 | 14.5332 | 89 |
134
+
135
+
136
+ ### Framework versions
137
+
138
+ - Transformers 4.27.0.dev0
139
+ - TensorFlow 2.11.0
140
+ - Datasets 2.9.0
141
+ - Tokenizers 0.13.2
config.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "BertForPreTraining"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.1,
6
+ "classifier_dropout": null,
7
+ "directionality": "bidi",
8
+ "hidden_act": "gelu",
9
+ "hidden_dropout_prob": 0.1,
10
+ "hidden_size": 768,
11
+ "initializer_range": 0.02,
12
+ "intermediate_size": 3072,
13
+ "layer_norm_eps": 1e-12,
14
+ "max_position_embeddings": 512,
15
+ "model_type": "bert",
16
+ "num_attention_heads": 12,
17
+ "num_hidden_layers": 12,
18
+ "pad_token_id": 0,
19
+ "pooler_fc_size": 768,
20
+ "pooler_num_attention_heads": 12,
21
+ "pooler_num_fc_layers": 3,
22
+ "pooler_size_per_head": 128,
23
+ "pooler_type": "first_token_transform",
24
+ "position_embedding_type": "absolute",
25
+ "transformers_version": "4.27.0.dev0",
26
+ "type_vocab_size": 2,
27
+ "use_cache": true,
28
+ "vocab_size": 119547
29
+ }
tf_model.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79399a00574c19cba1b872f606b096b0f2c7116f25c9439877edfe36d8edee1e
3
+ size 1083389236