Hugging Face's logo

pradeep4321
/
valve_model

Text Generation
Transformers
TensorFlow
gpt2
generated_from_keras_callback
Model card Files
xet
Community
pradeep4321 commited on
Commit
56d1620
·
1 Parent(s): a09f698

Upload model

Browse files
Files changed (4) hide show
  1. README.md +152 -0
  2. config.json +38 -0
  3. generation_config.json +6 -0
  4. tf_model.h5 +3 -0
README.md ADDED
@@ -0,0 +1,152 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - generated_from_keras_callback
5
+ model-index:
6
+ - name: valve_model
7
+ results: []
8
+ ---
9
+
10
+ <!-- This model card has been generated automatically according to the information Keras had access to. You should
11
+ probably proofread and complete it, then remove this comment. -->
12
+
13
+ # valve_model
14
+
15
+ This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unknown dataset.
16
+ It achieves the following results on the evaluation set:
17
+ - Train Loss: 3.1831
18
+ - Validation Loss: 5.9072
19
+ - Epoch: 99
20
+
21
+ ## Model description
22
+
23
+ More information needed
24
+
25
+ ## Intended uses & limitations
26
+
27
+ More information needed
28
+
29
+ ## Training and evaluation data
30
+
31
+ More information needed
32
+
33
+ ## Training procedure
34
+
35
+ ### Training hyperparameters
36
+
37
+ The following hyperparameters were used during training:
38
+ - optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 5e-05, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5e-05, 'decay_steps': -999, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, '__passive_serialization__': True}, 'warmup_steps': 1000, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
39
+ - training_precision: mixed_float16
40
+
41
+ ### Training results
42
+
43
+ | Train Loss | Validation Loss | Epoch |
44
+ |:----------:|:---------------:|:-----:|
45
+ | 6.4756 | 6.5449 | 0 |
46
+ | 6.5200 | 6.5443 | 1 |
47
+ | 6.6058 | 6.5432 | 2 |
48
+ | 6.5443 | 6.5416 | 3 |
49
+ | 6.6271 | 6.5395 | 4 |
50
+ | 6.5184 | 6.5369 | 5 |
51
+ | 6.4858 | 6.5337 | 6 |
52
+ | 6.4394 | 6.5301 | 7 |
53
+ | 6.5547 | 6.5259 | 8 |
54
+ | 6.4873 | 6.5211 | 9 |
55
+ | 6.4768 | 6.5160 | 10 |
56
+ | 6.5326 | 6.5102 | 11 |
57
+ | 6.4251 | 6.5041 | 12 |
58
+ | 6.4698 | 6.4974 | 13 |
59
+ | 6.4125 | 6.4903 | 14 |
60
+ | 6.3133 | 6.4827 | 15 |
61
+ | 6.3633 | 6.4748 | 16 |
62
+ | 6.3025 | 6.4665 | 17 |
63
+ | 6.3038 | 6.4579 | 18 |
64
+ | 6.2443 | 6.4490 | 19 |
65
+ | 6.2891 | 6.4398 | 20 |
66
+ | 6.1881 | 6.4304 | 21 |
67
+ | 6.0868 | 6.4208 | 22 |
68
+ | 6.1109 | 6.4110 | 23 |
69
+ | 6.1498 | 6.4010 | 24 |
70
+ | 5.9289 | 6.3908 | 25 |
71
+ | 6.0533 | 6.3804 | 26 |
72
+ | 6.0183 | 6.3699 | 27 |
73
+ | 5.8904 | 6.3593 | 28 |
74
+ | 5.8638 | 6.3486 | 29 |
75
+ | 5.8532 | 6.3379 | 30 |
76
+ | 5.8279 | 6.3273 | 31 |
77
+ | 5.7185 | 6.3167 | 32 |
78
+ | 5.7309 | 6.3062 | 33 |
79
+ | 5.6457 | 6.2958 | 34 |
80
+ | 5.5979 | 6.2855 | 35 |
81
+ | 5.5968 | 6.2750 | 36 |
82
+ | 5.6015 | 6.2646 | 37 |
83
+ | 5.4416 | 6.2542 | 38 |
84
+ | 5.5024 | 6.2441 | 39 |
85
+ | 5.4739 | 6.2342 | 40 |
86
+ | 5.3524 | 6.2245 | 41 |
87
+ | 5.3214 | 6.2149 | 42 |
88
+ | 5.2997 | 6.2052 | 43 |
89
+ | 5.2619 | 6.1955 | 44 |
90
+ | 5.2155 | 6.1859 | 45 |
91
+ | 5.2030 | 6.1765 | 46 |
92
+ | 5.1632 | 6.1672 | 47 |
93
+ | 5.1386 | 6.1581 | 48 |
94
+ | 5.0821 | 6.1492 | 49 |
95
+ | 5.0143 | 6.1406 | 50 |
96
+ | 5.0254 | 6.1318 | 51 |
97
+ | 4.9244 | 6.1235 | 52 |
98
+ | 4.8945 | 6.1151 | 53 |
99
+ | 4.9138 | 6.1066 | 54 |
100
+ | 4.8516 | 6.0985 | 55 |
101
+ | 4.8212 | 6.0899 | 56 |
102
+ | 4.6900 | 6.0817 | 57 |
103
+ | 4.7051 | 6.0738 | 58 |
104
+ | 4.6742 | 6.0657 | 59 |
105
+ | 4.6304 | 6.0577 | 60 |
106
+ | 4.5756 | 6.0498 | 61 |
107
+ | 4.5728 | 6.0420 | 62 |
108
+ | 4.5041 | 6.0341 | 63 |
109
+ | 4.5266 | 6.0264 | 64 |
110
+ | 4.4364 | 6.0193 | 65 |
111
+ | 4.3653 | 6.0125 | 66 |
112
+ | 4.3792 | 6.0054 | 67 |
113
+ | 4.3870 | 5.9979 | 68 |
114
+ | 4.2865 | 5.9911 | 69 |
115
+ | 4.2443 | 5.9845 | 70 |
116
+ | 4.2388 | 5.9783 | 71 |
117
+ | 4.2063 | 5.9723 | 72 |
118
+ | 4.1534 | 5.9667 | 73 |
119
+ | 4.0811 | 5.9616 | 74 |
120
+ | 4.1064 | 5.9565 | 75 |
121
+ | 4.1281 | 5.9512 | 76 |
122
+ | 4.0124 | 5.9467 | 77 |
123
+ | 3.9703 | 5.9430 | 78 |
124
+ | 3.8858 | 5.9389 | 79 |
125
+ | 3.9194 | 5.9351 | 80 |
126
+ | 3.8257 | 5.9309 | 81 |
127
+ | 3.8251 | 5.9270 | 82 |
128
+ | 3.8499 | 5.9234 | 83 |
129
+ | 3.7903 | 5.9206 | 84 |
130
+ | 3.7851 | 5.9190 | 85 |
131
+ | 3.7319 | 5.9174 | 86 |
132
+ | 3.6612 | 5.9169 | 87 |
133
+ | 3.6404 | 5.9162 | 88 |
134
+ | 3.5339 | 5.9162 | 89 |
135
+ | 3.5073 | 5.9158 | 90 |
136
+ | 3.4569 | 5.9157 | 91 |
137
+ | 3.5231 | 5.9153 | 92 |
138
+ | 3.3952 | 5.9152 | 93 |
139
+ | 3.3774 | 5.9144 | 94 |
140
+ | 3.3776 | 5.9126 | 95 |
141
+ | 3.2881 | 5.9112 | 96 |
142
+ | 3.2130 | 5.9099 | 97 |
143
+ | 3.2514 | 5.9088 | 98 |
144
+ | 3.1831 | 5.9072 | 99 |
145
+
146
+
147
+ ### Framework versions
148
+
149
+ - Transformers 4.29.0.dev0
150
+ - TensorFlow 2.9.1
151
+ - Datasets 2.5.1
152
+ - Tokenizers 0.13.3
config.json ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "gpt2",
3
+ "activation_function": "gelu_new",
4
+ "architectures": [
5
+ "GPT2LMHeadModel"
6
+ ],
7
+ "attn_pdrop": 0.1,
8
+ "bos_token_id": 0,
9
+ "embd_pdrop": 0.1,
10
+ "eos_token_id": 0,
11
+ "initializer_range": 0.02,
12
+ "layer_norm_epsilon": 1e-05,
13
+ "model_type": "gpt2",
14
+ "n_ctx": 40,
15
+ "n_embd": 768,
16
+ "n_head": 12,
17
+ "n_inner": null,
18
+ "n_layer": 12,
19
+ "n_positions": 1024,
20
+ "reorder_and_upcast_attn": false,
21
+ "resid_pdrop": 0.1,
22
+ "scale_attn_by_inverse_layer_idx": false,
23
+ "scale_attn_weights": true,
24
+ "summary_activation": null,
25
+ "summary_first_dropout": 0.1,
26
+ "summary_proj_to_labels": true,
27
+ "summary_type": "cls_index",
28
+ "summary_use_proj": true,
29
+ "task_specific_params": {
30
+ "text-generation": {
31
+ "do_sample": true,
32
+ "max_length": 50
33
+ }
34
+ },
35
+ "transformers_version": "4.29.0.dev0",
36
+ "use_cache": true,
37
+ "vocab_size": 665
38
+ }
generation_config.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 0,
4
+ "eos_token_id": 0,
5
+ "transformers_version": "4.29.0.dev0"
6
+ }
tf_model.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6311b4e3a92717f413ce150c3791712ecebed73ff280364d1b10eaf8e2f7226a
3
+ size 345588840