Hugging Face's logo

ManojAlexender
/
final_roberta_with_new_400k_plus_37k

Text Classification
Transformers
Safetensors
roberta
Generated from Trainer
text-embeddings-inference
Model card Files
xet
 Community
ManojAlexender commited on
Commit
cf587bc
·
verified ·
1 Parent(s): c652ed0

End of training

Browse files
Files changed (9) hide show
  1. README.md +150 -0
  2. config.json +38 -0
  3. merges.txt +0 -0
  4. model.safetensors +3 -0
  5. special_tokens_map.json +51 -0
  6. tokenizer.json +0 -0
  7. tokenizer_config.json +57 -0
  8. training_args.bin +3 -0
  9. vocab.json +0 -0
README.md ADDED
@@ -0,0 +1,150 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ metrics:
5
+ - accuracy
6
+ - f1
7
+ - precision
8
+ - recall
9
+ model-index:
10
+ - name: final_roberta_with_new_400k_plus_37k
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # final_roberta_with_new_400k_plus_37k
18
+
19
+ This model was trained from scratch on an unknown dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 0.2668
22
+ - Accuracy: 0.9031
23
+ - F1: 0.9027
24
+ - Precision: 0.9042
25
+ - Recall: 0.9031
26
+
27
+ ## Model description
28
+
29
+ More information needed
30
+
31
+ ## Intended uses & limitations
32
+
33
+ More information needed
34
+
35
+ ## Training and evaluation data
36
+
37
+ More information needed
38
+
39
+ ## Training procedure
40
+
41
+ ### Training hyperparameters
42
+
43
+ The following hyperparameters were used during training:
44
+ - learning_rate: 5e-05
45
+ - train_batch_size: 16
46
+ - eval_batch_size: 64
47
+ - seed: 42
48
+ - gradient_accumulation_steps: 2
49
+ - total_train_batch_size: 32
50
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
+ - lr_scheduler_type: linear
52
+ - lr_scheduler_warmup_steps: 500
53
+ - num_epochs: 3
54
+ - mixed_precision_training: Native AMP
55
+
56
+ ### Training results
57
+
58
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Precision | Recall |
59
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:---------:|:------:|
60
+ | 0.3174 | 0.01 | 100 | 0.3254 | 0.8928 | 0.8919 | 0.8964 | 0.8928 |
61
+ | 0.3285 | 0.01 | 200 | 0.2578 | 0.8956 | 0.8951 | 0.8968 | 0.8956 |
62
+ | 0.247 | 0.02 | 300 | 0.3913 | 0.8621 | 0.8588 | 0.8783 | 0.8621 |
63
+ | 0.2853 | 0.03 | 400 | 0.3394 | 0.8737 | 0.8711 | 0.8871 | 0.8737 |
64
+ | 0.3031 | 0.04 | 500 | 0.3924 | 0.8537 | 0.8491 | 0.8770 | 0.8537 |
65
+ | 0.2747 | 0.04 | 600 | 0.2532 | 0.9079 | 0.9079 | 0.9080 | 0.9079 |
66
+ | 0.2797 | 0.05 | 700 | 0.3607 | 0.8637 | 0.8607 | 0.8781 | 0.8637 |
67
+ | 0.2211 | 0.06 | 800 | 0.2910 | 0.8880 | 0.8872 | 0.8909 | 0.8880 |
68
+ | 0.2769 | 0.06 | 900 | 0.2834 | 0.8824 | 0.8810 | 0.8884 | 0.8824 |
69
+ | 0.2412 | 0.07 | 1000 | 0.2394 | 0.9063 | 0.9061 | 0.9069 | 0.9063 |
70
+ | 0.3386 | 0.08 | 1100 | 0.2400 | 0.9016 | 0.9013 | 0.9020 | 0.9016 |
71
+ | 0.2743 | 0.09 | 1200 | 0.2421 | 0.9047 | 0.9048 | 0.9048 | 0.9047 |
72
+ | 0.2682 | 0.09 | 1300 | 0.2833 | 0.8768 | 0.8752 | 0.8839 | 0.8768 |
73
+ | 0.3219 | 0.1 | 1400 | 0.2383 | 0.9071 | 0.9070 | 0.9071 | 0.9071 |
74
+ | 0.2211 | 0.11 | 1500 | 0.2454 | 0.9047 | 0.9047 | 0.9047 | 0.9047 |
75
+ | 0.2606 | 0.11 | 1600 | 0.2083 | 0.9223 | 0.9221 | 0.9228 | 0.9223 |
76
+ | 0.1966 | 0.12 | 1700 | 0.2688 | 0.9004 | 0.9001 | 0.9007 | 0.9004 |
77
+ | 0.2205 | 0.13 | 1800 | 0.3076 | 0.8776 | 0.8752 | 0.8911 | 0.8776 |
78
+ | 0.2242 | 0.14 | 1900 | 0.2171 | 0.9151 | 0.9150 | 0.9153 | 0.9151 |
79
+ | 0.257 | 0.14 | 2000 | 0.2643 | 0.8912 | 0.8905 | 0.8932 | 0.8912 |
80
+ | 0.2238 | 0.15 | 2100 | 0.2165 | 0.9131 | 0.9128 | 0.9141 | 0.9131 |
81
+ | 0.2313 | 0.16 | 2200 | 0.2312 | 0.8996 | 0.8996 | 0.8996 | 0.8996 |
82
+ | 0.1856 | 0.16 | 2300 | 0.2269 | 0.9107 | 0.9108 | 0.9109 | 0.9107 |
83
+ | 0.2201 | 0.17 | 2400 | 0.2425 | 0.9059 | 0.9056 | 0.9065 | 0.9059 |
84
+ | 0.3332 | 0.18 | 2500 | 0.2254 | 0.9043 | 0.9044 | 0.9048 | 0.9043 |
85
+ | 0.1843 | 0.19 | 2600 | 0.2524 | 0.8980 | 0.8971 | 0.9020 | 0.8980 |
86
+ | 0.2728 | 0.19 | 2700 | 0.2348 | 0.8968 | 0.8957 | 0.9017 | 0.8968 |
87
+ | 0.2131 | 0.2 | 2800 | 0.2210 | 0.9135 | 0.9136 | 0.9138 | 0.9135 |
88
+ | 0.19 | 0.21 | 2900 | 0.2259 | 0.9123 | 0.9120 | 0.9130 | 0.9123 |
89
+ | 0.2099 | 0.21 | 3000 | 0.2814 | 0.9024 | 0.9016 | 0.9054 | 0.9024 |
90
+ | 0.2209 | 0.22 | 3100 | 0.2473 | 0.9051 | 0.9046 | 0.9070 | 0.9051 |
91
+ | 0.2366 | 0.23 | 3200 | 0.2561 | 0.8992 | 0.8983 | 0.9029 | 0.8992 |
92
+ | 0.3156 | 0.24 | 3300 | 0.2192 | 0.9095 | 0.9094 | 0.9095 | 0.9095 |
93
+ | 0.197 | 0.24 | 3400 | 0.2382 | 0.9063 | 0.9057 | 0.9093 | 0.9063 |
94
+ | 0.2371 | 0.25 | 3500 | 0.2243 | 0.9139 | 0.9141 | 0.9166 | 0.9139 |
95
+ | 0.2273 | 0.26 | 3600 | 0.2362 | 0.9135 | 0.9131 | 0.9153 | 0.9135 |
96
+ | 0.2504 | 0.26 | 3700 | 0.2671 | 0.8888 | 0.8873 | 0.8965 | 0.8888 |
97
+ | 0.1978 | 0.27 | 3800 | 0.2049 | 0.9171 | 0.9170 | 0.9172 | 0.9171 |
98
+ | 0.2189 | 0.28 | 3900 | 0.2268 | 0.9099 | 0.9099 | 0.9099 | 0.9099 |
99
+ | 0.2171 | 0.29 | 4000 | 0.2135 | 0.9163 | 0.9162 | 0.9164 | 0.9163 |
100
+ | 0.2325 | 0.29 | 4100 | 0.2624 | 0.8916 | 0.8905 | 0.8966 | 0.8916 |
101
+ | 0.1888 | 0.3 | 4200 | 0.2878 | 0.8924 | 0.8911 | 0.8987 | 0.8924 |
102
+ | 0.2345 | 0.31 | 4300 | 0.2444 | 0.8964 | 0.8953 | 0.9013 | 0.8964 |
103
+ | 0.1688 | 0.31 | 4400 | 0.2479 | 0.9083 | 0.9077 | 0.9109 | 0.9083 |
104
+ | 0.2083 | 0.32 | 4500 | 0.2200 | 0.9135 | 0.9131 | 0.9150 | 0.9135 |
105
+ | 0.2475 | 0.33 | 4600 | 0.2353 | 0.9035 | 0.9030 | 0.9052 | 0.9035 |
106
+ | 0.1928 | 0.34 | 4700 | 0.2987 | 0.8944 | 0.8933 | 0.8992 | 0.8944 |
107
+ | 0.2008 | 0.34 | 4800 | 0.2993 | 0.8760 | 0.8735 | 0.8897 | 0.8760 |
108
+ | 0.22 | 0.35 | 4900 | 0.2431 | 0.9035 | 0.9033 | 0.9039 | 0.9035 |
109
+ | 0.1844 | 0.36 | 5000 | 0.2590 | 0.9171 | 0.9171 | 0.9171 | 0.9171 |
110
+ | 0.2235 | 0.36 | 5100 | 0.2421 | 0.9047 | 0.9041 | 0.9072 | 0.9047 |
111
+ | 0.2222 | 0.37 | 5200 | 0.2958 | 0.8948 | 0.8941 | 0.8973 | 0.8948 |
112
+ | 0.2241 | 0.38 | 5300 | 0.2031 | 0.9211 | 0.9209 | 0.9216 | 0.9211 |
113
+ | 0.2307 | 0.39 | 5400 | 0.2277 | 0.9043 | 0.9036 | 0.9076 | 0.9043 |
114
+ | 0.1926 | 0.39 | 5500 | 0.2817 | 0.8900 | 0.8887 | 0.8959 | 0.8900 |
115
+ | 0.2119 | 0.4 | 5600 | 0.2151 | 0.9175 | 0.9174 | 0.9176 | 0.9175 |
116
+ | 0.1747 | 0.41 | 5700 | 0.2404 | 0.9123 | 0.9121 | 0.9126 | 0.9123 |
117
+ | 0.1809 | 0.41 | 5800 | 0.3013 | 0.8920 | 0.8908 | 0.8980 | 0.8920 |
118
+ | 0.1748 | 0.42 | 5900 | 0.3084 | 0.9063 | 0.9056 | 0.9097 | 0.9063 |
119
+ | 0.2101 | 0.43 | 6000 | 0.2129 | 0.9175 | 0.9173 | 0.9180 | 0.9175 |
120
+ | 0.202 | 0.44 | 6100 | 0.3794 | 0.8848 | 0.8834 | 0.8914 | 0.8848 |
121
+ | 0.1671 | 0.44 | 6200 | 0.2678 | 0.9043 | 0.9041 | 0.9046 | 0.9043 |
122
+ | 0.2808 | 0.45 | 6300 | 0.2613 | 0.9075 | 0.9070 | 0.9098 | 0.9075 |
123
+ | 0.2853 | 0.46 | 6400 | 0.2270 | 0.9087 | 0.9088 | 0.9088 | 0.9087 |
124
+ | 0.187 | 0.46 | 6500 | 0.2400 | 0.9111 | 0.9112 | 0.9115 | 0.9111 |
125
+ | 0.1382 | 0.47 | 6600 | 0.2454 | 0.9139 | 0.9136 | 0.9146 | 0.9139 |
126
+ | 0.2259 | 0.48 | 6700 | 0.3165 | 0.8904 | 0.8890 | 0.8976 | 0.8904 |
127
+ | 0.164 | 0.49 | 6800 | 0.3091 | 0.9031 | 0.9023 | 0.9074 | 0.9031 |
128
+ | 0.2557 | 0.49 | 6900 | 0.2708 | 0.9024 | 0.9015 | 0.9064 | 0.9024 |
129
+ | 0.1586 | 0.5 | 7000 | 0.2139 | 0.9247 | 0.9246 | 0.9247 | 0.9247 |
130
+ | 0.2391 | 0.51 | 7100 | 0.2087 | 0.9143 | 0.9141 | 0.9147 | 0.9143 |
131
+ | 0.1974 | 0.51 | 7200 | 0.2438 | 0.9171 | 0.9171 | 0.9171 | 0.9171 |
132
+ | 0.2507 | 0.52 | 7300 | 0.2323 | 0.9051 | 0.9044 | 0.9084 | 0.9051 |
133
+ | 0.226 | 0.53 | 7400 | 0.2465 | 0.9063 | 0.9056 | 0.9096 | 0.9063 |
134
+ | 0.1859 | 0.54 | 7500 | 0.2762 | 0.8960 | 0.8949 | 0.9012 | 0.8960 |
135
+ | 0.2208 | 0.54 | 7600 | 0.2705 | 0.8948 | 0.8937 | 0.9002 | 0.8948 |
136
+ | 0.2073 | 0.55 | 7700 | 0.2419 | 0.9008 | 0.9006 | 0.9007 | 0.9008 |
137
+ | 0.1557 | 0.56 | 7800 | 0.3004 | 0.8900 | 0.8899 | 0.8899 | 0.8900 |
138
+ | 0.1872 | 0.56 | 7900 | 0.2520 | 0.9059 | 0.9056 | 0.9066 | 0.9059 |
139
+ | 0.1749 | 0.57 | 8000 | 0.2757 | 0.9067 | 0.9067 | 0.9067 | 0.9067 |
140
+ | 0.2298 | 0.58 | 8100 | 0.2617 | 0.9075 | 0.9071 | 0.9092 | 0.9075 |
141
+ | 0.1781 | 0.59 | 8200 | 0.2380 | 0.9139 | 0.9137 | 0.9144 | 0.9139 |
142
+ | 0.1448 | 0.59 | 8300 | 0.2668 | 0.9031 | 0.9027 | 0.9042 | 0.9031 |
143
+
144
+
145
+ ### Framework versions
146
+
147
+ - Transformers 4.39.3
148
+ - Pytorch 2.1.2+cu121
149
+ - Datasets 2.16.1
150
+ - Tokenizers 0.15.1
config.json ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "/home/manojale/Documents/Performance_Dataset/roberta-base_MLM",
3
+ "architectures": [
4
+ "RobertaForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "classifier_dropout": null,
9
+ "eos_token_id": 2,
10
+ "hidden_act": "gelu",
11
+ "hidden_dropout_prob": 0.1,
12
+ "hidden_size": 768,
13
+ "id2label": {
14
+ "0": "LABEL_0",
15
+ "1": "LABEL_1",
16
+ "2": "LABEL_2"
17
+ },
18
+ "initializer_range": 0.02,
19
+ "intermediate_size": 3072,
20
+ "label2id": {
21
+ "LABEL_0": 0,
22
+ "LABEL_1": 1,
23
+ "LABEL_2": 2
24
+ },
25
+ "layer_norm_eps": 1e-05,
26
+ "max_position_embeddings": 514,
27
+ "model_type": "roberta",
28
+ "num_attention_heads": 12,
29
+ "num_hidden_layers": 12,
30
+ "pad_token_id": 1,
31
+ "position_embedding_type": "absolute",
32
+ "problem_type": "single_label_classification",
33
+ "torch_dtype": "float32",
34
+ "transformers_version": "4.39.3",
35
+ "type_vocab_size": 1,
36
+ "use_cache": true,
37
+ "vocab_size": 50265
38
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab6a98341a85a32efb1d70933eb4a246a92f88abcdd04d184c6084f2915c9296
3
+ size 498615900
special_tokens_map.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": true,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "cls_token": {
10
+ "content": "<s>",
11
+ "lstrip": false,
12
+ "normalized": true,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "eos_token": {
17
+ "content": "</s>",
18
+ "lstrip": false,
19
+ "normalized": true,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "mask_token": {
24
+ "content": "<mask>",
25
+ "lstrip": true,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "pad_token": {
31
+ "content": "<pad>",
32
+ "lstrip": false,
33
+ "normalized": true,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ },
37
+ "sep_token": {
38
+ "content": "</s>",
39
+ "lstrip": false,
40
+ "normalized": true,
41
+ "rstrip": false,
42
+ "single_word": false
43
+ },
44
+ "unk_token": {
45
+ "content": "<unk>",
46
+ "lstrip": false,
47
+ "normalized": true,
48
+ "rstrip": false,
49
+ "single_word": false
50
+ }
51
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<s>",
6
+ "lstrip": false,
7
+ "normalized": true,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<pad>",
14
+ "lstrip": false,
15
+ "normalized": true,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "2": {
21
+ "content": "</s>",
22
+ "lstrip": false,
23
+ "normalized": true,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "3": {
29
+ "content": "<unk>",
30
+ "lstrip": false,
31
+ "normalized": true,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "50264": {
37
+ "content": "<mask>",
38
+ "lstrip": true,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ }
44
+ },
45
+ "bos_token": "<s>",
46
+ "clean_up_tokenization_spaces": true,
47
+ "cls_token": "<s>",
48
+ "eos_token": "</s>",
49
+ "errors": "replace",
50
+ "mask_token": "<mask>",
51
+ "model_max_length": 512,
52
+ "pad_token": "<pad>",
53
+ "sep_token": "</s>",
54
+ "tokenizer_class": "RobertaTokenizer",
55
+ "trim_offsets": true,
56
+ "unk_token": "<unk>"
57
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6094449e535a3d27a71f8c5d156515b276b5d19fc2deb22b95f61c15444865c
3
+ size 4856
vocab.json ADDED
The diff for this file is too large to render. See raw diff