Hugging Face's logo

CIRCL
/
vulnerability-attack-technique-classification-roberta-base

Text Classification
Transformers
Safetensors
English
roberta
vulnerability
cybersecurity
security
cve
mitre-attack
attack-techniques
Generated from Trainer
text-embeddings-inference
Model card Files
xet
 Community
cedricbonhomme commited on
Commit
40ad73e
·
verified ·
1 Parent(s): f18da9c

End of training

Browse files
Files changed (8) hide show
  1. README.md +103 -0
  2. config.json +147 -0
  3. emissions.csv +2 -0
  4. metrics.json +13 -0
  5. model.safetensors +3 -0
  6. tokenizer.json +0 -0
  7. tokenizer_config.json +17 -0
  8. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,103 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: mit
4
+ base_model: roberta-base
5
+ tags:
6
+ - generated_from_trainer
7
+ model-index:
8
+ - name: vulnerability-attack-technique-classification-roberta-base
9
+ results: []
10
+ ---
11
+
12
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
+ should probably proofread and complete it, then remove this comment. -->
14
+
15
+ # vulnerability-attack-technique-classification-roberta-base
16
+
17
+ This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
18
+ It achieves the following results on the evaluation set:
19
+ - Loss: 0.6205
20
+ - F1 Micro: 0.4171
21
+ - F1 Macro: 0.2027
22
+ - Precision Micro: 0.3005
23
+ - Recall Micro: 0.6818
24
+ - Recall At 3: 0.4822
25
+ - Recall At 5: 0.6859
26
+
27
+ ## Model description
28
+
29
+ More information needed
30
+
31
+ ## Intended uses & limitations
32
+
33
+ More information needed
34
+
35
+ ## Training and evaluation data
36
+
37
+ More information needed
38
+
39
+ ## Training procedure
40
+
41
+ ### Training hyperparameters
42
+
43
+ The following hyperparameters were used during training:
44
+ - learning_rate: 1e-05
45
+ - train_batch_size: 64
46
+ - eval_batch_size: 64
47
+ - seed: 42
48
+ - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
49
+ - lr_scheduler_type: linear
50
+ - num_epochs: 40
51
+
52
+ ### Training results
53
+
54
+ | Training Loss | Epoch | Step | Validation Loss | F1 Micro | F1 Macro | Precision Micro | Recall Micro | Recall At 3 | Recall At 5 |
55
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:---------------:|:------------:|:-----------:|:-----------:|
56
+ | No log | 1.0 | 17 | 0.9141 | 0.1161 | 0.0311 | 0.0672 | 0.4280 | 0.2637 | 0.2959 |
57
+ | 0.9279 | 2.0 | 34 | 0.8081 | 0.2057 | 0.0305 | 0.1473 | 0.3409 | 0.3201 | 0.3587 |
58
+ | 0.8472 | 3.0 | 51 | 0.7795 | 0.2369 | 0.0429 | 0.1649 | 0.4205 | 0.3212 | 0.3859 |
59
+ | 0.7976 | 4.0 | 68 | 0.7677 | 0.2418 | 0.0375 | 0.1752 | 0.3902 | 0.3415 | 0.3950 |
60
+ | 0.7776 | 5.0 | 85 | 0.7555 | 0.2945 | 0.0616 | 0.2137 | 0.4735 | 0.3449 | 0.4593 |
61
+ | 0.7742 | 6.0 | 102 | 0.7483 | 0.2943 | 0.0702 | 0.2085 | 0.5 | 0.3530 | 0.4845 |
62
+ | 0.7742 | 7.0 | 119 | 0.7393 | 0.3159 | 0.0800 | 0.2199 | 0.5606 | 0.3599 | 0.5048 |
63
+ | 0.7523 | 8.0 | 136 | 0.7281 | 0.3246 | 0.0992 | 0.2212 | 0.6098 | 0.3553 | 0.5116 |
64
+ | 0.7418 | 9.0 | 153 | 0.7172 | 0.336 | 0.1068 | 0.2283 | 0.6364 | 0.4342 | 0.5668 |
65
+ | 0.7328 | 10.0 | 170 | 0.7078 | 0.3431 | 0.1230 | 0.2338 | 0.6439 | 0.4063 | 0.5864 |
66
+ | 0.7166 | 11.0 | 187 | 0.6942 | 0.3546 | 0.1306 | 0.2420 | 0.6629 | 0.4405 | 0.6021 |
67
+ | 0.6951 | 12.0 | 204 | 0.6858 | 0.3746 | 0.1355 | 0.2635 | 0.6477 | 0.4650 | 0.6256 |
68
+ | 0.6778 | 13.0 | 221 | 0.6750 | 0.3603 | 0.1339 | 0.2468 | 0.6667 | 0.4252 | 0.6116 |
69
+ | 0.6778 | 14.0 | 238 | 0.6698 | 0.3661 | 0.1390 | 0.2529 | 0.6629 | 0.4311 | 0.6200 |
70
+ | 0.6692 | 15.0 | 255 | 0.6623 | 0.3845 | 0.1412 | 0.2684 | 0.6780 | 0.4805 | 0.6796 |
71
+ | 0.6465 | 16.0 | 272 | 0.6591 | 0.3700 | 0.1510 | 0.2539 | 0.6818 | 0.4287 | 0.6547 |
72
+ | 0.6499 | 17.0 | 289 | 0.6528 | 0.4036 | 0.1567 | 0.2859 | 0.6856 | 0.4577 | 0.6653 |
73
+ | 0.6229 | 18.0 | 306 | 0.6505 | 0.4088 | 0.1634 | 0.2886 | 0.7008 | 0.4962 | 0.6785 |
74
+ | 0.6214 | 19.0 | 323 | 0.6489 | 0.3913 | 0.1485 | 0.2803 | 0.6477 | 0.4661 | 0.6453 |
75
+ | 0.6092 | 20.0 | 340 | 0.6484 | 0.3825 | 0.1475 | 0.2688 | 0.6629 | 0.4577 | 0.6627 |
76
+ | 0.6092 | 21.0 | 357 | 0.6409 | 0.4118 | 0.1652 | 0.2935 | 0.6894 | 0.5011 | 0.6852 |
77
+ | 0.5973 | 22.0 | 374 | 0.6419 | 0.3982 | 0.1585 | 0.2812 | 0.6818 | 0.4647 | 0.6502 |
78
+ | 0.6046 | 23.0 | 391 | 0.6368 | 0.4095 | 0.1664 | 0.2919 | 0.6856 | 0.4864 | 0.6670 |
79
+ | 0.5811 | 24.0 | 408 | 0.6395 | 0.3895 | 0.1560 | 0.2785 | 0.6477 | 0.4556 | 0.6572 |
80
+ | 0.5741 | 25.0 | 425 | 0.6310 | 0.4241 | 0.1740 | 0.3110 | 0.6667 | 0.5179 | 0.6775 |
81
+ | 0.5697 | 26.0 | 442 | 0.6309 | 0.4014 | 0.1657 | 0.2864 | 0.6705 | 0.4815 | 0.6642 |
82
+ | 0.5697 | 27.0 | 459 | 0.6292 | 0.4151 | 0.1680 | 0.3014 | 0.6667 | 0.4934 | 0.6880 |
83
+ | 0.5656 | 28.0 | 476 | 0.6289 | 0.4 | 0.1616 | 0.2901 | 0.6439 | 0.4579 | 0.6607 |
84
+ | 0.5540 | 29.0 | 493 | 0.6257 | 0.4009 | 0.1798 | 0.2874 | 0.6629 | 0.4710 | 0.6565 |
85
+ | 0.5506 | 30.0 | 510 | 0.6227 | 0.4110 | 0.1809 | 0.2941 | 0.6818 | 0.4815 | 0.6873 |
86
+ | 0.5547 | 31.0 | 527 | 0.6242 | 0.4164 | 0.1850 | 0.3012 | 0.6742 | 0.4773 | 0.6747 |
87
+ | 0.5395 | 32.0 | 544 | 0.6228 | 0.4182 | 0.1844 | 0.3024 | 0.6780 | 0.4661 | 0.6754 |
88
+ | 0.5407 | 33.0 | 561 | 0.6205 | 0.4171 | 0.2027 | 0.3005 | 0.6818 | 0.4822 | 0.6859 |
89
+ | 0.5407 | 34.0 | 578 | 0.6187 | 0.4218 | 0.2007 | 0.3038 | 0.6894 | 0.5274 | 0.6971 |
90
+ | 0.5368 | 35.0 | 595 | 0.6192 | 0.4162 | 0.2018 | 0.2995 | 0.6818 | 0.5025 | 0.6831 |
91
+ | 0.5342 | 36.0 | 612 | 0.6192 | 0.4176 | 0.1833 | 0.3010 | 0.6818 | 0.5134 | 0.6936 |
92
+ | 0.5339 | 37.0 | 629 | 0.6205 | 0.4126 | 0.1832 | 0.2980 | 0.6705 | 0.4775 | 0.6859 |
93
+ | 0.5254 | 38.0 | 646 | 0.6198 | 0.4175 | 0.1851 | 0.3031 | 0.6705 | 0.4817 | 0.6901 |
94
+ | 0.5318 | 39.0 | 663 | 0.6193 | 0.4159 | 0.1846 | 0.3007 | 0.6742 | 0.4859 | 0.6901 |
95
+ | 0.5279 | 40.0 | 680 | 0.6191 | 0.4159 | 0.1843 | 0.3007 | 0.6742 | 0.4831 | 0.6859 |
96
+
97
+
98
+ ### Framework versions
99
+
100
+ - Transformers 5.13.0
101
+ - Pytorch 2.12.1+cu130
102
+ - Datasets 4.8.5
103
+ - Tokenizers 0.22.2
config.json ADDED
@@ -0,0 +1,147 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_cross_attention": false,
3
+ "architectures": [
4
+ "RobertaForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "classifier_dropout": null,
9
+ "dtype": "float32",
10
+ "eos_token_id": 2,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout_prob": 0.1,
13
+ "hidden_size": 768,
14
+ "id2label": {
15
+ "0": "T1003",
16
+ "1": "T1005",
17
+ "2": "T1021",
18
+ "3": "T1027",
19
+ "4": "T1036",
20
+ "5": "T1040",
21
+ "6": "T1041",
22
+ "7": "T1046",
23
+ "8": "T1055",
24
+ "9": "T1059",
25
+ "10": "T1068",
26
+ "11": "T1070",
27
+ "12": "T1071",
28
+ "13": "T1078",
29
+ "14": "T1082",
30
+ "15": "T1083",
31
+ "16": "T1087",
32
+ "17": "T1091",
33
+ "18": "T1098",
34
+ "19": "T1105",
35
+ "20": "T1106",
36
+ "21": "T1110",
37
+ "22": "T1114",
38
+ "23": "T1133",
39
+ "24": "T1136",
40
+ "25": "T1185",
41
+ "26": "T1189",
42
+ "27": "T1190",
43
+ "28": "T1202",
44
+ "29": "T1203",
45
+ "30": "T1204",
46
+ "31": "T1210",
47
+ "32": "T1211",
48
+ "33": "T1212",
49
+ "34": "T1485",
50
+ "35": "T1486",
51
+ "36": "T1489",
52
+ "37": "T1496",
53
+ "38": "T1497",
54
+ "39": "T1498",
55
+ "40": "T1499",
56
+ "41": "T1505",
57
+ "42": "T1528",
58
+ "43": "T1542",
59
+ "44": "T1543",
60
+ "45": "T1548",
61
+ "46": "T1550",
62
+ "47": "T1552",
63
+ "48": "T1555",
64
+ "49": "T1557",
65
+ "50": "T1563",
66
+ "51": "T1565",
67
+ "52": "T1566",
68
+ "53": "T1574",
69
+ "54": "T1588",
70
+ "55": "T1608",
71
+ "56": "T1685"
72
+ },
73
+ "initializer_range": 0.02,
74
+ "intermediate_size": 3072,
75
+ "is_decoder": false,
76
+ "label2id": {
77
+ "T1003": 0,
78
+ "T1005": 1,
79
+ "T1021": 2,
80
+ "T1027": 3,
81
+ "T1036": 4,
82
+ "T1040": 5,
83
+ "T1041": 6,
84
+ "T1046": 7,
85
+ "T1055": 8,
86
+ "T1059": 9,
87
+ "T1068": 10,
88
+ "T1070": 11,
89
+ "T1071": 12,
90
+ "T1078": 13,
91
+ "T1082": 14,
92
+ "T1083": 15,
93
+ "T1087": 16,
94
+ "T1091": 17,
95
+ "T1098": 18,
96
+ "T1105": 19,
97
+ "T1106": 20,
98
+ "T1110": 21,
99
+ "T1114": 22,
100
+ "T1133": 23,
101
+ "T1136": 24,
102
+ "T1185": 25,
103
+ "T1189": 26,
104
+ "T1190": 27,
105
+ "T1202": 28,
106
+ "T1203": 29,
107
+ "T1204": 30,
108
+ "T1210": 31,
109
+ "T1211": 32,
110
+ "T1212": 33,
111
+ "T1485": 34,
112
+ "T1486": 35,
113
+ "T1489": 36,
114
+ "T1496": 37,
115
+ "T1497": 38,
116
+ "T1498": 39,
117
+ "T1499": 40,
118
+ "T1505": 41,
119
+ "T1528": 42,
120
+ "T1542": 43,
121
+ "T1543": 44,
122
+ "T1548": 45,
123
+ "T1550": 46,
124
+ "T1552": 47,
125
+ "T1555": 48,
126
+ "T1557": 49,
127
+ "T1563": 50,
128
+ "T1565": 51,
129
+ "T1566": 52,
130
+ "T1574": 53,
131
+ "T1588": 54,
132
+ "T1608": 55,
133
+ "T1685": 56
134
+ },
135
+ "layer_norm_eps": 1e-05,
136
+ "max_position_embeddings": 514,
137
+ "model_type": "roberta",
138
+ "num_attention_heads": 12,
139
+ "num_hidden_layers": 12,
140
+ "pad_token_id": 1,
141
+ "problem_type": "multi_label_classification",
142
+ "tie_word_embeddings": true,
143
+ "transformers_version": "5.13.0",
144
+ "type_vocab_size": 1,
145
+ "use_cache": false,
146
+ "vocab_size": 50265
147
+ }
emissions.csv ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ timestamp,project_name,run_id,experiment_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,water_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,cpu_utilization_percent,gpu_utilization_percent,ram_utilization_percent,ram_used_gb,on_cloud,pue,wue
2
+ 2026-07-13T09:31:32,VulnTrain,27786f01-d628-4803-945e-3163bbe2fb2a,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,188.17171251389664,0.005013825539117526,2.6644948234433754e-05,70.00017005407693,776.1263394402708,70.0,0.0035231828401046865,0.040585318579339995,0.003522912649292913,0.047631414068737596,0.0,Luxembourg,LUX,luxembourg,,,Linux-6.8.0-134-generic-x86_64-with-glibc2.39,3.12.3,3.2.8,224,Intel(R) Xeon(R) Platinum 8480+,4,4 x NVIDIA L40S,6.1327,49.6098,2015.3354263305664,machine,0.8086486486486486,60.96081081081081,1.1308108108108108,22.578279134389515,N,1.0,0.0
metrics.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "eval_loss": 0.620545506477356,
3
+ "eval_f1_micro": 0.4171494785631518,
4
+ "eval_f1_macro": 0.2027471380403296,
5
+ "eval_precision_micro": 0.3005008347245409,
6
+ "eval_recall_micro": 0.6818181818181818,
7
+ "eval_recall_at_3": 0.4821628651460584,
8
+ "eval_recall_at_5": 0.6858743497398959,
9
+ "eval_runtime": 0.1973,
10
+ "eval_samples_per_second": 603.064,
11
+ "eval_steps_per_second": 10.136,
12
+ "epoch": 40.0
13
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:89b72f8dd4ec1ae890a5177916efd824618cb34fb1958d8d043405f5f8d0b039
3
+ size 498782004
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": "<s>",
5
+ "cls_token": "<s>",
6
+ "eos_token": "</s>",
7
+ "errors": "replace",
8
+ "is_local": false,
9
+ "local_files_only": false,
10
+ "mask_token": "<mask>",
11
+ "model_max_length": 512,
12
+ "pad_token": "<pad>",
13
+ "sep_token": "</s>",
14
+ "tokenizer_class": "RobertaTokenizer",
15
+ "trim_offsets": true,
16
+ "unk_token": "<unk>"
17
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db0dc8f71629bbe3179b8ee7446ca7380373dc2885ee224a5385938421e73c4e
3
+ size 5265