Hugging Face's logo

ctokx
/
cti-attack-mapper-modernbert

Text Classification
Transformers
Safetensors
English
modernbert
cyber-threat-intelligence
mitre-attack
multi-label-classification
defensive-security
blue-team
threat-intelligence
text-embeddings-inference
Model card Files
xet
 Community
ctokx commited on
Commit
d94be6a
·
verified ·
1 Parent(s): db268d9

Add results/

Browse files
Files changed (9) hide show
  1. results/RESULTS.md +74 -0
  2. results/baselines__document.json +2294 -0
  3. results/baselines__random.json +2294 -0
  4. results/cv_5fold__modernbert.json +154 -0
  5. results/ensemble_tfidf_modernbert__document.json +779 -0
  6. results/modernbert__document.json +769 -0
  7. results/modernbert__random.json +769 -0
  8. results/securebert__document.json +769 -0
  9. results/securebert__random.json +769 -0
results/RESULTS.md ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!-- generated by scripts/04_report.py - do not edit by hand -->
2
+ ### Test-set results
3
+
4
+ Per-class thresholds tuned on dev. Macro-F1 is the headline metric. It weights all 49 techniques equally, so a handful of common ones cannot cover for a weak long tail.
5
+
6
+ | Model | doc macro-F1 | doc micro-F1 | random macro-F1 | random micro-F1 | inflation |
7
+ |---|---|---|---|---|---|
8
+ | Frequency prior | 0.000 | 0.000 | 0.000 | 0.000 | n/a |
9
+ | ATT&CK keyword match | 0.167 | 0.170 | 0.171 | 0.153 | +2.4% |
10
+ | TF-IDF + one-vs-rest LR | 0.454 | 0.500 | 0.506 | 0.545 | +11.5% |
11
+ | **ModernBERT-base (fine-tuned)** | 0.445 | 0.478 | 0.482 | 0.540 | +8.3% |
12
+ | SecureBERT (fine-tuned) | 0.369 | 0.252 | 0.415 | 0.373 | +12.5% |
13
+
14
+ ### Same table, single global threshold
15
+
16
+ | Model | doc macro-F1 | doc micro-F1 | random macro-F1 | random micro-F1 | inflation |
17
+ |---|---|---|---|---|---|
18
+ | Frequency prior | 0.000 | 0.000 | 0.000 | 0.000 | n/a |
19
+ | ATT&CK keyword match | 0.167 | 0.170 | 0.171 | 0.153 | +2.4% |
20
+ | TF-IDF + one-vs-rest LR | 0.449 | 0.472 | 0.515 | 0.552 | +14.7% |
21
+ | **ModernBERT-base (fine-tuned)** | 0.425 | 0.494 | 0.504 | 0.557 | +18.6% |
22
+ | SecureBERT (fine-tuned) | 0.352 | 0.425 | 0.388 | 0.470 | +10.2% |
23
+
24
+ ### Ensemble + cross-validated results
25
+
26
+ One 70/15/15 split of a 151-document corpus is one draw, so the gap between models can be noise. To check it, here is 5-fold cross-validation grouped by document. Each report lands in exactly one test fold. The blend weight and thresholds are tuned on each fold's own dev set, never on its test set.
27
+
28
+ | Model | doc macro-F1 (5-fold CV, mean +/- std) |
29
+ |---|---|
30
+ | TF-IDF + LR (per-class) | 0.4326 +/- 0.0349 |
31
+ | TF-IDF + LR (global) | 0.4521 +/- 0.0420 |
32
+ | ModernBERT (per-class) | 0.4263 +/- 0.0256 |
33
+ | **Ensemble TF-IDF+ModernBERT (per-class)** | **0.4738 +/- 0.0223** |
34
+
35
+ Paired per-fold test, per-class thresholds on both sides:
36
+ - ensemble vs TF-IDF, same per-class regime: **+0.0412**, 5/5 folds positive, p = 0.005
37
+ - ensemble vs ModernBERT, same per-class regime: **+0.0475**, 5/5 folds positive, p = 0.010
38
+ - ensemble vs TF-IDF at its stronger global regime: **+0.0217**, 4/5 folds positive, p = 0.100
39
+
40
+ On the original single 70/15/15 document split, the same ensemble (70% TF-IDF, 30% ModernBERT) scores **0.4965** macro-F1, against 0.4536 for TF-IDF and 0.4454 for ModernBERT.
41
+
42
+ How to read this: the ensemble has the highest mean and the lowest variance of everything tested. Compared against each base model at the same threshold setting, it wins in all five folds. The one comparison it does not clearly win is against TF-IDF at its own best threshold setting, where the lead is small enough to be noise over five folds. That row is in the table on purpose.
43
+
44
+ ### ModernBERT per-technique, document split (12 most frequent)
45
+
46
+ | Technique | Name | Support | P | R | F1 |
47
+ |---|---|---|---|---|---|
48
+ | `T1027` | Obfuscated Files or Information | 102 | 0.57 | 0.54 | 0.55 |
49
+ | `T1140` | Deobfuscate/Decode Files or Information | 68 | 0.73 | 0.84 | 0.78 |
50
+ | `T1105` | Ingress Tool Transfer | 57 | 0.51 | 0.44 | 0.47 |
51
+ | `T1059.003` | Command and Scripting Interpreter: Windows … | 51 | 0.39 | 0.59 | 0.47 |
52
+ | `T1055` | Process Injection | 40 | 0.68 | 0.53 | 0.59 |
53
+ | `T1106` | Native API | 34 | 0.64 | 0.47 | 0.54 |
54
+ | `T1047` | Windows Management Instrumentation | 28 | 0.69 | 0.64 | 0.67 |
55
+ | `T1053.005` | Scheduled Task/Job: Scheduled Task | 26 | 0.71 | 0.92 | 0.80 |
56
+ | `T1562.001` | Impair Defenses: Disable or Modify Tools ⚠️*revoked* | 26 | 0.92 | 0.46 | 0.62 |
57
+ | `T1574.002` | Hijack Execution Flow: DLL Side-Loading ⚠️*revoked* | 26 | 0.83 | 0.38 | 0.53 |
58
+ | `T1082` | System Information Discovery | 24 | 0.83 | 0.21 | 0.33 |
59
+ | `T1078` | Valid Accounts | 23 | 0.23 | 0.48 | 0.31 |
60
+
61
+ ### Head vs tail, document split
62
+
63
+ | Bucket | Techniques | Mean F1 |
64
+ |---|---|---|
65
+ | head (>=20 test examples) | 15 | 0.570 |
66
+ | tail (<20 test examples) | 34 | 0.391 |
67
+
68
+ ### Corpus
69
+
70
+ - 19,178 raw sentences, 18,437 after cleaning and dedup
71
+ - 4,024 carry at least one technique (21.8%)
72
+ - 49 techniques across 151 source documents
73
+ - 714 duplicate sentences removed (370 of them appeared in more than one document)
74
+ - dropped for having too few documents to split: T1557.001
results/baselines__document.json ADDED
@@ -0,0 +1,2294 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "frequency": {
3
+ "global_threshold": {
4
+ "threshold": 0.05,
5
+ "micro_f1": 0.0,
6
+ "macro_f1": 0.0,
7
+ "micro_precision": 0.0,
8
+ "micro_recall": 0.0,
9
+ "macro_precision": 0.0,
10
+ "macro_recall": 0.0,
11
+ "n_examples": 3635,
12
+ "n_positive_labels": 911,
13
+ "per_class": {
14
+ "T1003.001": {
15
+ "precision": 0.0,
16
+ "recall": 0.0,
17
+ "f1": 0.0,
18
+ "support": 21,
19
+ "predicted": 0
20
+ },
21
+ "T1005": {
22
+ "precision": 0.0,
23
+ "recall": 0.0,
24
+ "f1": 0.0,
25
+ "support": 8,
26
+ "predicted": 0
27
+ },
28
+ "T1012": {
29
+ "precision": 0.0,
30
+ "recall": 0.0,
31
+ "f1": 0.0,
32
+ "support": 4,
33
+ "predicted": 0
34
+ },
35
+ "T1016": {
36
+ "precision": 0.0,
37
+ "recall": 0.0,
38
+ "f1": 0.0,
39
+ "support": 11,
40
+ "predicted": 0
41
+ },
42
+ "T1021.001": {
43
+ "precision": 0.0,
44
+ "recall": 0.0,
45
+ "f1": 0.0,
46
+ "support": 12,
47
+ "predicted": 0
48
+ },
49
+ "T1027": {
50
+ "precision": 0.0,
51
+ "recall": 0.0,
52
+ "f1": 0.0,
53
+ "support": 102,
54
+ "predicted": 0
55
+ },
56
+ "T1033": {
57
+ "precision": 0.0,
58
+ "recall": 0.0,
59
+ "f1": 0.0,
60
+ "support": 13,
61
+ "predicted": 0
62
+ },
63
+ "T1036.005": {
64
+ "precision": 0.0,
65
+ "recall": 0.0,
66
+ "f1": 0.0,
67
+ "support": 10,
68
+ "predicted": 0
69
+ },
70
+ "T1041": {
71
+ "precision": 0.0,
72
+ "recall": 0.0,
73
+ "f1": 0.0,
74
+ "support": 17,
75
+ "predicted": 0
76
+ },
77
+ "T1047": {
78
+ "precision": 0.0,
79
+ "recall": 0.0,
80
+ "f1": 0.0,
81
+ "support": 28,
82
+ "predicted": 0
83
+ },
84
+ "T1053.005": {
85
+ "precision": 0.0,
86
+ "recall": 0.0,
87
+ "f1": 0.0,
88
+ "support": 26,
89
+ "predicted": 0
90
+ },
91
+ "T1055": {
92
+ "precision": 0.0,
93
+ "recall": 0.0,
94
+ "f1": 0.0,
95
+ "support": 40,
96
+ "predicted": 0
97
+ },
98
+ "T1056.001": {
99
+ "precision": 0.0,
100
+ "recall": 0.0,
101
+ "f1": 0.0,
102
+ "support": 15,
103
+ "predicted": 0
104
+ },
105
+ "T1057": {
106
+ "precision": 0.0,
107
+ "recall": 0.0,
108
+ "f1": 0.0,
109
+ "support": 15,
110
+ "predicted": 0
111
+ },
112
+ "T1059.003": {
113
+ "precision": 0.0,
114
+ "recall": 0.0,
115
+ "f1": 0.0,
116
+ "support": 51,
117
+ "predicted": 0
118
+ },
119
+ "T1068": {
120
+ "precision": 0.0,
121
+ "recall": 0.0,
122
+ "f1": 0.0,
123
+ "support": 3,
124
+ "predicted": 0
125
+ },
126
+ "T1070.004": {
127
+ "precision": 0.0,
128
+ "recall": 0.0,
129
+ "f1": 0.0,
130
+ "support": 12,
131
+ "predicted": 0
132
+ },
133
+ "T1071.001": {
134
+ "precision": 0.0,
135
+ "recall": 0.0,
136
+ "f1": 0.0,
137
+ "support": 21,
138
+ "predicted": 0
139
+ },
140
+ "T1072": {
141
+ "precision": 0.0,
142
+ "recall": 0.0,
143
+ "f1": 0.0,
144
+ "support": 3,
145
+ "predicted": 0
146
+ },
147
+ "T1074.001": {
148
+ "precision": 0.0,
149
+ "recall": 0.0,
150
+ "f1": 0.0,
151
+ "support": 3,
152
+ "predicted": 0
153
+ },
154
+ "T1078": {
155
+ "precision": 0.0,
156
+ "recall": 0.0,
157
+ "f1": 0.0,
158
+ "support": 23,
159
+ "predicted": 0
160
+ },
161
+ "T1082": {
162
+ "precision": 0.0,
163
+ "recall": 0.0,
164
+ "f1": 0.0,
165
+ "support": 24,
166
+ "predicted": 0
167
+ },
168
+ "T1083": {
169
+ "precision": 0.0,
170
+ "recall": 0.0,
171
+ "f1": 0.0,
172
+ "support": 15,
173
+ "predicted": 0
174
+ },
175
+ "T1090": {
176
+ "precision": 0.0,
177
+ "recall": 0.0,
178
+ "f1": 0.0,
179
+ "support": 23,
180
+ "predicted": 0
181
+ },
182
+ "T1095": {
183
+ "precision": 0.0,
184
+ "recall": 0.0,
185
+ "f1": 0.0,
186
+ "support": 3,
187
+ "predicted": 0
188
+ },
189
+ "T1105": {
190
+ "precision": 0.0,
191
+ "recall": 0.0,
192
+ "f1": 0.0,
193
+ "support": 57,
194
+ "predicted": 0
195
+ },
196
+ "T1106": {
197
+ "precision": 0.0,
198
+ "recall": 0.0,
199
+ "f1": 0.0,
200
+ "support": 34,
201
+ "predicted": 0
202
+ },
203
+ "T1110": {
204
+ "precision": 0.0,
205
+ "recall": 0.0,
206
+ "f1": 0.0,
207
+ "support": 19,
208
+ "predicted": 0
209
+ },
210
+ "T1112": {
211
+ "precision": 0.0,
212
+ "recall": 0.0,
213
+ "f1": 0.0,
214
+ "support": 14,
215
+ "predicted": 0
216
+ },
217
+ "T1113": {
218
+ "precision": 0.0,
219
+ "recall": 0.0,
220
+ "f1": 0.0,
221
+ "support": 9,
222
+ "predicted": 0
223
+ },
224
+ "T1140": {
225
+ "precision": 0.0,
226
+ "recall": 0.0,
227
+ "f1": 0.0,
228
+ "support": 68,
229
+ "predicted": 0
230
+ },
231
+ "T1190": {
232
+ "precision": 0.0,
233
+ "recall": 0.0,
234
+ "f1": 0.0,
235
+ "support": 15,
236
+ "predicted": 0
237
+ },
238
+ "T1204.002": {
239
+ "precision": 0.0,
240
+ "recall": 0.0,
241
+ "f1": 0.0,
242
+ "support": 15,
243
+ "predicted": 0
244
+ },
245
+ "T1210": {
246
+ "precision": 0.0,
247
+ "recall": 0.0,
248
+ "f1": 0.0,
249
+ "support": 2,
250
+ "predicted": 0
251
+ },
252
+ "T1218.011": {
253
+ "precision": 0.0,
254
+ "recall": 0.0,
255
+ "f1": 0.0,
256
+ "support": 14,
257
+ "predicted": 0
258
+ },
259
+ "T1219": {
260
+ "precision": 0.0,
261
+ "recall": 0.0,
262
+ "f1": 0.0,
263
+ "support": 19,
264
+ "predicted": 0
265
+ },
266
+ "T1484.001": {
267
+ "precision": 0.0,
268
+ "recall": 0.0,
269
+ "f1": 0.0,
270
+ "support": 5,
271
+ "predicted": 0
272
+ },
273
+ "T1518.001": {
274
+ "precision": 0.0,
275
+ "recall": 0.0,
276
+ "f1": 0.0,
277
+ "support": 7,
278
+ "predicted": 0
279
+ },
280
+ "T1543.003": {
281
+ "precision": 0.0,
282
+ "recall": 0.0,
283
+ "f1": 0.0,
284
+ "support": 13,
285
+ "predicted": 0
286
+ },
287
+ "T1547.001": {
288
+ "precision": 0.0,
289
+ "recall": 0.0,
290
+ "f1": 0.0,
291
+ "support": 16,
292
+ "predicted": 0
293
+ },
294
+ "T1548.002": {
295
+ "precision": 0.0,
296
+ "recall": 0.0,
297
+ "f1": 0.0,
298
+ "support": 2,
299
+ "predicted": 0
300
+ },
301
+ "T1552.001": {
302
+ "precision": 0.0,
303
+ "recall": 0.0,
304
+ "f1": 0.0,
305
+ "support": 4,
306
+ "predicted": 0
307
+ },
308
+ "T1562.001": {
309
+ "precision": 0.0,
310
+ "recall": 0.0,
311
+ "f1": 0.0,
312
+ "support": 26,
313
+ "predicted": 0
314
+ },
315
+ "T1564.001": {
316
+ "precision": 0.0,
317
+ "recall": 0.0,
318
+ "f1": 0.0,
319
+ "support": 3,
320
+ "predicted": 0
321
+ },
322
+ "T1566.001": {
323
+ "precision": 0.0,
324
+ "recall": 0.0,
325
+ "f1": 0.0,
326
+ "support": 13,
327
+ "predicted": 0
328
+ },
329
+ "T1569.002": {
330
+ "precision": 0.0,
331
+ "recall": 0.0,
332
+ "f1": 0.0,
333
+ "support": 5,
334
+ "predicted": 0
335
+ },
336
+ "T1570": {
337
+ "precision": 0.0,
338
+ "recall": 0.0,
339
+ "f1": 0.0,
340
+ "support": 6,
341
+ "predicted": 0
342
+ },
343
+ "T1573.001": {
344
+ "precision": 0.0,
345
+ "recall": 0.0,
346
+ "f1": 0.0,
347
+ "support": 16,
348
+ "predicted": 0
349
+ },
350
+ "T1574.002": {
351
+ "precision": 0.0,
352
+ "recall": 0.0,
353
+ "f1": 0.0,
354
+ "support": 26,
355
+ "predicted": 0
356
+ }
357
+ }
358
+ },
359
+ "per_class_threshold": {
360
+ "thresholds": {
361
+ "T1003.001": 0.05,
362
+ "T1005": 0.05,
363
+ "T1012": 0.05,
364
+ "T1016": 0.05,
365
+ "T1021.001": 0.05,
366
+ "T1027": 0.05,
367
+ "T1033": 0.05,
368
+ "T1036.005": 0.05,
369
+ "T1041": 0.05,
370
+ "T1047": 0.05,
371
+ "T1053.005": 0.05,
372
+ "T1055": 0.05,
373
+ "T1056.001": 0.05,
374
+ "T1057": 0.05,
375
+ "T1059.003": 0.05,
376
+ "T1068": 0.05,
377
+ "T1070.004": 0.05,
378
+ "T1071.001": 0.05,
379
+ "T1072": 0.05,
380
+ "T1074.001": 0.05,
381
+ "T1078": 0.05,
382
+ "T1082": 0.05,
383
+ "T1083": 0.05,
384
+ "T1090": 0.05,
385
+ "T1095": 0.05,
386
+ "T1105": 0.05,
387
+ "T1106": 0.05,
388
+ "T1110": 0.05,
389
+ "T1112": 0.05,
390
+ "T1113": 0.05,
391
+ "T1140": 0.05,
392
+ "T1190": 0.05,
393
+ "T1204.002": 0.05,
394
+ "T1210": 0.05,
395
+ "T1218.011": 0.05,
396
+ "T1219": 0.05,
397
+ "T1484.001": 0.05,
398
+ "T1518.001": 0.05,
399
+ "T1543.003": 0.05,
400
+ "T1547.001": 0.05,
401
+ "T1548.002": 0.05,
402
+ "T1552.001": 0.05,
403
+ "T1562.001": 0.05,
404
+ "T1564.001": 0.05,
405
+ "T1566.001": 0.05,
406
+ "T1569.002": 0.05,
407
+ "T1570": 0.05,
408
+ "T1573.001": 0.05,
409
+ "T1574.002": 0.05
410
+ },
411
+ "micro_f1": 0.0,
412
+ "macro_f1": 0.0,
413
+ "micro_precision": 0.0,
414
+ "micro_recall": 0.0,
415
+ "macro_precision": 0.0,
416
+ "macro_recall": 0.0,
417
+ "n_examples": 3635,
418
+ "n_positive_labels": 911,
419
+ "per_class": {
420
+ "T1003.001": {
421
+ "precision": 0.0,
422
+ "recall": 0.0,
423
+ "f1": 0.0,
424
+ "support": 21,
425
+ "predicted": 0
426
+ },
427
+ "T1005": {
428
+ "precision": 0.0,
429
+ "recall": 0.0,
430
+ "f1": 0.0,
431
+ "support": 8,
432
+ "predicted": 0
433
+ },
434
+ "T1012": {
435
+ "precision": 0.0,
436
+ "recall": 0.0,
437
+ "f1": 0.0,
438
+ "support": 4,
439
+ "predicted": 0
440
+ },
441
+ "T1016": {
442
+ "precision": 0.0,
443
+ "recall": 0.0,
444
+ "f1": 0.0,
445
+ "support": 11,
446
+ "predicted": 0
447
+ },
448
+ "T1021.001": {
449
+ "precision": 0.0,
450
+ "recall": 0.0,
451
+ "f1": 0.0,
452
+ "support": 12,
453
+ "predicted": 0
454
+ },
455
+ "T1027": {
456
+ "precision": 0.0,
457
+ "recall": 0.0,
458
+ "f1": 0.0,
459
+ "support": 102,
460
+ "predicted": 0
461
+ },
462
+ "T1033": {
463
+ "precision": 0.0,
464
+ "recall": 0.0,
465
+ "f1": 0.0,
466
+ "support": 13,
467
+ "predicted": 0
468
+ },
469
+ "T1036.005": {
470
+ "precision": 0.0,
471
+ "recall": 0.0,
472
+ "f1": 0.0,
473
+ "support": 10,
474
+ "predicted": 0
475
+ },
476
+ "T1041": {
477
+ "precision": 0.0,
478
+ "recall": 0.0,
479
+ "f1": 0.0,
480
+ "support": 17,
481
+ "predicted": 0
482
+ },
483
+ "T1047": {
484
+ "precision": 0.0,
485
+ "recall": 0.0,
486
+ "f1": 0.0,
487
+ "support": 28,
488
+ "predicted": 0
489
+ },
490
+ "T1053.005": {
491
+ "precision": 0.0,
492
+ "recall": 0.0,
493
+ "f1": 0.0,
494
+ "support": 26,
495
+ "predicted": 0
496
+ },
497
+ "T1055": {
498
+ "precision": 0.0,
499
+ "recall": 0.0,
500
+ "f1": 0.0,
501
+ "support": 40,
502
+ "predicted": 0
503
+ },
504
+ "T1056.001": {
505
+ "precision": 0.0,
506
+ "recall": 0.0,
507
+ "f1": 0.0,
508
+ "support": 15,
509
+ "predicted": 0
510
+ },
511
+ "T1057": {
512
+ "precision": 0.0,
513
+ "recall": 0.0,
514
+ "f1": 0.0,
515
+ "support": 15,
516
+ "predicted": 0
517
+ },
518
+ "T1059.003": {
519
+ "precision": 0.0,
520
+ "recall": 0.0,
521
+ "f1": 0.0,
522
+ "support": 51,
523
+ "predicted": 0
524
+ },
525
+ "T1068": {
526
+ "precision": 0.0,
527
+ "recall": 0.0,
528
+ "f1": 0.0,
529
+ "support": 3,
530
+ "predicted": 0
531
+ },
532
+ "T1070.004": {
533
+ "precision": 0.0,
534
+ "recall": 0.0,
535
+ "f1": 0.0,
536
+ "support": 12,
537
+ "predicted": 0
538
+ },
539
+ "T1071.001": {
540
+ "precision": 0.0,
541
+ "recall": 0.0,
542
+ "f1": 0.0,
543
+ "support": 21,
544
+ "predicted": 0
545
+ },
546
+ "T1072": {
547
+ "precision": 0.0,
548
+ "recall": 0.0,
549
+ "f1": 0.0,
550
+ "support": 3,
551
+ "predicted": 0
552
+ },
553
+ "T1074.001": {
554
+ "precision": 0.0,
555
+ "recall": 0.0,
556
+ "f1": 0.0,
557
+ "support": 3,
558
+ "predicted": 0
559
+ },
560
+ "T1078": {
561
+ "precision": 0.0,
562
+ "recall": 0.0,
563
+ "f1": 0.0,
564
+ "support": 23,
565
+ "predicted": 0
566
+ },
567
+ "T1082": {
568
+ "precision": 0.0,
569
+ "recall": 0.0,
570
+ "f1": 0.0,
571
+ "support": 24,
572
+ "predicted": 0
573
+ },
574
+ "T1083": {
575
+ "precision": 0.0,
576
+ "recall": 0.0,
577
+ "f1": 0.0,
578
+ "support": 15,
579
+ "predicted": 0
580
+ },
581
+ "T1090": {
582
+ "precision": 0.0,
583
+ "recall": 0.0,
584
+ "f1": 0.0,
585
+ "support": 23,
586
+ "predicted": 0
587
+ },
588
+ "T1095": {
589
+ "precision": 0.0,
590
+ "recall": 0.0,
591
+ "f1": 0.0,
592
+ "support": 3,
593
+ "predicted": 0
594
+ },
595
+ "T1105": {
596
+ "precision": 0.0,
597
+ "recall": 0.0,
598
+ "f1": 0.0,
599
+ "support": 57,
600
+ "predicted": 0
601
+ },
602
+ "T1106": {
603
+ "precision": 0.0,
604
+ "recall": 0.0,
605
+ "f1": 0.0,
606
+ "support": 34,
607
+ "predicted": 0
608
+ },
609
+ "T1110": {
610
+ "precision": 0.0,
611
+ "recall": 0.0,
612
+ "f1": 0.0,
613
+ "support": 19,
614
+ "predicted": 0
615
+ },
616
+ "T1112": {
617
+ "precision": 0.0,
618
+ "recall": 0.0,
619
+ "f1": 0.0,
620
+ "support": 14,
621
+ "predicted": 0
622
+ },
623
+ "T1113": {
624
+ "precision": 0.0,
625
+ "recall": 0.0,
626
+ "f1": 0.0,
627
+ "support": 9,
628
+ "predicted": 0
629
+ },
630
+ "T1140": {
631
+ "precision": 0.0,
632
+ "recall": 0.0,
633
+ "f1": 0.0,
634
+ "support": 68,
635
+ "predicted": 0
636
+ },
637
+ "T1190": {
638
+ "precision": 0.0,
639
+ "recall": 0.0,
640
+ "f1": 0.0,
641
+ "support": 15,
642
+ "predicted": 0
643
+ },
644
+ "T1204.002": {
645
+ "precision": 0.0,
646
+ "recall": 0.0,
647
+ "f1": 0.0,
648
+ "support": 15,
649
+ "predicted": 0
650
+ },
651
+ "T1210": {
652
+ "precision": 0.0,
653
+ "recall": 0.0,
654
+ "f1": 0.0,
655
+ "support": 2,
656
+ "predicted": 0
657
+ },
658
+ "T1218.011": {
659
+ "precision": 0.0,
660
+ "recall": 0.0,
661
+ "f1": 0.0,
662
+ "support": 14,
663
+ "predicted": 0
664
+ },
665
+ "T1219": {
666
+ "precision": 0.0,
667
+ "recall": 0.0,
668
+ "f1": 0.0,
669
+ "support": 19,
670
+ "predicted": 0
671
+ },
672
+ "T1484.001": {
673
+ "precision": 0.0,
674
+ "recall": 0.0,
675
+ "f1": 0.0,
676
+ "support": 5,
677
+ "predicted": 0
678
+ },
679
+ "T1518.001": {
680
+ "precision": 0.0,
681
+ "recall": 0.0,
682
+ "f1": 0.0,
683
+ "support": 7,
684
+ "predicted": 0
685
+ },
686
+ "T1543.003": {
687
+ "precision": 0.0,
688
+ "recall": 0.0,
689
+ "f1": 0.0,
690
+ "support": 13,
691
+ "predicted": 0
692
+ },
693
+ "T1547.001": {
694
+ "precision": 0.0,
695
+ "recall": 0.0,
696
+ "f1": 0.0,
697
+ "support": 16,
698
+ "predicted": 0
699
+ },
700
+ "T1548.002": {
701
+ "precision": 0.0,
702
+ "recall": 0.0,
703
+ "f1": 0.0,
704
+ "support": 2,
705
+ "predicted": 0
706
+ },
707
+ "T1552.001": {
708
+ "precision": 0.0,
709
+ "recall": 0.0,
710
+ "f1": 0.0,
711
+ "support": 4,
712
+ "predicted": 0
713
+ },
714
+ "T1562.001": {
715
+ "precision": 0.0,
716
+ "recall": 0.0,
717
+ "f1": 0.0,
718
+ "support": 26,
719
+ "predicted": 0
720
+ },
721
+ "T1564.001": {
722
+ "precision": 0.0,
723
+ "recall": 0.0,
724
+ "f1": 0.0,
725
+ "support": 3,
726
+ "predicted": 0
727
+ },
728
+ "T1566.001": {
729
+ "precision": 0.0,
730
+ "recall": 0.0,
731
+ "f1": 0.0,
732
+ "support": 13,
733
+ "predicted": 0
734
+ },
735
+ "T1569.002": {
736
+ "precision": 0.0,
737
+ "recall": 0.0,
738
+ "f1": 0.0,
739
+ "support": 5,
740
+ "predicted": 0
741
+ },
742
+ "T1570": {
743
+ "precision": 0.0,
744
+ "recall": 0.0,
745
+ "f1": 0.0,
746
+ "support": 6,
747
+ "predicted": 0
748
+ },
749
+ "T1573.001": {
750
+ "precision": 0.0,
751
+ "recall": 0.0,
752
+ "f1": 0.0,
753
+ "support": 16,
754
+ "predicted": 0
755
+ },
756
+ "T1574.002": {
757
+ "precision": 0.0,
758
+ "recall": 0.0,
759
+ "f1": 0.0,
760
+ "support": 26,
761
+ "predicted": 0
762
+ }
763
+ }
764
+ }
765
+ },
766
+ "keyword": {
767
+ "global_threshold": {
768
+ "threshold": 0.05,
769
+ "micro_f1": 0.1701,
770
+ "macro_f1": 0.1666,
771
+ "micro_precision": 0.2759,
772
+ "micro_recall": 0.1229,
773
+ "macro_precision": 0.2387,
774
+ "macro_recall": 0.1516,
775
+ "n_examples": 3635,
776
+ "n_positive_labels": 911,
777
+ "per_class": {
778
+ "T1003.001": {
779
+ "precision": 0.2308,
780
+ "recall": 0.1429,
781
+ "f1": 0.1765,
782
+ "support": 21,
783
+ "predicted": 13
784
+ },
785
+ "T1005": {
786
+ "precision": 0.3333,
787
+ "recall": 0.125,
788
+ "f1": 0.1818,
789
+ "support": 8,
790
+ "predicted": 3
791
+ },
792
+ "T1012": {
793
+ "precision": 0.0,
794
+ "recall": 0.0,
795
+ "f1": 0.0,
796
+ "support": 4,
797
+ "predicted": 5
798
+ },
799
+ "T1016": {
800
+ "precision": 0.1,
801
+ "recall": 0.0909,
802
+ "f1": 0.0952,
803
+ "support": 11,
804
+ "predicted": 10
805
+ },
806
+ "T1021.001": {
807
+ "precision": 0.2632,
808
+ "recall": 0.4167,
809
+ "f1": 0.3226,
810
+ "support": 12,
811
+ "predicted": 19
812
+ },
813
+ "T1027": {
814
+ "precision": 0.1667,
815
+ "recall": 0.0098,
816
+ "f1": 0.0185,
817
+ "support": 102,
818
+ "predicted": 6
819
+ },
820
+ "T1033": {
821
+ "precision": 0.2,
822
+ "recall": 0.0769,
823
+ "f1": 0.1111,
824
+ "support": 13,
825
+ "predicted": 5
826
+ },
827
+ "T1036.005": {
828
+ "precision": 0.2,
829
+ "recall": 0.2,
830
+ "f1": 0.2,
831
+ "support": 10,
832
+ "predicted": 10
833
+ },
834
+ "T1041": {
835
+ "precision": 0.25,
836
+ "recall": 0.0588,
837
+ "f1": 0.0952,
838
+ "support": 17,
839
+ "predicted": 4
840
+ },
841
+ "T1047": {
842
+ "precision": 0.3333,
843
+ "recall": 0.1071,
844
+ "f1": 0.1622,
845
+ "support": 28,
846
+ "predicted": 9
847
+ },
848
+ "T1053.005": {
849
+ "precision": 0.6389,
850
+ "recall": 0.8846,
851
+ "f1": 0.7419,
852
+ "support": 26,
853
+ "predicted": 36
854
+ },
855
+ "T1055": {
856
+ "precision": 0.75,
857
+ "recall": 0.15,
858
+ "f1": 0.25,
859
+ "support": 40,
860
+ "predicted": 8
861
+ },
862
+ "T1056.001": {
863
+ "precision": 0.6667,
864
+ "recall": 0.4,
865
+ "f1": 0.5,
866
+ "support": 15,
867
+ "predicted": 9
868
+ },
869
+ "T1057": {
870
+ "precision": 0.0,
871
+ "recall": 0.0,
872
+ "f1": 0.0,
873
+ "support": 15,
874
+ "predicted": 9
875
+ },
876
+ "T1059.003": {
877
+ "precision": 0.0,
878
+ "recall": 0.0,
879
+ "f1": 0.0,
880
+ "support": 51,
881
+ "predicted": 18
882
+ },
883
+ "T1068": {
884
+ "precision": 1.0,
885
+ "recall": 0.3333,
886
+ "f1": 0.5,
887
+ "support": 3,
888
+ "predicted": 1
889
+ },
890
+ "T1070.004": {
891
+ "precision": 0.0909,
892
+ "recall": 0.0833,
893
+ "f1": 0.087,
894
+ "support": 12,
895
+ "predicted": 11
896
+ },
897
+ "T1071.001": {
898
+ "precision": 0.1429,
899
+ "recall": 0.0476,
900
+ "f1": 0.0714,
901
+ "support": 21,
902
+ "predicted": 7
903
+ },
904
+ "T1072": {
905
+ "precision": 0.0,
906
+ "recall": 0.0,
907
+ "f1": 0.0,
908
+ "support": 3,
909
+ "predicted": 0
910
+ },
911
+ "T1074.001": {
912
+ "precision": 0.5,
913
+ "recall": 0.3333,
914
+ "f1": 0.4,
915
+ "support": 3,
916
+ "predicted": 2
917
+ },
918
+ "T1078": {
919
+ "precision": 0.5,
920
+ "recall": 0.2174,
921
+ "f1": 0.303,
922
+ "support": 23,
923
+ "predicted": 10
924
+ },
925
+ "T1082": {
926
+ "precision": 0.1429,
927
+ "recall": 0.0417,
928
+ "f1": 0.0645,
929
+ "support": 24,
930
+ "predicted": 7
931
+ },
932
+ "T1083": {
933
+ "precision": 0.1111,
934
+ "recall": 0.0667,
935
+ "f1": 0.0833,
936
+ "support": 15,
937
+ "predicted": 9
938
+ },
939
+ "T1090": {
940
+ "precision": 0.5238,
941
+ "recall": 0.4783,
942
+ "f1": 0.5,
943
+ "support": 23,
944
+ "predicted": 21
945
+ },
946
+ "T1095": {
947
+ "precision": 0.0,
948
+ "recall": 0.0,
949
+ "f1": 0.0,
950
+ "support": 3,
951
+ "predicted": 0
952
+ },
953
+ "T1105": {
954
+ "precision": 0.2,
955
+ "recall": 0.0175,
956
+ "f1": 0.0323,
957
+ "support": 57,
958
+ "predicted": 5
959
+ },
960
+ "T1106": {
961
+ "precision": 0.0,
962
+ "recall": 0.0,
963
+ "f1": 0.0,
964
+ "support": 34,
965
+ "predicted": 0
966
+ },
967
+ "T1110": {
968
+ "precision": 0.4286,
969
+ "recall": 0.1579,
970
+ "f1": 0.2308,
971
+ "support": 19,
972
+ "predicted": 7
973
+ },
974
+ "T1112": {
975
+ "precision": 0.2,
976
+ "recall": 0.0714,
977
+ "f1": 0.1053,
978
+ "support": 14,
979
+ "predicted": 5
980
+ },
981
+ "T1113": {
982
+ "precision": 0.5,
983
+ "recall": 0.2222,
984
+ "f1": 0.3077,
985
+ "support": 9,
986
+ "predicted": 4
987
+ },
988
+ "T1140": {
989
+ "precision": 0.0,
990
+ "recall": 0.0,
991
+ "f1": 0.0,
992
+ "support": 68,
993
+ "predicted": 2
994
+ },
995
+ "T1190": {
996
+ "precision": 0.4,
997
+ "recall": 0.1333,
998
+ "f1": 0.2,
999
+ "support": 15,
1000
+ "predicted": 5
1001
+ },
1002
+ "T1204.002": {
1003
+ "precision": 0.0909,
1004
+ "recall": 0.0667,
1005
+ "f1": 0.0769,
1006
+ "support": 15,
1007
+ "predicted": 11
1008
+ },
1009
+ "T1210": {
1010
+ "precision": 0.0,
1011
+ "recall": 0.0,
1012
+ "f1": 0.0,
1013
+ "support": 2,
1014
+ "predicted": 0
1015
+ },
1016
+ "T1218.011": {
1017
+ "precision": 0.5,
1018
+ "recall": 0.7857,
1019
+ "f1": 0.6111,
1020
+ "support": 14,
1021
+ "predicted": 22
1022
+ },
1023
+ "T1219": {
1024
+ "precision": 0.0,
1025
+ "recall": 0.0,
1026
+ "f1": 0.0,
1027
+ "support": 19,
1028
+ "predicted": 0
1029
+ },
1030
+ "T1484.001": {
1031
+ "precision": 0.0,
1032
+ "recall": 0.0,
1033
+ "f1": 0.0,
1034
+ "support": 5,
1035
+ "predicted": 1
1036
+ },
1037
+ "T1518.001": {
1038
+ "precision": 0.2,
1039
+ "recall": 0.1429,
1040
+ "f1": 0.1667,
1041
+ "support": 7,
1042
+ "predicted": 5
1043
+ },
1044
+ "T1543.003": {
1045
+ "precision": 0.7143,
1046
+ "recall": 0.3846,
1047
+ "f1": 0.5,
1048
+ "support": 13,
1049
+ "predicted": 7
1050
+ },
1051
+ "T1547.001": {
1052
+ "precision": 0.3333,
1053
+ "recall": 0.125,
1054
+ "f1": 0.1818,
1055
+ "support": 16,
1056
+ "predicted": 6
1057
+ },
1058
+ "T1548.002": {
1059
+ "precision": 0.0,
1060
+ "recall": 0.0,
1061
+ "f1": 0.0,
1062
+ "support": 2,
1063
+ "predicted": 2
1064
+ },
1065
+ "T1552.001": {
1066
+ "precision": 0.5,
1067
+ "recall": 0.25,
1068
+ "f1": 0.3333,
1069
+ "support": 4,
1070
+ "predicted": 2
1071
+ },
1072
+ "T1562.001": {
1073
+ "precision": 0.0,
1074
+ "recall": 0.0,
1075
+ "f1": 0.0,
1076
+ "support": 26,
1077
+ "predicted": 0
1078
+ },
1079
+ "T1564.001": {
1080
+ "precision": 0.0,
1081
+ "recall": 0.0,
1082
+ "f1": 0.0,
1083
+ "support": 3,
1084
+ "predicted": 2
1085
+ },
1086
+ "T1566.001": {
1087
+ "precision": 0.0541,
1088
+ "recall": 0.3077,
1089
+ "f1": 0.092,
1090
+ "support": 13,
1091
+ "predicted": 74
1092
+ },
1093
+ "T1569.002": {
1094
+ "precision": 0.0,
1095
+ "recall": 0.0,
1096
+ "f1": 0.0,
1097
+ "support": 5,
1098
+ "predicted": 4
1099
+ },
1100
+ "T1570": {
1101
+ "precision": 0.4286,
1102
+ "recall": 0.5,
1103
+ "f1": 0.4615,
1104
+ "support": 6,
1105
+ "predicted": 7
1106
+ },
1107
+ "T1573.001": {
1108
+ "precision": 0.0,
1109
+ "recall": 0.0,
1110
+ "f1": 0.0,
1111
+ "support": 16,
1112
+ "predicted": 3
1113
+ },
1114
+ "T1574.002": {
1115
+ "precision": 0.0,
1116
+ "recall": 0.0,
1117
+ "f1": 0.0,
1118
+ "support": 26,
1119
+ "predicted": 0
1120
+ }
1121
+ }
1122
+ },
1123
+ "per_class_threshold": {
1124
+ "thresholds": {
1125
+ "T1003.001": 0.05,
1126
+ "T1005": 0.05,
1127
+ "T1012": 0.05,
1128
+ "T1016": 0.05,
1129
+ "T1021.001": 0.05,
1130
+ "T1027": 0.05,
1131
+ "T1033": 0.05,
1132
+ "T1036.005": 0.05,
1133
+ "T1041": 0.05,
1134
+ "T1047": 0.05,
1135
+ "T1053.005": 0.05,
1136
+ "T1055": 0.05,
1137
+ "T1056.001": 0.05,
1138
+ "T1057": 0.05,
1139
+ "T1059.003": 0.05,
1140
+ "T1068": 0.05,
1141
+ "T1070.004": 0.05,
1142
+ "T1071.001": 0.05,
1143
+ "T1072": 0.05,
1144
+ "T1074.001": 0.05,
1145
+ "T1078": 0.05,
1146
+ "T1082": 0.05,
1147
+ "T1083": 0.05,
1148
+ "T1090": 0.05,
1149
+ "T1095": 0.05,
1150
+ "T1105": 0.05,
1151
+ "T1106": 0.05,
1152
+ "T1110": 0.05,
1153
+ "T1112": 0.05,
1154
+ "T1113": 0.05,
1155
+ "T1140": 0.05,
1156
+ "T1190": 0.05,
1157
+ "T1204.002": 0.05,
1158
+ "T1210": 0.05,
1159
+ "T1218.011": 0.05,
1160
+ "T1219": 0.05,
1161
+ "T1484.001": 0.05,
1162
+ "T1518.001": 0.05,
1163
+ "T1543.003": 0.05,
1164
+ "T1547.001": 0.05,
1165
+ "T1548.002": 0.05,
1166
+ "T1552.001": 0.05,
1167
+ "T1562.001": 0.05,
1168
+ "T1564.001": 0.05,
1169
+ "T1566.001": 0.05,
1170
+ "T1569.002": 0.05,
1171
+ "T1570": 0.05,
1172
+ "T1573.001": 0.05,
1173
+ "T1574.002": 0.05
1174
+ },
1175
+ "micro_f1": 0.1701,
1176
+ "macro_f1": 0.1666,
1177
+ "micro_precision": 0.2759,
1178
+ "micro_recall": 0.1229,
1179
+ "macro_precision": 0.2387,
1180
+ "macro_recall": 0.1516,
1181
+ "n_examples": 3635,
1182
+ "n_positive_labels": 911,
1183
+ "per_class": {
1184
+ "T1003.001": {
1185
+ "precision": 0.2308,
1186
+ "recall": 0.1429,
1187
+ "f1": 0.1765,
1188
+ "support": 21,
1189
+ "predicted": 13
1190
+ },
1191
+ "T1005": {
1192
+ "precision": 0.3333,
1193
+ "recall": 0.125,
1194
+ "f1": 0.1818,
1195
+ "support": 8,
1196
+ "predicted": 3
1197
+ },
1198
+ "T1012": {
1199
+ "precision": 0.0,
1200
+ "recall": 0.0,
1201
+ "f1": 0.0,
1202
+ "support": 4,
1203
+ "predicted": 5
1204
+ },
1205
+ "T1016": {
1206
+ "precision": 0.1,
1207
+ "recall": 0.0909,
1208
+ "f1": 0.0952,
1209
+ "support": 11,
1210
+ "predicted": 10
1211
+ },
1212
+ "T1021.001": {
1213
+ "precision": 0.2632,
1214
+ "recall": 0.4167,
1215
+ "f1": 0.3226,
1216
+ "support": 12,
1217
+ "predicted": 19
1218
+ },
1219
+ "T1027": {
1220
+ "precision": 0.1667,
1221
+ "recall": 0.0098,
1222
+ "f1": 0.0185,
1223
+ "support": 102,
1224
+ "predicted": 6
1225
+ },
1226
+ "T1033": {
1227
+ "precision": 0.2,
1228
+ "recall": 0.0769,
1229
+ "f1": 0.1111,
1230
+ "support": 13,
1231
+ "predicted": 5
1232
+ },
1233
+ "T1036.005": {
1234
+ "precision": 0.2,
1235
+ "recall": 0.2,
1236
+ "f1": 0.2,
1237
+ "support": 10,
1238
+ "predicted": 10
1239
+ },
1240
+ "T1041": {
1241
+ "precision": 0.25,
1242
+ "recall": 0.0588,
1243
+ "f1": 0.0952,
1244
+ "support": 17,
1245
+ "predicted": 4
1246
+ },
1247
+ "T1047": {
1248
+ "precision": 0.3333,
1249
+ "recall": 0.1071,
1250
+ "f1": 0.1622,
1251
+ "support": 28,
1252
+ "predicted": 9
1253
+ },
1254
+ "T1053.005": {
1255
+ "precision": 0.6389,
1256
+ "recall": 0.8846,
1257
+ "f1": 0.7419,
1258
+ "support": 26,
1259
+ "predicted": 36
1260
+ },
1261
+ "T1055": {
1262
+ "precision": 0.75,
1263
+ "recall": 0.15,
1264
+ "f1": 0.25,
1265
+ "support": 40,
1266
+ "predicted": 8
1267
+ },
1268
+ "T1056.001": {
1269
+ "precision": 0.6667,
1270
+ "recall": 0.4,
1271
+ "f1": 0.5,
1272
+ "support": 15,
1273
+ "predicted": 9
1274
+ },
1275
+ "T1057": {
1276
+ "precision": 0.0,
1277
+ "recall": 0.0,
1278
+ "f1": 0.0,
1279
+ "support": 15,
1280
+ "predicted": 9
1281
+ },
1282
+ "T1059.003": {
1283
+ "precision": 0.0,
1284
+ "recall": 0.0,
1285
+ "f1": 0.0,
1286
+ "support": 51,
1287
+ "predicted": 18
1288
+ },
1289
+ "T1068": {
1290
+ "precision": 1.0,
1291
+ "recall": 0.3333,
1292
+ "f1": 0.5,
1293
+ "support": 3,
1294
+ "predicted": 1
1295
+ },
1296
+ "T1070.004": {
1297
+ "precision": 0.0909,
1298
+ "recall": 0.0833,
1299
+ "f1": 0.087,
1300
+ "support": 12,
1301
+ "predicted": 11
1302
+ },
1303
+ "T1071.001": {
1304
+ "precision": 0.1429,
1305
+ "recall": 0.0476,
1306
+ "f1": 0.0714,
1307
+ "support": 21,
1308
+ "predicted": 7
1309
+ },
1310
+ "T1072": {
1311
+ "precision": 0.0,
1312
+ "recall": 0.0,
1313
+ "f1": 0.0,
1314
+ "support": 3,
1315
+ "predicted": 0
1316
+ },
1317
+ "T1074.001": {
1318
+ "precision": 0.5,
1319
+ "recall": 0.3333,
1320
+ "f1": 0.4,
1321
+ "support": 3,
1322
+ "predicted": 2
1323
+ },
1324
+ "T1078": {
1325
+ "precision": 0.5,
1326
+ "recall": 0.2174,
1327
+ "f1": 0.303,
1328
+ "support": 23,
1329
+ "predicted": 10
1330
+ },
1331
+ "T1082": {
1332
+ "precision": 0.1429,
1333
+ "recall": 0.0417,
1334
+ "f1": 0.0645,
1335
+ "support": 24,
1336
+ "predicted": 7
1337
+ },
1338
+ "T1083": {
1339
+ "precision": 0.1111,
1340
+ "recall": 0.0667,
1341
+ "f1": 0.0833,
1342
+ "support": 15,
1343
+ "predicted": 9
1344
+ },
1345
+ "T1090": {
1346
+ "precision": 0.5238,
1347
+ "recall": 0.4783,
1348
+ "f1": 0.5,
1349
+ "support": 23,
1350
+ "predicted": 21
1351
+ },
1352
+ "T1095": {
1353
+ "precision": 0.0,
1354
+ "recall": 0.0,
1355
+ "f1": 0.0,
1356
+ "support": 3,
1357
+ "predicted": 0
1358
+ },
1359
+ "T1105": {
1360
+ "precision": 0.2,
1361
+ "recall": 0.0175,
1362
+ "f1": 0.0323,
1363
+ "support": 57,
1364
+ "predicted": 5
1365
+ },
1366
+ "T1106": {
1367
+ "precision": 0.0,
1368
+ "recall": 0.0,
1369
+ "f1": 0.0,
1370
+ "support": 34,
1371
+ "predicted": 0
1372
+ },
1373
+ "T1110": {
1374
+ "precision": 0.4286,
1375
+ "recall": 0.1579,
1376
+ "f1": 0.2308,
1377
+ "support": 19,
1378
+ "predicted": 7
1379
+ },
1380
+ "T1112": {
1381
+ "precision": 0.2,
1382
+ "recall": 0.0714,
1383
+ "f1": 0.1053,
1384
+ "support": 14,
1385
+ "predicted": 5
1386
+ },
1387
+ "T1113": {
1388
+ "precision": 0.5,
1389
+ "recall": 0.2222,
1390
+ "f1": 0.3077,
1391
+ "support": 9,
1392
+ "predicted": 4
1393
+ },
1394
+ "T1140": {
1395
+ "precision": 0.0,
1396
+ "recall": 0.0,
1397
+ "f1": 0.0,
1398
+ "support": 68,
1399
+ "predicted": 2
1400
+ },
1401
+ "T1190": {
1402
+ "precision": 0.4,
1403
+ "recall": 0.1333,
1404
+ "f1": 0.2,
1405
+ "support": 15,
1406
+ "predicted": 5
1407
+ },
1408
+ "T1204.002": {
1409
+ "precision": 0.0909,
1410
+ "recall": 0.0667,
1411
+ "f1": 0.0769,
1412
+ "support": 15,
1413
+ "predicted": 11
1414
+ },
1415
+ "T1210": {
1416
+ "precision": 0.0,
1417
+ "recall": 0.0,
1418
+ "f1": 0.0,
1419
+ "support": 2,
1420
+ "predicted": 0
1421
+ },
1422
+ "T1218.011": {
1423
+ "precision": 0.5,
1424
+ "recall": 0.7857,
1425
+ "f1": 0.6111,
1426
+ "support": 14,
1427
+ "predicted": 22
1428
+ },
1429
+ "T1219": {
1430
+ "precision": 0.0,
1431
+ "recall": 0.0,
1432
+ "f1": 0.0,
1433
+ "support": 19,
1434
+ "predicted": 0
1435
+ },
1436
+ "T1484.001": {
1437
+ "precision": 0.0,
1438
+ "recall": 0.0,
1439
+ "f1": 0.0,
1440
+ "support": 5,
1441
+ "predicted": 1
1442
+ },
1443
+ "T1518.001": {
1444
+ "precision": 0.2,
1445
+ "recall": 0.1429,
1446
+ "f1": 0.1667,
1447
+ "support": 7,
1448
+ "predicted": 5
1449
+ },
1450
+ "T1543.003": {
1451
+ "precision": 0.7143,
1452
+ "recall": 0.3846,
1453
+ "f1": 0.5,
1454
+ "support": 13,
1455
+ "predicted": 7
1456
+ },
1457
+ "T1547.001": {
1458
+ "precision": 0.3333,
1459
+ "recall": 0.125,
1460
+ "f1": 0.1818,
1461
+ "support": 16,
1462
+ "predicted": 6
1463
+ },
1464
+ "T1548.002": {
1465
+ "precision": 0.0,
1466
+ "recall": 0.0,
1467
+ "f1": 0.0,
1468
+ "support": 2,
1469
+ "predicted": 2
1470
+ },
1471
+ "T1552.001": {
1472
+ "precision": 0.5,
1473
+ "recall": 0.25,
1474
+ "f1": 0.3333,
1475
+ "support": 4,
1476
+ "predicted": 2
1477
+ },
1478
+ "T1562.001": {
1479
+ "precision": 0.0,
1480
+ "recall": 0.0,
1481
+ "f1": 0.0,
1482
+ "support": 26,
1483
+ "predicted": 0
1484
+ },
1485
+ "T1564.001": {
1486
+ "precision": 0.0,
1487
+ "recall": 0.0,
1488
+ "f1": 0.0,
1489
+ "support": 3,
1490
+ "predicted": 2
1491
+ },
1492
+ "T1566.001": {
1493
+ "precision": 0.0541,
1494
+ "recall": 0.3077,
1495
+ "f1": 0.092,
1496
+ "support": 13,
1497
+ "predicted": 74
1498
+ },
1499
+ "T1569.002": {
1500
+ "precision": 0.0,
1501
+ "recall": 0.0,
1502
+ "f1": 0.0,
1503
+ "support": 5,
1504
+ "predicted": 4
1505
+ },
1506
+ "T1570": {
1507
+ "precision": 0.4286,
1508
+ "recall": 0.5,
1509
+ "f1": 0.4615,
1510
+ "support": 6,
1511
+ "predicted": 7
1512
+ },
1513
+ "T1573.001": {
1514
+ "precision": 0.0,
1515
+ "recall": 0.0,
1516
+ "f1": 0.0,
1517
+ "support": 16,
1518
+ "predicted": 3
1519
+ },
1520
+ "T1574.002": {
1521
+ "precision": 0.0,
1522
+ "recall": 0.0,
1523
+ "f1": 0.0,
1524
+ "support": 26,
1525
+ "predicted": 0
1526
+ }
1527
+ }
1528
+ }
1529
+ },
1530
+ "tfidf_lr": {
1531
+ "global_threshold": {
1532
+ "threshold": 0.2,
1533
+ "micro_f1": 0.4724,
1534
+ "macro_f1": 0.4492,
1535
+ "micro_precision": 0.3509,
1536
+ "micro_recall": 0.7223,
1537
+ "macro_precision": 0.3785,
1538
+ "macro_recall": 0.6336,
1539
+ "n_examples": 3635,
1540
+ "n_positive_labels": 911,
1541
+ "per_class": {
1542
+ "T1003.001": {
1543
+ "precision": 0.4474,
1544
+ "recall": 0.8095,
1545
+ "f1": 0.5763,
1546
+ "support": 21,
1547
+ "predicted": 38
1548
+ },
1549
+ "T1005": {
1550
+ "precision": 0.093,
1551
+ "recall": 0.5,
1552
+ "f1": 0.1569,
1553
+ "support": 8,
1554
+ "predicted": 43
1555
+ },
1556
+ "T1012": {
1557
+ "precision": 0.1333,
1558
+ "recall": 0.5,
1559
+ "f1": 0.2105,
1560
+ "support": 4,
1561
+ "predicted": 15
1562
+ },
1563
+ "T1016": {
1564
+ "precision": 0.2632,
1565
+ "recall": 0.4545,
1566
+ "f1": 0.3333,
1567
+ "support": 11,
1568
+ "predicted": 19
1569
+ },
1570
+ "T1021.001": {
1571
+ "precision": 0.3243,
1572
+ "recall": 1.0,
1573
+ "f1": 0.4898,
1574
+ "support": 12,
1575
+ "predicted": 37
1576
+ },
1577
+ "T1027": {
1578
+ "precision": 0.2971,
1579
+ "recall": 0.8039,
1580
+ "f1": 0.4339,
1581
+ "support": 102,
1582
+ "predicted": 276
1583
+ },
1584
+ "T1033": {
1585
+ "precision": 0.4615,
1586
+ "recall": 0.9231,
1587
+ "f1": 0.6154,
1588
+ "support": 13,
1589
+ "predicted": 26
1590
+ },
1591
+ "T1036.005": {
1592
+ "precision": 0.2727,
1593
+ "recall": 0.6,
1594
+ "f1": 0.375,
1595
+ "support": 10,
1596
+ "predicted": 22
1597
+ },
1598
+ "T1041": {
1599
+ "precision": 0.1695,
1600
+ "recall": 0.5882,
1601
+ "f1": 0.2632,
1602
+ "support": 17,
1603
+ "predicted": 59
1604
+ },
1605
+ "T1047": {
1606
+ "precision": 0.6286,
1607
+ "recall": 0.7857,
1608
+ "f1": 0.6984,
1609
+ "support": 28,
1610
+ "predicted": 35
1611
+ },
1612
+ "T1053.005": {
1613
+ "precision": 0.5333,
1614
+ "recall": 0.9231,
1615
+ "f1": 0.6761,
1616
+ "support": 26,
1617
+ "predicted": 45
1618
+ },
1619
+ "T1055": {
1620
+ "precision": 0.4921,
1621
+ "recall": 0.775,
1622
+ "f1": 0.6019,
1623
+ "support": 40,
1624
+ "predicted": 63
1625
+ },
1626
+ "T1056.001": {
1627
+ "precision": 0.6667,
1628
+ "recall": 0.9333,
1629
+ "f1": 0.7778,
1630
+ "support": 15,
1631
+ "predicted": 21
1632
+ },
1633
+ "T1057": {
1634
+ "precision": 0.3448,
1635
+ "recall": 0.6667,
1636
+ "f1": 0.4545,
1637
+ "support": 15,
1638
+ "predicted": 29
1639
+ },
1640
+ "T1059.003": {
1641
+ "precision": 0.3362,
1642
+ "recall": 0.7647,
1643
+ "f1": 0.4671,
1644
+ "support": 51,
1645
+ "predicted": 116
1646
+ },
1647
+ "T1068": {
1648
+ "precision": 1.0,
1649
+ "recall": 0.3333,
1650
+ "f1": 0.5,
1651
+ "support": 3,
1652
+ "predicted": 1
1653
+ },
1654
+ "T1070.004": {
1655
+ "precision": 0.3,
1656
+ "recall": 0.75,
1657
+ "f1": 0.4286,
1658
+ "support": 12,
1659
+ "predicted": 30
1660
+ },
1661
+ "T1071.001": {
1662
+ "precision": 0.2769,
1663
+ "recall": 0.8571,
1664
+ "f1": 0.4186,
1665
+ "support": 21,
1666
+ "predicted": 65
1667
+ },
1668
+ "T1072": {
1669
+ "precision": 0.0,
1670
+ "recall": 0.0,
1671
+ "f1": 0.0,
1672
+ "support": 3,
1673
+ "predicted": 0
1674
+ },
1675
+ "T1074.001": {
1676
+ "precision": 0.2857,
1677
+ "recall": 0.6667,
1678
+ "f1": 0.4,
1679
+ "support": 3,
1680
+ "predicted": 7
1681
+ },
1682
+ "T1078": {
1683
+ "precision": 0.2143,
1684
+ "recall": 0.7826,
1685
+ "f1": 0.3364,
1686
+ "support": 23,
1687
+ "predicted": 84
1688
+ },
1689
+ "T1082": {
1690
+ "precision": 0.2037,
1691
+ "recall": 0.4583,
1692
+ "f1": 0.2821,
1693
+ "support": 24,
1694
+ "predicted": 54
1695
+ },
1696
+ "T1083": {
1697
+ "precision": 0.2391,
1698
+ "recall": 0.7333,
1699
+ "f1": 0.3607,
1700
+ "support": 15,
1701
+ "predicted": 46
1702
+ },
1703
+ "T1090": {
1704
+ "precision": 0.3889,
1705
+ "recall": 0.6087,
1706
+ "f1": 0.4746,
1707
+ "support": 23,
1708
+ "predicted": 36
1709
+ },
1710
+ "T1095": {
1711
+ "precision": 0.25,
1712
+ "recall": 0.3333,
1713
+ "f1": 0.2857,
1714
+ "support": 3,
1715
+ "predicted": 4
1716
+ },
1717
+ "T1105": {
1718
+ "precision": 0.3101,
1719
+ "recall": 0.7018,
1720
+ "f1": 0.4301,
1721
+ "support": 57,
1722
+ "predicted": 129
1723
+ },
1724
+ "T1106": {
1725
+ "precision": 0.3333,
1726
+ "recall": 0.6176,
1727
+ "f1": 0.433,
1728
+ "support": 34,
1729
+ "predicted": 63
1730
+ },
1731
+ "T1110": {
1732
+ "precision": 0.5556,
1733
+ "recall": 0.7895,
1734
+ "f1": 0.6522,
1735
+ "support": 19,
1736
+ "predicted": 27
1737
+ },
1738
+ "T1112": {
1739
+ "precision": 0.2683,
1740
+ "recall": 0.7857,
1741
+ "f1": 0.4,
1742
+ "support": 14,
1743
+ "predicted": 41
1744
+ },
1745
+ "T1113": {
1746
+ "precision": 0.5,
1747
+ "recall": 0.8889,
1748
+ "f1": 0.64,
1749
+ "support": 9,
1750
+ "predicted": 16
1751
+ },
1752
+ "T1140": {
1753
+ "precision": 0.5039,
1754
+ "recall": 0.9412,
1755
+ "f1": 0.6564,
1756
+ "support": 68,
1757
+ "predicted": 127
1758
+ },
1759
+ "T1190": {
1760
+ "precision": 0.3182,
1761
+ "recall": 0.4667,
1762
+ "f1": 0.3784,
1763
+ "support": 15,
1764
+ "predicted": 22
1765
+ },
1766
+ "T1204.002": {
1767
+ "precision": 0.2647,
1768
+ "recall": 0.6,
1769
+ "f1": 0.3673,
1770
+ "support": 15,
1771
+ "predicted": 34
1772
+ },
1773
+ "T1210": {
1774
+ "precision": 0.0,
1775
+ "recall": 0.0,
1776
+ "f1": 0.0,
1777
+ "support": 2,
1778
+ "predicted": 1
1779
+ },
1780
+ "T1218.011": {
1781
+ "precision": 0.6429,
1782
+ "recall": 0.6429,
1783
+ "f1": 0.6429,
1784
+ "support": 14,
1785
+ "predicted": 14
1786
+ },
1787
+ "T1219": {
1788
+ "precision": 0.375,
1789
+ "recall": 0.3158,
1790
+ "f1": 0.3429,
1791
+ "support": 19,
1792
+ "predicted": 16
1793
+ },
1794
+ "T1484.001": {
1795
+ "precision": 0.5,
1796
+ "recall": 0.4,
1797
+ "f1": 0.4444,
1798
+ "support": 5,
1799
+ "predicted": 4
1800
+ },
1801
+ "T1518.001": {
1802
+ "precision": 0.3333,
1803
+ "recall": 0.5714,
1804
+ "f1": 0.4211,
1805
+ "support": 7,
1806
+ "predicted": 12
1807
+ },
1808
+ "T1543.003": {
1809
+ "precision": 0.6667,
1810
+ "recall": 0.6154,
1811
+ "f1": 0.64,
1812
+ "support": 13,
1813
+ "predicted": 12
1814
+ },
1815
+ "T1547.001": {
1816
+ "precision": 0.4483,
1817
+ "recall": 0.8125,
1818
+ "f1": 0.5778,
1819
+ "support": 16,
1820
+ "predicted": 29
1821
+ },
1822
+ "T1548.002": {
1823
+ "precision": 1.0,
1824
+ "recall": 1.0,
1825
+ "f1": 1.0,
1826
+ "support": 2,
1827
+ "predicted": 2
1828
+ },
1829
+ "T1552.001": {
1830
+ "precision": 0.1111,
1831
+ "recall": 0.25,
1832
+ "f1": 0.1538,
1833
+ "support": 4,
1834
+ "predicted": 9
1835
+ },
1836
+ "T1562.001": {
1837
+ "precision": 0.6,
1838
+ "recall": 0.8077,
1839
+ "f1": 0.6885,
1840
+ "support": 26,
1841
+ "predicted": 35
1842
+ },
1843
+ "T1564.001": {
1844
+ "precision": 0.5,
1845
+ "recall": 0.6667,
1846
+ "f1": 0.5714,
1847
+ "support": 3,
1848
+ "predicted": 4
1849
+ },
1850
+ "T1566.001": {
1851
+ "precision": 0.1786,
1852
+ "recall": 0.3846,
1853
+ "f1": 0.2439,
1854
+ "support": 13,
1855
+ "predicted": 28
1856
+ },
1857
+ "T1569.002": {
1858
+ "precision": 0.1429,
1859
+ "recall": 0.2,
1860
+ "f1": 0.1667,
1861
+ "support": 5,
1862
+ "predicted": 7
1863
+ },
1864
+ "T1570": {
1865
+ "precision": 0.1667,
1866
+ "recall": 0.6667,
1867
+ "f1": 0.2667,
1868
+ "support": 6,
1869
+ "predicted": 24
1870
+ },
1871
+ "T1573.001": {
1872
+ "precision": 0.4286,
1873
+ "recall": 0.5625,
1874
+ "f1": 0.4865,
1875
+ "support": 16,
1876
+ "predicted": 21
1877
+ },
1878
+ "T1574.002": {
1879
+ "precision": 0.7778,
1880
+ "recall": 0.8077,
1881
+ "f1": 0.7925,
1882
+ "support": 26,
1883
+ "predicted": 27
1884
+ }
1885
+ }
1886
+ },
1887
+ "per_class_threshold": {
1888
+ "thresholds": {
1889
+ "T1003.001": 0.65,
1890
+ "T1005": 0.5,
1891
+ "T1012": 0.1,
1892
+ "T1016": 0.25,
1893
+ "T1021.001": 0.4,
1894
+ "T1027": 0.3,
1895
+ "T1033": 0.5,
1896
+ "T1036.005": 0.25,
1897
+ "T1041": 0.4,
1898
+ "T1047": 0.2,
1899
+ "T1053.005": 0.35,
1900
+ "T1055": 0.4,
1901
+ "T1056.001": 0.1,
1902
+ "T1057": 0.55,
1903
+ "T1059.003": 0.6,
1904
+ "T1068": 0.15,
1905
+ "T1070.004": 0.15,
1906
+ "T1071.001": 0.55,
1907
+ "T1072": 0.05,
1908
+ "T1074.001": 0.05,
1909
+ "T1078": 0.35,
1910
+ "T1082": 0.8,
1911
+ "T1083": 0.45,
1912
+ "T1090": 0.4,
1913
+ "T1095": 0.2,
1914
+ "T1105": 0.65,
1915
+ "T1106": 0.5,
1916
+ "T1110": 0.2,
1917
+ "T1112": 0.6,
1918
+ "T1113": 0.25,
1919
+ "T1140": 0.5,
1920
+ "T1190": 0.4,
1921
+ "T1204.002": 0.4,
1922
+ "T1210": 0.05,
1923
+ "T1218.011": 0.15,
1924
+ "T1219": 0.2,
1925
+ "T1484.001": 0.1,
1926
+ "T1518.001": 0.1,
1927
+ "T1543.003": 0.3,
1928
+ "T1547.001": 0.45,
1929
+ "T1548.002": 0.05,
1930
+ "T1552.001": 0.05,
1931
+ "T1562.001": 0.55,
1932
+ "T1564.001": 0.2,
1933
+ "T1566.001": 0.55,
1934
+ "T1569.002": 0.05,
1935
+ "T1570": 0.55,
1936
+ "T1573.001": 0.8,
1937
+ "T1574.002": 0.25
1938
+ },
1939
+ "micro_f1": 0.5002,
1940
+ "macro_f1": 0.4536,
1941
+ "micro_precision": 0.4276,
1942
+ "micro_recall": 0.6026,
1943
+ "macro_precision": 0.4343,
1944
+ "macro_recall": 0.5681,
1945
+ "n_examples": 3635,
1946
+ "n_positive_labels": 911,
1947
+ "per_class": {
1948
+ "T1003.001": {
1949
+ "precision": 0.5926,
1950
+ "recall": 0.7619,
1951
+ "f1": 0.6667,
1952
+ "support": 21,
1953
+ "predicted": 27
1954
+ },
1955
+ "T1005": {
1956
+ "precision": 0.1875,
1957
+ "recall": 0.375,
1958
+ "f1": 0.25,
1959
+ "support": 8,
1960
+ "predicted": 16
1961
+ },
1962
+ "T1012": {
1963
+ "precision": 0.1667,
1964
+ "recall": 0.75,
1965
+ "f1": 0.2727,
1966
+ "support": 4,
1967
+ "predicted": 18
1968
+ },
1969
+ "T1016": {
1970
+ "precision": 0.1875,
1971
+ "recall": 0.2727,
1972
+ "f1": 0.2222,
1973
+ "support": 11,
1974
+ "predicted": 16
1975
+ },
1976
+ "T1021.001": {
1977
+ "precision": 0.3448,
1978
+ "recall": 0.8333,
1979
+ "f1": 0.4878,
1980
+ "support": 12,
1981
+ "predicted": 29
1982
+ },
1983
+ "T1027": {
1984
+ "precision": 0.3636,
1985
+ "recall": 0.7451,
1986
+ "f1": 0.4887,
1987
+ "support": 102,
1988
+ "predicted": 209
1989
+ },
1990
+ "T1033": {
1991
+ "precision": 0.6364,
1992
+ "recall": 0.5385,
1993
+ "f1": 0.5833,
1994
+ "support": 13,
1995
+ "predicted": 11
1996
+ },
1997
+ "T1036.005": {
1998
+ "precision": 0.3125,
1999
+ "recall": 0.5,
2000
+ "f1": 0.3846,
2001
+ "support": 10,
2002
+ "predicted": 16
2003
+ },
2004
+ "T1041": {
2005
+ "precision": 0.2571,
2006
+ "recall": 0.5294,
2007
+ "f1": 0.3462,
2008
+ "support": 17,
2009
+ "predicted": 35
2010
+ },
2011
+ "T1047": {
2012
+ "precision": 0.6286,
2013
+ "recall": 0.7857,
2014
+ "f1": 0.6984,
2015
+ "support": 28,
2016
+ "predicted": 35
2017
+ },
2018
+ "T1053.005": {
2019
+ "precision": 0.55,
2020
+ "recall": 0.8462,
2021
+ "f1": 0.6667,
2022
+ "support": 26,
2023
+ "predicted": 40
2024
+ },
2025
+ "T1055": {
2026
+ "precision": 0.5918,
2027
+ "recall": 0.725,
2028
+ "f1": 0.6517,
2029
+ "support": 40,
2030
+ "predicted": 49
2031
+ },
2032
+ "T1056.001": {
2033
+ "precision": 0.6,
2034
+ "recall": 1.0,
2035
+ "f1": 0.75,
2036
+ "support": 15,
2037
+ "predicted": 25
2038
+ },
2039
+ "T1057": {
2040
+ "precision": 0.75,
2041
+ "recall": 0.6,
2042
+ "f1": 0.6667,
2043
+ "support": 15,
2044
+ "predicted": 12
2045
+ },
2046
+ "T1059.003": {
2047
+ "precision": 0.5526,
2048
+ "recall": 0.4118,
2049
+ "f1": 0.4719,
2050
+ "support": 51,
2051
+ "predicted": 38
2052
+ },
2053
+ "T1068": {
2054
+ "precision": 1.0,
2055
+ "recall": 0.3333,
2056
+ "f1": 0.5,
2057
+ "support": 3,
2058
+ "predicted": 1
2059
+ },
2060
+ "T1070.004": {
2061
+ "precision": 0.2703,
2062
+ "recall": 0.8333,
2063
+ "f1": 0.4082,
2064
+ "support": 12,
2065
+ "predicted": 37
2066
+ },
2067
+ "T1071.001": {
2068
+ "precision": 0.4375,
2069
+ "recall": 0.6667,
2070
+ "f1": 0.5283,
2071
+ "support": 21,
2072
+ "predicted": 32
2073
+ },
2074
+ "T1072": {
2075
+ "precision": 0.0,
2076
+ "recall": 0.0,
2077
+ "f1": 0.0,
2078
+ "support": 3,
2079
+ "predicted": 0
2080
+ },
2081
+ "T1074.001": {
2082
+ "precision": 0.0714,
2083
+ "recall": 0.6667,
2084
+ "f1": 0.129,
2085
+ "support": 3,
2086
+ "predicted": 28
2087
+ },
2088
+ "T1078": {
2089
+ "precision": 0.2321,
2090
+ "recall": 0.5652,
2091
+ "f1": 0.3291,
2092
+ "support": 23,
2093
+ "predicted": 56
2094
+ },
2095
+ "T1082": {
2096
+ "precision": 0.6667,
2097
+ "recall": 0.3333,
2098
+ "f1": 0.4444,
2099
+ "support": 24,
2100
+ "predicted": 12
2101
+ },
2102
+ "T1083": {
2103
+ "precision": 0.3478,
2104
+ "recall": 0.5333,
2105
+ "f1": 0.4211,
2106
+ "support": 15,
2107
+ "predicted": 23
2108
+ },
2109
+ "T1090": {
2110
+ "precision": 0.5238,
2111
+ "recall": 0.4783,
2112
+ "f1": 0.5,
2113
+ "support": 23,
2114
+ "predicted": 21
2115
+ },
2116
+ "T1095": {
2117
+ "precision": 0.25,
2118
+ "recall": 0.3333,
2119
+ "f1": 0.2857,
2120
+ "support": 3,
2121
+ "predicted": 4
2122
+ },
2123
+ "T1105": {
2124
+ "precision": 0.5152,
2125
+ "recall": 0.2982,
2126
+ "f1": 0.3778,
2127
+ "support": 57,
2128
+ "predicted": 33
2129
+ },
2130
+ "T1106": {
2131
+ "precision": 0.5,
2132
+ "recall": 0.4118,
2133
+ "f1": 0.4516,
2134
+ "support": 34,
2135
+ "predicted": 28
2136
+ },
2137
+ "T1110": {
2138
+ "precision": 0.5556,
2139
+ "recall": 0.7895,
2140
+ "f1": 0.6522,
2141
+ "support": 19,
2142
+ "predicted": 27
2143
+ },
2144
+ "T1112": {
2145
+ "precision": 0.4,
2146
+ "recall": 0.7143,
2147
+ "f1": 0.5128,
2148
+ "support": 14,
2149
+ "predicted": 25
2150
+ },
2151
+ "T1113": {
2152
+ "precision": 0.5,
2153
+ "recall": 0.8889,
2154
+ "f1": 0.64,
2155
+ "support": 9,
2156
+ "predicted": 16
2157
+ },
2158
+ "T1140": {
2159
+ "precision": 0.7073,
2160
+ "recall": 0.8529,
2161
+ "f1": 0.7733,
2162
+ "support": 68,
2163
+ "predicted": 82
2164
+ },
2165
+ "T1190": {
2166
+ "precision": 0.5,
2167
+ "recall": 0.4667,
2168
+ "f1": 0.4828,
2169
+ "support": 15,
2170
+ "predicted": 14
2171
+ },
2172
+ "T1204.002": {
2173
+ "precision": 0.4211,
2174
+ "recall": 0.5333,
2175
+ "f1": 0.4706,
2176
+ "support": 15,
2177
+ "predicted": 19
2178
+ },
2179
+ "T1210": {
2180
+ "precision": 0.0,
2181
+ "recall": 0.0,
2182
+ "f1": 0.0,
2183
+ "support": 2,
2184
+ "predicted": 34
2185
+ },
2186
+ "T1218.011": {
2187
+ "precision": 0.625,
2188
+ "recall": 0.7143,
2189
+ "f1": 0.6667,
2190
+ "support": 14,
2191
+ "predicted": 16
2192
+ },
2193
+ "T1219": {
2194
+ "precision": 0.375,
2195
+ "recall": 0.3158,
2196
+ "f1": 0.3429,
2197
+ "support": 19,
2198
+ "predicted": 16
2199
+ },
2200
+ "T1484.001": {
2201
+ "precision": 0.3333,
2202
+ "recall": 0.6,
2203
+ "f1": 0.4286,
2204
+ "support": 5,
2205
+ "predicted": 9
2206
+ },
2207
+ "T1518.001": {
2208
+ "precision": 0.2381,
2209
+ "recall": 0.7143,
2210
+ "f1": 0.3571,
2211
+ "support": 7,
2212
+ "predicted": 21
2213
+ },
2214
+ "T1543.003": {
2215
+ "precision": 0.7778,
2216
+ "recall": 0.5385,
2217
+ "f1": 0.6364,
2218
+ "support": 13,
2219
+ "predicted": 9
2220
+ },
2221
+ "T1547.001": {
2222
+ "precision": 0.5789,
2223
+ "recall": 0.6875,
2224
+ "f1": 0.6286,
2225
+ "support": 16,
2226
+ "predicted": 19
2227
+ },
2228
+ "T1548.002": {
2229
+ "precision": 0.4,
2230
+ "recall": 1.0,
2231
+ "f1": 0.5714,
2232
+ "support": 2,
2233
+ "predicted": 5
2234
+ },
2235
+ "T1552.001": {
2236
+ "precision": 0.0833,
2237
+ "recall": 0.5,
2238
+ "f1": 0.1429,
2239
+ "support": 4,
2240
+ "predicted": 24
2241
+ },
2242
+ "T1562.001": {
2243
+ "precision": 0.7619,
2244
+ "recall": 0.6154,
2245
+ "f1": 0.6809,
2246
+ "support": 26,
2247
+ "predicted": 21
2248
+ },
2249
+ "T1564.001": {
2250
+ "precision": 0.5,
2251
+ "recall": 0.6667,
2252
+ "f1": 0.5714,
2253
+ "support": 3,
2254
+ "predicted": 4
2255
+ },
2256
+ "T1566.001": {
2257
+ "precision": 0.1111,
2258
+ "recall": 0.0769,
2259
+ "f1": 0.0909,
2260
+ "support": 13,
2261
+ "predicted": 9
2262
+ },
2263
+ "T1569.002": {
2264
+ "precision": 0.1304,
2265
+ "recall": 0.6,
2266
+ "f1": 0.2143,
2267
+ "support": 5,
2268
+ "predicted": 23
2269
+ },
2270
+ "T1570": {
2271
+ "precision": 0.3,
2272
+ "recall": 0.5,
2273
+ "f1": 0.375,
2274
+ "support": 6,
2275
+ "predicted": 10
2276
+ },
2277
+ "T1573.001": {
2278
+ "precision": 0.6667,
2279
+ "recall": 0.125,
2280
+ "f1": 0.2105,
2281
+ "support": 16,
2282
+ "predicted": 3
2283
+ },
2284
+ "T1574.002": {
2285
+ "precision": 0.7778,
2286
+ "recall": 0.8077,
2287
+ "f1": 0.7925,
2288
+ "support": 26,
2289
+ "predicted": 27
2290
+ }
2291
+ }
2292
+ }
2293
+ }
2294
+ }
results/baselines__random.json ADDED
@@ -0,0 +1,2294 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "frequency": {
3
+ "global_threshold": {
4
+ "threshold": 0.05,
5
+ "micro_f1": 0.0,
6
+ "macro_f1": 0.0,
7
+ "micro_precision": 0.0,
8
+ "micro_recall": 0.0,
9
+ "macro_precision": 0.0,
10
+ "macro_recall": 0.0,
11
+ "n_examples": 2767,
12
+ "n_positive_labels": 737,
13
+ "per_class": {
14
+ "T1003.001": {
15
+ "precision": 0.0,
16
+ "recall": 0.0,
17
+ "f1": 0.0,
18
+ "support": 17,
19
+ "predicted": 0
20
+ },
21
+ "T1005": {
22
+ "precision": 0.0,
23
+ "recall": 0.0,
24
+ "f1": 0.0,
25
+ "support": 12,
26
+ "predicted": 0
27
+ },
28
+ "T1012": {
29
+ "precision": 0.0,
30
+ "recall": 0.0,
31
+ "f1": 0.0,
32
+ "support": 4,
33
+ "predicted": 0
34
+ },
35
+ "T1016": {
36
+ "precision": 0.0,
37
+ "recall": 0.0,
38
+ "f1": 0.0,
39
+ "support": 9,
40
+ "predicted": 0
41
+ },
42
+ "T1021.001": {
43
+ "precision": 0.0,
44
+ "recall": 0.0,
45
+ "f1": 0.0,
46
+ "support": 16,
47
+ "predicted": 0
48
+ },
49
+ "T1027": {
50
+ "precision": 0.0,
51
+ "recall": 0.0,
52
+ "f1": 0.0,
53
+ "support": 96,
54
+ "predicted": 0
55
+ },
56
+ "T1033": {
57
+ "precision": 0.0,
58
+ "recall": 0.0,
59
+ "f1": 0.0,
60
+ "support": 6,
61
+ "predicted": 0
62
+ },
63
+ "T1036.005": {
64
+ "precision": 0.0,
65
+ "recall": 0.0,
66
+ "f1": 0.0,
67
+ "support": 8,
68
+ "predicted": 0
69
+ },
70
+ "T1041": {
71
+ "precision": 0.0,
72
+ "recall": 0.0,
73
+ "f1": 0.0,
74
+ "support": 13,
75
+ "predicted": 0
76
+ },
77
+ "T1047": {
78
+ "precision": 0.0,
79
+ "recall": 0.0,
80
+ "f1": 0.0,
81
+ "support": 10,
82
+ "predicted": 0
83
+ },
84
+ "T1053.005": {
85
+ "precision": 0.0,
86
+ "recall": 0.0,
87
+ "f1": 0.0,
88
+ "support": 14,
89
+ "predicted": 0
90
+ },
91
+ "T1055": {
92
+ "precision": 0.0,
93
+ "recall": 0.0,
94
+ "f1": 0.0,
95
+ "support": 43,
96
+ "predicted": 0
97
+ },
98
+ "T1056.001": {
99
+ "precision": 0.0,
100
+ "recall": 0.0,
101
+ "f1": 0.0,
102
+ "support": 6,
103
+ "predicted": 0
104
+ },
105
+ "T1057": {
106
+ "precision": 0.0,
107
+ "recall": 0.0,
108
+ "f1": 0.0,
109
+ "support": 12,
110
+ "predicted": 0
111
+ },
112
+ "T1059.003": {
113
+ "precision": 0.0,
114
+ "recall": 0.0,
115
+ "f1": 0.0,
116
+ "support": 45,
117
+ "predicted": 0
118
+ },
119
+ "T1068": {
120
+ "precision": 0.0,
121
+ "recall": 0.0,
122
+ "f1": 0.0,
123
+ "support": 1,
124
+ "predicted": 0
125
+ },
126
+ "T1070.004": {
127
+ "precision": 0.0,
128
+ "recall": 0.0,
129
+ "f1": 0.0,
130
+ "support": 15,
131
+ "predicted": 0
132
+ },
133
+ "T1071.001": {
134
+ "precision": 0.0,
135
+ "recall": 0.0,
136
+ "f1": 0.0,
137
+ "support": 28,
138
+ "predicted": 0
139
+ },
140
+ "T1072": {
141
+ "precision": 0.0,
142
+ "recall": 0.0,
143
+ "f1": 0.0,
144
+ "support": 2,
145
+ "predicted": 0
146
+ },
147
+ "T1074.001": {
148
+ "precision": 0.0,
149
+ "recall": 0.0,
150
+ "f1": 0.0,
151
+ "support": 3,
152
+ "predicted": 0
153
+ },
154
+ "T1078": {
155
+ "precision": 0.0,
156
+ "recall": 0.0,
157
+ "f1": 0.0,
158
+ "support": 21,
159
+ "predicted": 0
160
+ },
161
+ "T1082": {
162
+ "precision": 0.0,
163
+ "recall": 0.0,
164
+ "f1": 0.0,
165
+ "support": 17,
166
+ "predicted": 0
167
+ },
168
+ "T1083": {
169
+ "precision": 0.0,
170
+ "recall": 0.0,
171
+ "f1": 0.0,
172
+ "support": 13,
173
+ "predicted": 0
174
+ },
175
+ "T1090": {
176
+ "precision": 0.0,
177
+ "recall": 0.0,
178
+ "f1": 0.0,
179
+ "support": 18,
180
+ "predicted": 0
181
+ },
182
+ "T1095": {
183
+ "precision": 0.0,
184
+ "recall": 0.0,
185
+ "f1": 0.0,
186
+ "support": 12,
187
+ "predicted": 0
188
+ },
189
+ "T1105": {
190
+ "precision": 0.0,
191
+ "recall": 0.0,
192
+ "f1": 0.0,
193
+ "support": 40,
194
+ "predicted": 0
195
+ },
196
+ "T1106": {
197
+ "precision": 0.0,
198
+ "recall": 0.0,
199
+ "f1": 0.0,
200
+ "support": 21,
201
+ "predicted": 0
202
+ },
203
+ "T1110": {
204
+ "precision": 0.0,
205
+ "recall": 0.0,
206
+ "f1": 0.0,
207
+ "support": 3,
208
+ "predicted": 0
209
+ },
210
+ "T1112": {
211
+ "precision": 0.0,
212
+ "recall": 0.0,
213
+ "f1": 0.0,
214
+ "support": 15,
215
+ "predicted": 0
216
+ },
217
+ "T1113": {
218
+ "precision": 0.0,
219
+ "recall": 0.0,
220
+ "f1": 0.0,
221
+ "support": 8,
222
+ "predicted": 0
223
+ },
224
+ "T1140": {
225
+ "precision": 0.0,
226
+ "recall": 0.0,
227
+ "f1": 0.0,
228
+ "support": 67,
229
+ "predicted": 0
230
+ },
231
+ "T1190": {
232
+ "precision": 0.0,
233
+ "recall": 0.0,
234
+ "f1": 0.0,
235
+ "support": 13,
236
+ "predicted": 0
237
+ },
238
+ "T1204.002": {
239
+ "precision": 0.0,
240
+ "recall": 0.0,
241
+ "f1": 0.0,
242
+ "support": 10,
243
+ "predicted": 0
244
+ },
245
+ "T1210": {
246
+ "precision": 0.0,
247
+ "recall": 0.0,
248
+ "f1": 0.0,
249
+ "support": 2,
250
+ "predicted": 0
251
+ },
252
+ "T1218.011": {
253
+ "precision": 0.0,
254
+ "recall": 0.0,
255
+ "f1": 0.0,
256
+ "support": 11,
257
+ "predicted": 0
258
+ },
259
+ "T1219": {
260
+ "precision": 0.0,
261
+ "recall": 0.0,
262
+ "f1": 0.0,
263
+ "support": 4,
264
+ "predicted": 0
265
+ },
266
+ "T1484.001": {
267
+ "precision": 0.0,
268
+ "recall": 0.0,
269
+ "f1": 0.0,
270
+ "support": 4,
271
+ "predicted": 0
272
+ },
273
+ "T1518.001": {
274
+ "precision": 0.0,
275
+ "recall": 0.0,
276
+ "f1": 0.0,
277
+ "support": 4,
278
+ "predicted": 0
279
+ },
280
+ "T1543.003": {
281
+ "precision": 0.0,
282
+ "recall": 0.0,
283
+ "f1": 0.0,
284
+ "support": 4,
285
+ "predicted": 0
286
+ },
287
+ "T1547.001": {
288
+ "precision": 0.0,
289
+ "recall": 0.0,
290
+ "f1": 0.0,
291
+ "support": 10,
292
+ "predicted": 0
293
+ },
294
+ "T1548.002": {
295
+ "precision": 0.0,
296
+ "recall": 0.0,
297
+ "f1": 0.0,
298
+ "support": 6,
299
+ "predicted": 0
300
+ },
301
+ "T1552.001": {
302
+ "precision": 0.0,
303
+ "recall": 0.0,
304
+ "f1": 0.0,
305
+ "support": 3,
306
+ "predicted": 0
307
+ },
308
+ "T1562.001": {
309
+ "precision": 0.0,
310
+ "recall": 0.0,
311
+ "f1": 0.0,
312
+ "support": 13,
313
+ "predicted": 0
314
+ },
315
+ "T1564.001": {
316
+ "precision": 0.0,
317
+ "recall": 0.0,
318
+ "f1": 0.0,
319
+ "support": 1,
320
+ "predicted": 0
321
+ },
322
+ "T1566.001": {
323
+ "precision": 0.0,
324
+ "recall": 0.0,
325
+ "f1": 0.0,
326
+ "support": 15,
327
+ "predicted": 0
328
+ },
329
+ "T1569.002": {
330
+ "precision": 0.0,
331
+ "recall": 0.0,
332
+ "f1": 0.0,
333
+ "support": 2,
334
+ "predicted": 0
335
+ },
336
+ "T1570": {
337
+ "precision": 0.0,
338
+ "recall": 0.0,
339
+ "f1": 0.0,
340
+ "support": 13,
341
+ "predicted": 0
342
+ },
343
+ "T1573.001": {
344
+ "precision": 0.0,
345
+ "recall": 0.0,
346
+ "f1": 0.0,
347
+ "support": 9,
348
+ "predicted": 0
349
+ },
350
+ "T1574.002": {
351
+ "precision": 0.0,
352
+ "recall": 0.0,
353
+ "f1": 0.0,
354
+ "support": 18,
355
+ "predicted": 0
356
+ }
357
+ }
358
+ },
359
+ "per_class_threshold": {
360
+ "thresholds": {
361
+ "T1003.001": 0.05,
362
+ "T1005": 0.05,
363
+ "T1012": 0.05,
364
+ "T1016": 0.05,
365
+ "T1021.001": 0.05,
366
+ "T1027": 0.05,
367
+ "T1033": 0.05,
368
+ "T1036.005": 0.05,
369
+ "T1041": 0.05,
370
+ "T1047": 0.05,
371
+ "T1053.005": 0.05,
372
+ "T1055": 0.05,
373
+ "T1056.001": 0.05,
374
+ "T1057": 0.05,
375
+ "T1059.003": 0.05,
376
+ "T1068": 0.05,
377
+ "T1070.004": 0.05,
378
+ "T1071.001": 0.05,
379
+ "T1072": 0.05,
380
+ "T1074.001": 0.05,
381
+ "T1078": 0.05,
382
+ "T1082": 0.05,
383
+ "T1083": 0.05,
384
+ "T1090": 0.05,
385
+ "T1095": 0.05,
386
+ "T1105": 0.05,
387
+ "T1106": 0.05,
388
+ "T1110": 0.05,
389
+ "T1112": 0.05,
390
+ "T1113": 0.05,
391
+ "T1140": 0.05,
392
+ "T1190": 0.05,
393
+ "T1204.002": 0.05,
394
+ "T1210": 0.05,
395
+ "T1218.011": 0.05,
396
+ "T1219": 0.05,
397
+ "T1484.001": 0.05,
398
+ "T1518.001": 0.05,
399
+ "T1543.003": 0.05,
400
+ "T1547.001": 0.05,
401
+ "T1548.002": 0.05,
402
+ "T1552.001": 0.05,
403
+ "T1562.001": 0.05,
404
+ "T1564.001": 0.05,
405
+ "T1566.001": 0.05,
406
+ "T1569.002": 0.05,
407
+ "T1570": 0.05,
408
+ "T1573.001": 0.05,
409
+ "T1574.002": 0.05
410
+ },
411
+ "micro_f1": 0.0,
412
+ "macro_f1": 0.0,
413
+ "micro_precision": 0.0,
414
+ "micro_recall": 0.0,
415
+ "macro_precision": 0.0,
416
+ "macro_recall": 0.0,
417
+ "n_examples": 2767,
418
+ "n_positive_labels": 737,
419
+ "per_class": {
420
+ "T1003.001": {
421
+ "precision": 0.0,
422
+ "recall": 0.0,
423
+ "f1": 0.0,
424
+ "support": 17,
425
+ "predicted": 0
426
+ },
427
+ "T1005": {
428
+ "precision": 0.0,
429
+ "recall": 0.0,
430
+ "f1": 0.0,
431
+ "support": 12,
432
+ "predicted": 0
433
+ },
434
+ "T1012": {
435
+ "precision": 0.0,
436
+ "recall": 0.0,
437
+ "f1": 0.0,
438
+ "support": 4,
439
+ "predicted": 0
440
+ },
441
+ "T1016": {
442
+ "precision": 0.0,
443
+ "recall": 0.0,
444
+ "f1": 0.0,
445
+ "support": 9,
446
+ "predicted": 0
447
+ },
448
+ "T1021.001": {
449
+ "precision": 0.0,
450
+ "recall": 0.0,
451
+ "f1": 0.0,
452
+ "support": 16,
453
+ "predicted": 0
454
+ },
455
+ "T1027": {
456
+ "precision": 0.0,
457
+ "recall": 0.0,
458
+ "f1": 0.0,
459
+ "support": 96,
460
+ "predicted": 0
461
+ },
462
+ "T1033": {
463
+ "precision": 0.0,
464
+ "recall": 0.0,
465
+ "f1": 0.0,
466
+ "support": 6,
467
+ "predicted": 0
468
+ },
469
+ "T1036.005": {
470
+ "precision": 0.0,
471
+ "recall": 0.0,
472
+ "f1": 0.0,
473
+ "support": 8,
474
+ "predicted": 0
475
+ },
476
+ "T1041": {
477
+ "precision": 0.0,
478
+ "recall": 0.0,
479
+ "f1": 0.0,
480
+ "support": 13,
481
+ "predicted": 0
482
+ },
483
+ "T1047": {
484
+ "precision": 0.0,
485
+ "recall": 0.0,
486
+ "f1": 0.0,
487
+ "support": 10,
488
+ "predicted": 0
489
+ },
490
+ "T1053.005": {
491
+ "precision": 0.0,
492
+ "recall": 0.0,
493
+ "f1": 0.0,
494
+ "support": 14,
495
+ "predicted": 0
496
+ },
497
+ "T1055": {
498
+ "precision": 0.0,
499
+ "recall": 0.0,
500
+ "f1": 0.0,
501
+ "support": 43,
502
+ "predicted": 0
503
+ },
504
+ "T1056.001": {
505
+ "precision": 0.0,
506
+ "recall": 0.0,
507
+ "f1": 0.0,
508
+ "support": 6,
509
+ "predicted": 0
510
+ },
511
+ "T1057": {
512
+ "precision": 0.0,
513
+ "recall": 0.0,
514
+ "f1": 0.0,
515
+ "support": 12,
516
+ "predicted": 0
517
+ },
518
+ "T1059.003": {
519
+ "precision": 0.0,
520
+ "recall": 0.0,
521
+ "f1": 0.0,
522
+ "support": 45,
523
+ "predicted": 0
524
+ },
525
+ "T1068": {
526
+ "precision": 0.0,
527
+ "recall": 0.0,
528
+ "f1": 0.0,
529
+ "support": 1,
530
+ "predicted": 0
531
+ },
532
+ "T1070.004": {
533
+ "precision": 0.0,
534
+ "recall": 0.0,
535
+ "f1": 0.0,
536
+ "support": 15,
537
+ "predicted": 0
538
+ },
539
+ "T1071.001": {
540
+ "precision": 0.0,
541
+ "recall": 0.0,
542
+ "f1": 0.0,
543
+ "support": 28,
544
+ "predicted": 0
545
+ },
546
+ "T1072": {
547
+ "precision": 0.0,
548
+ "recall": 0.0,
549
+ "f1": 0.0,
550
+ "support": 2,
551
+ "predicted": 0
552
+ },
553
+ "T1074.001": {
554
+ "precision": 0.0,
555
+ "recall": 0.0,
556
+ "f1": 0.0,
557
+ "support": 3,
558
+ "predicted": 0
559
+ },
560
+ "T1078": {
561
+ "precision": 0.0,
562
+ "recall": 0.0,
563
+ "f1": 0.0,
564
+ "support": 21,
565
+ "predicted": 0
566
+ },
567
+ "T1082": {
568
+ "precision": 0.0,
569
+ "recall": 0.0,
570
+ "f1": 0.0,
571
+ "support": 17,
572
+ "predicted": 0
573
+ },
574
+ "T1083": {
575
+ "precision": 0.0,
576
+ "recall": 0.0,
577
+ "f1": 0.0,
578
+ "support": 13,
579
+ "predicted": 0
580
+ },
581
+ "T1090": {
582
+ "precision": 0.0,
583
+ "recall": 0.0,
584
+ "f1": 0.0,
585
+ "support": 18,
586
+ "predicted": 0
587
+ },
588
+ "T1095": {
589
+ "precision": 0.0,
590
+ "recall": 0.0,
591
+ "f1": 0.0,
592
+ "support": 12,
593
+ "predicted": 0
594
+ },
595
+ "T1105": {
596
+ "precision": 0.0,
597
+ "recall": 0.0,
598
+ "f1": 0.0,
599
+ "support": 40,
600
+ "predicted": 0
601
+ },
602
+ "T1106": {
603
+ "precision": 0.0,
604
+ "recall": 0.0,
605
+ "f1": 0.0,
606
+ "support": 21,
607
+ "predicted": 0
608
+ },
609
+ "T1110": {
610
+ "precision": 0.0,
611
+ "recall": 0.0,
612
+ "f1": 0.0,
613
+ "support": 3,
614
+ "predicted": 0
615
+ },
616
+ "T1112": {
617
+ "precision": 0.0,
618
+ "recall": 0.0,
619
+ "f1": 0.0,
620
+ "support": 15,
621
+ "predicted": 0
622
+ },
623
+ "T1113": {
624
+ "precision": 0.0,
625
+ "recall": 0.0,
626
+ "f1": 0.0,
627
+ "support": 8,
628
+ "predicted": 0
629
+ },
630
+ "T1140": {
631
+ "precision": 0.0,
632
+ "recall": 0.0,
633
+ "f1": 0.0,
634
+ "support": 67,
635
+ "predicted": 0
636
+ },
637
+ "T1190": {
638
+ "precision": 0.0,
639
+ "recall": 0.0,
640
+ "f1": 0.0,
641
+ "support": 13,
642
+ "predicted": 0
643
+ },
644
+ "T1204.002": {
645
+ "precision": 0.0,
646
+ "recall": 0.0,
647
+ "f1": 0.0,
648
+ "support": 10,
649
+ "predicted": 0
650
+ },
651
+ "T1210": {
652
+ "precision": 0.0,
653
+ "recall": 0.0,
654
+ "f1": 0.0,
655
+ "support": 2,
656
+ "predicted": 0
657
+ },
658
+ "T1218.011": {
659
+ "precision": 0.0,
660
+ "recall": 0.0,
661
+ "f1": 0.0,
662
+ "support": 11,
663
+ "predicted": 0
664
+ },
665
+ "T1219": {
666
+ "precision": 0.0,
667
+ "recall": 0.0,
668
+ "f1": 0.0,
669
+ "support": 4,
670
+ "predicted": 0
671
+ },
672
+ "T1484.001": {
673
+ "precision": 0.0,
674
+ "recall": 0.0,
675
+ "f1": 0.0,
676
+ "support": 4,
677
+ "predicted": 0
678
+ },
679
+ "T1518.001": {
680
+ "precision": 0.0,
681
+ "recall": 0.0,
682
+ "f1": 0.0,
683
+ "support": 4,
684
+ "predicted": 0
685
+ },
686
+ "T1543.003": {
687
+ "precision": 0.0,
688
+ "recall": 0.0,
689
+ "f1": 0.0,
690
+ "support": 4,
691
+ "predicted": 0
692
+ },
693
+ "T1547.001": {
694
+ "precision": 0.0,
695
+ "recall": 0.0,
696
+ "f1": 0.0,
697
+ "support": 10,
698
+ "predicted": 0
699
+ },
700
+ "T1548.002": {
701
+ "precision": 0.0,
702
+ "recall": 0.0,
703
+ "f1": 0.0,
704
+ "support": 6,
705
+ "predicted": 0
706
+ },
707
+ "T1552.001": {
708
+ "precision": 0.0,
709
+ "recall": 0.0,
710
+ "f1": 0.0,
711
+ "support": 3,
712
+ "predicted": 0
713
+ },
714
+ "T1562.001": {
715
+ "precision": 0.0,
716
+ "recall": 0.0,
717
+ "f1": 0.0,
718
+ "support": 13,
719
+ "predicted": 0
720
+ },
721
+ "T1564.001": {
722
+ "precision": 0.0,
723
+ "recall": 0.0,
724
+ "f1": 0.0,
725
+ "support": 1,
726
+ "predicted": 0
727
+ },
728
+ "T1566.001": {
729
+ "precision": 0.0,
730
+ "recall": 0.0,
731
+ "f1": 0.0,
732
+ "support": 15,
733
+ "predicted": 0
734
+ },
735
+ "T1569.002": {
736
+ "precision": 0.0,
737
+ "recall": 0.0,
738
+ "f1": 0.0,
739
+ "support": 2,
740
+ "predicted": 0
741
+ },
742
+ "T1570": {
743
+ "precision": 0.0,
744
+ "recall": 0.0,
745
+ "f1": 0.0,
746
+ "support": 13,
747
+ "predicted": 0
748
+ },
749
+ "T1573.001": {
750
+ "precision": 0.0,
751
+ "recall": 0.0,
752
+ "f1": 0.0,
753
+ "support": 9,
754
+ "predicted": 0
755
+ },
756
+ "T1574.002": {
757
+ "precision": 0.0,
758
+ "recall": 0.0,
759
+ "f1": 0.0,
760
+ "support": 18,
761
+ "predicted": 0
762
+ }
763
+ }
764
+ }
765
+ },
766
+ "keyword": {
767
+ "global_threshold": {
768
+ "threshold": 0.05,
769
+ "micro_f1": 0.1535,
770
+ "macro_f1": 0.1709,
771
+ "micro_precision": 0.2895,
772
+ "micro_recall": 0.1045,
773
+ "macro_precision": 0.3017,
774
+ "macro_recall": 0.1484,
775
+ "n_examples": 2767,
776
+ "n_positive_labels": 737,
777
+ "per_class": {
778
+ "T1003.001": {
779
+ "precision": 0.4286,
780
+ "recall": 0.1765,
781
+ "f1": 0.25,
782
+ "support": 17,
783
+ "predicted": 7
784
+ },
785
+ "T1005": {
786
+ "precision": 0.5,
787
+ "recall": 0.0833,
788
+ "f1": 0.1429,
789
+ "support": 12,
790
+ "predicted": 2
791
+ },
792
+ "T1012": {
793
+ "precision": 0.0,
794
+ "recall": 0.0,
795
+ "f1": 0.0,
796
+ "support": 4,
797
+ "predicted": 4
798
+ },
799
+ "T1016": {
800
+ "precision": 0.0,
801
+ "recall": 0.0,
802
+ "f1": 0.0,
803
+ "support": 9,
804
+ "predicted": 2
805
+ },
806
+ "T1021.001": {
807
+ "precision": 0.25,
808
+ "recall": 0.1875,
809
+ "f1": 0.2143,
810
+ "support": 16,
811
+ "predicted": 12
812
+ },
813
+ "T1027": {
814
+ "precision": 0.2857,
815
+ "recall": 0.0208,
816
+ "f1": 0.0388,
817
+ "support": 96,
818
+ "predicted": 7
819
+ },
820
+ "T1033": {
821
+ "precision": 0.25,
822
+ "recall": 0.1667,
823
+ "f1": 0.2,
824
+ "support": 6,
825
+ "predicted": 4
826
+ },
827
+ "T1036.005": {
828
+ "precision": 0.5,
829
+ "recall": 0.25,
830
+ "f1": 0.3333,
831
+ "support": 8,
832
+ "predicted": 4
833
+ },
834
+ "T1041": {
835
+ "precision": 0.0,
836
+ "recall": 0.0,
837
+ "f1": 0.0,
838
+ "support": 13,
839
+ "predicted": 1
840
+ },
841
+ "T1047": {
842
+ "precision": 0.25,
843
+ "recall": 0.1,
844
+ "f1": 0.1429,
845
+ "support": 10,
846
+ "predicted": 4
847
+ },
848
+ "T1053.005": {
849
+ "precision": 0.6,
850
+ "recall": 0.6429,
851
+ "f1": 0.6207,
852
+ "support": 14,
853
+ "predicted": 15
854
+ },
855
+ "T1055": {
856
+ "precision": 0.5,
857
+ "recall": 0.0698,
858
+ "f1": 0.1224,
859
+ "support": 43,
860
+ "predicted": 6
861
+ },
862
+ "T1056.001": {
863
+ "precision": 0.5,
864
+ "recall": 0.1667,
865
+ "f1": 0.25,
866
+ "support": 6,
867
+ "predicted": 2
868
+ },
869
+ "T1057": {
870
+ "precision": 0.0,
871
+ "recall": 0.0,
872
+ "f1": 0.0,
873
+ "support": 12,
874
+ "predicted": 4
875
+ },
876
+ "T1059.003": {
877
+ "precision": 0.0909,
878
+ "recall": 0.0222,
879
+ "f1": 0.0357,
880
+ "support": 45,
881
+ "predicted": 11
882
+ },
883
+ "T1068": {
884
+ "precision": 0.0,
885
+ "recall": 0.0,
886
+ "f1": 0.0,
887
+ "support": 1,
888
+ "predicted": 1
889
+ },
890
+ "T1070.004": {
891
+ "precision": 0.2,
892
+ "recall": 0.1333,
893
+ "f1": 0.16,
894
+ "support": 15,
895
+ "predicted": 10
896
+ },
897
+ "T1071.001": {
898
+ "precision": 0.1429,
899
+ "recall": 0.0357,
900
+ "f1": 0.0571,
901
+ "support": 28,
902
+ "predicted": 7
903
+ },
904
+ "T1072": {
905
+ "precision": 0.5,
906
+ "recall": 0.5,
907
+ "f1": 0.5,
908
+ "support": 2,
909
+ "predicted": 2
910
+ },
911
+ "T1074.001": {
912
+ "precision": 1.0,
913
+ "recall": 0.3333,
914
+ "f1": 0.5,
915
+ "support": 3,
916
+ "predicted": 1
917
+ },
918
+ "T1078": {
919
+ "precision": 0.5,
920
+ "recall": 0.1429,
921
+ "f1": 0.2222,
922
+ "support": 21,
923
+ "predicted": 6
924
+ },
925
+ "T1082": {
926
+ "precision": 0.0,
927
+ "recall": 0.0,
928
+ "f1": 0.0,
929
+ "support": 17,
930
+ "predicted": 5
931
+ },
932
+ "T1083": {
933
+ "precision": 0.0,
934
+ "recall": 0.0,
935
+ "f1": 0.0,
936
+ "support": 13,
937
+ "predicted": 5
938
+ },
939
+ "T1090": {
940
+ "precision": 0.2632,
941
+ "recall": 0.2778,
942
+ "f1": 0.2703,
943
+ "support": 18,
944
+ "predicted": 19
945
+ },
946
+ "T1095": {
947
+ "precision": 0.75,
948
+ "recall": 0.25,
949
+ "f1": 0.375,
950
+ "support": 12,
951
+ "predicted": 4
952
+ },
953
+ "T1105": {
954
+ "precision": 0.0,
955
+ "recall": 0.0,
956
+ "f1": 0.0,
957
+ "support": 40,
958
+ "predicted": 1
959
+ },
960
+ "T1106": {
961
+ "precision": 0.5,
962
+ "recall": 0.0476,
963
+ "f1": 0.087,
964
+ "support": 21,
965
+ "predicted": 2
966
+ },
967
+ "T1110": {
968
+ "precision": 0.25,
969
+ "recall": 0.3333,
970
+ "f1": 0.2857,
971
+ "support": 3,
972
+ "predicted": 4
973
+ },
974
+ "T1112": {
975
+ "precision": 0.1429,
976
+ "recall": 0.0667,
977
+ "f1": 0.0909,
978
+ "support": 15,
979
+ "predicted": 7
980
+ },
981
+ "T1113": {
982
+ "precision": 1.0,
983
+ "recall": 0.125,
984
+ "f1": 0.2222,
985
+ "support": 8,
986
+ "predicted": 1
987
+ },
988
+ "T1140": {
989
+ "precision": 0.2,
990
+ "recall": 0.0149,
991
+ "f1": 0.0278,
992
+ "support": 67,
993
+ "predicted": 5
994
+ },
995
+ "T1190": {
996
+ "precision": 0.6667,
997
+ "recall": 0.1538,
998
+ "f1": 0.25,
999
+ "support": 13,
1000
+ "predicted": 3
1001
+ },
1002
+ "T1204.002": {
1003
+ "precision": 0.1429,
1004
+ "recall": 0.2,
1005
+ "f1": 0.1667,
1006
+ "support": 10,
1007
+ "predicted": 14
1008
+ },
1009
+ "T1210": {
1010
+ "precision": 0.0,
1011
+ "recall": 0.0,
1012
+ "f1": 0.0,
1013
+ "support": 2,
1014
+ "predicted": 1
1015
+ },
1016
+ "T1218.011": {
1017
+ "precision": 0.5,
1018
+ "recall": 0.6364,
1019
+ "f1": 0.56,
1020
+ "support": 11,
1021
+ "predicted": 14
1022
+ },
1023
+ "T1219": {
1024
+ "precision": 1.0,
1025
+ "recall": 0.25,
1026
+ "f1": 0.4,
1027
+ "support": 4,
1028
+ "predicted": 1
1029
+ },
1030
+ "T1484.001": {
1031
+ "precision": 1.0,
1032
+ "recall": 0.25,
1033
+ "f1": 0.4,
1034
+ "support": 4,
1035
+ "predicted": 1
1036
+ },
1037
+ "T1518.001": {
1038
+ "precision": 0.0,
1039
+ "recall": 0.0,
1040
+ "f1": 0.0,
1041
+ "support": 4,
1042
+ "predicted": 0
1043
+ },
1044
+ "T1543.003": {
1045
+ "precision": 0.5,
1046
+ "recall": 0.5,
1047
+ "f1": 0.5,
1048
+ "support": 4,
1049
+ "predicted": 4
1050
+ },
1051
+ "T1547.001": {
1052
+ "precision": 0.0,
1053
+ "recall": 0.0,
1054
+ "f1": 0.0,
1055
+ "support": 10,
1056
+ "predicted": 2
1057
+ },
1058
+ "T1548.002": {
1059
+ "precision": 0.25,
1060
+ "recall": 0.1667,
1061
+ "f1": 0.2,
1062
+ "support": 6,
1063
+ "predicted": 4
1064
+ },
1065
+ "T1552.001": {
1066
+ "precision": 0.0,
1067
+ "recall": 0.0,
1068
+ "f1": 0.0,
1069
+ "support": 3,
1070
+ "predicted": 0
1071
+ },
1072
+ "T1562.001": {
1073
+ "precision": 0.0,
1074
+ "recall": 0.0,
1075
+ "f1": 0.0,
1076
+ "support": 13,
1077
+ "predicted": 0
1078
+ },
1079
+ "T1564.001": {
1080
+ "precision": 0.0,
1081
+ "recall": 0.0,
1082
+ "f1": 0.0,
1083
+ "support": 1,
1084
+ "predicted": 1
1085
+ },
1086
+ "T1566.001": {
1087
+ "precision": 0.2174,
1088
+ "recall": 0.6667,
1089
+ "f1": 0.3279,
1090
+ "support": 15,
1091
+ "predicted": 46
1092
+ },
1093
+ "T1569.002": {
1094
+ "precision": 0.0,
1095
+ "recall": 0.0,
1096
+ "f1": 0.0,
1097
+ "support": 2,
1098
+ "predicted": 3
1099
+ },
1100
+ "T1570": {
1101
+ "precision": 0.5,
1102
+ "recall": 0.0769,
1103
+ "f1": 0.1333,
1104
+ "support": 13,
1105
+ "predicted": 2
1106
+ },
1107
+ "T1573.001": {
1108
+ "precision": 0.4,
1109
+ "recall": 0.2222,
1110
+ "f1": 0.2857,
1111
+ "support": 9,
1112
+ "predicted": 5
1113
+ },
1114
+ "T1574.002": {
1115
+ "precision": 0.0,
1116
+ "recall": 0.0,
1117
+ "f1": 0.0,
1118
+ "support": 18,
1119
+ "predicted": 0
1120
+ }
1121
+ }
1122
+ },
1123
+ "per_class_threshold": {
1124
+ "thresholds": {
1125
+ "T1003.001": 0.05,
1126
+ "T1005": 0.05,
1127
+ "T1012": 0.05,
1128
+ "T1016": 0.05,
1129
+ "T1021.001": 0.05,
1130
+ "T1027": 0.05,
1131
+ "T1033": 0.05,
1132
+ "T1036.005": 0.05,
1133
+ "T1041": 0.05,
1134
+ "T1047": 0.05,
1135
+ "T1053.005": 0.05,
1136
+ "T1055": 0.05,
1137
+ "T1056.001": 0.05,
1138
+ "T1057": 0.05,
1139
+ "T1059.003": 0.05,
1140
+ "T1068": 0.05,
1141
+ "T1070.004": 0.05,
1142
+ "T1071.001": 0.05,
1143
+ "T1072": 0.05,
1144
+ "T1074.001": 0.05,
1145
+ "T1078": 0.05,
1146
+ "T1082": 0.05,
1147
+ "T1083": 0.05,
1148
+ "T1090": 0.05,
1149
+ "T1095": 0.05,
1150
+ "T1105": 0.05,
1151
+ "T1106": 0.05,
1152
+ "T1110": 0.05,
1153
+ "T1112": 0.05,
1154
+ "T1113": 0.05,
1155
+ "T1140": 0.05,
1156
+ "T1190": 0.05,
1157
+ "T1204.002": 0.05,
1158
+ "T1210": 0.05,
1159
+ "T1218.011": 0.05,
1160
+ "T1219": 0.05,
1161
+ "T1484.001": 0.05,
1162
+ "T1518.001": 0.05,
1163
+ "T1543.003": 0.05,
1164
+ "T1547.001": 0.05,
1165
+ "T1548.002": 0.05,
1166
+ "T1552.001": 0.05,
1167
+ "T1562.001": 0.05,
1168
+ "T1564.001": 0.05,
1169
+ "T1566.001": 0.05,
1170
+ "T1569.002": 0.05,
1171
+ "T1570": 0.05,
1172
+ "T1573.001": 0.05,
1173
+ "T1574.002": 0.05
1174
+ },
1175
+ "micro_f1": 0.1535,
1176
+ "macro_f1": 0.1709,
1177
+ "micro_precision": 0.2895,
1178
+ "micro_recall": 0.1045,
1179
+ "macro_precision": 0.3017,
1180
+ "macro_recall": 0.1484,
1181
+ "n_examples": 2767,
1182
+ "n_positive_labels": 737,
1183
+ "per_class": {
1184
+ "T1003.001": {
1185
+ "precision": 0.4286,
1186
+ "recall": 0.1765,
1187
+ "f1": 0.25,
1188
+ "support": 17,
1189
+ "predicted": 7
1190
+ },
1191
+ "T1005": {
1192
+ "precision": 0.5,
1193
+ "recall": 0.0833,
1194
+ "f1": 0.1429,
1195
+ "support": 12,
1196
+ "predicted": 2
1197
+ },
1198
+ "T1012": {
1199
+ "precision": 0.0,
1200
+ "recall": 0.0,
1201
+ "f1": 0.0,
1202
+ "support": 4,
1203
+ "predicted": 4
1204
+ },
1205
+ "T1016": {
1206
+ "precision": 0.0,
1207
+ "recall": 0.0,
1208
+ "f1": 0.0,
1209
+ "support": 9,
1210
+ "predicted": 2
1211
+ },
1212
+ "T1021.001": {
1213
+ "precision": 0.25,
1214
+ "recall": 0.1875,
1215
+ "f1": 0.2143,
1216
+ "support": 16,
1217
+ "predicted": 12
1218
+ },
1219
+ "T1027": {
1220
+ "precision": 0.2857,
1221
+ "recall": 0.0208,
1222
+ "f1": 0.0388,
1223
+ "support": 96,
1224
+ "predicted": 7
1225
+ },
1226
+ "T1033": {
1227
+ "precision": 0.25,
1228
+ "recall": 0.1667,
1229
+ "f1": 0.2,
1230
+ "support": 6,
1231
+ "predicted": 4
1232
+ },
1233
+ "T1036.005": {
1234
+ "precision": 0.5,
1235
+ "recall": 0.25,
1236
+ "f1": 0.3333,
1237
+ "support": 8,
1238
+ "predicted": 4
1239
+ },
1240
+ "T1041": {
1241
+ "precision": 0.0,
1242
+ "recall": 0.0,
1243
+ "f1": 0.0,
1244
+ "support": 13,
1245
+ "predicted": 1
1246
+ },
1247
+ "T1047": {
1248
+ "precision": 0.25,
1249
+ "recall": 0.1,
1250
+ "f1": 0.1429,
1251
+ "support": 10,
1252
+ "predicted": 4
1253
+ },
1254
+ "T1053.005": {
1255
+ "precision": 0.6,
1256
+ "recall": 0.6429,
1257
+ "f1": 0.6207,
1258
+ "support": 14,
1259
+ "predicted": 15
1260
+ },
1261
+ "T1055": {
1262
+ "precision": 0.5,
1263
+ "recall": 0.0698,
1264
+ "f1": 0.1224,
1265
+ "support": 43,
1266
+ "predicted": 6
1267
+ },
1268
+ "T1056.001": {
1269
+ "precision": 0.5,
1270
+ "recall": 0.1667,
1271
+ "f1": 0.25,
1272
+ "support": 6,
1273
+ "predicted": 2
1274
+ },
1275
+ "T1057": {
1276
+ "precision": 0.0,
1277
+ "recall": 0.0,
1278
+ "f1": 0.0,
1279
+ "support": 12,
1280
+ "predicted": 4
1281
+ },
1282
+ "T1059.003": {
1283
+ "precision": 0.0909,
1284
+ "recall": 0.0222,
1285
+ "f1": 0.0357,
1286
+ "support": 45,
1287
+ "predicted": 11
1288
+ },
1289
+ "T1068": {
1290
+ "precision": 0.0,
1291
+ "recall": 0.0,
1292
+ "f1": 0.0,
1293
+ "support": 1,
1294
+ "predicted": 1
1295
+ },
1296
+ "T1070.004": {
1297
+ "precision": 0.2,
1298
+ "recall": 0.1333,
1299
+ "f1": 0.16,
1300
+ "support": 15,
1301
+ "predicted": 10
1302
+ },
1303
+ "T1071.001": {
1304
+ "precision": 0.1429,
1305
+ "recall": 0.0357,
1306
+ "f1": 0.0571,
1307
+ "support": 28,
1308
+ "predicted": 7
1309
+ },
1310
+ "T1072": {
1311
+ "precision": 0.5,
1312
+ "recall": 0.5,
1313
+ "f1": 0.5,
1314
+ "support": 2,
1315
+ "predicted": 2
1316
+ },
1317
+ "T1074.001": {
1318
+ "precision": 1.0,
1319
+ "recall": 0.3333,
1320
+ "f1": 0.5,
1321
+ "support": 3,
1322
+ "predicted": 1
1323
+ },
1324
+ "T1078": {
1325
+ "precision": 0.5,
1326
+ "recall": 0.1429,
1327
+ "f1": 0.2222,
1328
+ "support": 21,
1329
+ "predicted": 6
1330
+ },
1331
+ "T1082": {
1332
+ "precision": 0.0,
1333
+ "recall": 0.0,
1334
+ "f1": 0.0,
1335
+ "support": 17,
1336
+ "predicted": 5
1337
+ },
1338
+ "T1083": {
1339
+ "precision": 0.0,
1340
+ "recall": 0.0,
1341
+ "f1": 0.0,
1342
+ "support": 13,
1343
+ "predicted": 5
1344
+ },
1345
+ "T1090": {
1346
+ "precision": 0.2632,
1347
+ "recall": 0.2778,
1348
+ "f1": 0.2703,
1349
+ "support": 18,
1350
+ "predicted": 19
1351
+ },
1352
+ "T1095": {
1353
+ "precision": 0.75,
1354
+ "recall": 0.25,
1355
+ "f1": 0.375,
1356
+ "support": 12,
1357
+ "predicted": 4
1358
+ },
1359
+ "T1105": {
1360
+ "precision": 0.0,
1361
+ "recall": 0.0,
1362
+ "f1": 0.0,
1363
+ "support": 40,
1364
+ "predicted": 1
1365
+ },
1366
+ "T1106": {
1367
+ "precision": 0.5,
1368
+ "recall": 0.0476,
1369
+ "f1": 0.087,
1370
+ "support": 21,
1371
+ "predicted": 2
1372
+ },
1373
+ "T1110": {
1374
+ "precision": 0.25,
1375
+ "recall": 0.3333,
1376
+ "f1": 0.2857,
1377
+ "support": 3,
1378
+ "predicted": 4
1379
+ },
1380
+ "T1112": {
1381
+ "precision": 0.1429,
1382
+ "recall": 0.0667,
1383
+ "f1": 0.0909,
1384
+ "support": 15,
1385
+ "predicted": 7
1386
+ },
1387
+ "T1113": {
1388
+ "precision": 1.0,
1389
+ "recall": 0.125,
1390
+ "f1": 0.2222,
1391
+ "support": 8,
1392
+ "predicted": 1
1393
+ },
1394
+ "T1140": {
1395
+ "precision": 0.2,
1396
+ "recall": 0.0149,
1397
+ "f1": 0.0278,
1398
+ "support": 67,
1399
+ "predicted": 5
1400
+ },
1401
+ "T1190": {
1402
+ "precision": 0.6667,
1403
+ "recall": 0.1538,
1404
+ "f1": 0.25,
1405
+ "support": 13,
1406
+ "predicted": 3
1407
+ },
1408
+ "T1204.002": {
1409
+ "precision": 0.1429,
1410
+ "recall": 0.2,
1411
+ "f1": 0.1667,
1412
+ "support": 10,
1413
+ "predicted": 14
1414
+ },
1415
+ "T1210": {
1416
+ "precision": 0.0,
1417
+ "recall": 0.0,
1418
+ "f1": 0.0,
1419
+ "support": 2,
1420
+ "predicted": 1
1421
+ },
1422
+ "T1218.011": {
1423
+ "precision": 0.5,
1424
+ "recall": 0.6364,
1425
+ "f1": 0.56,
1426
+ "support": 11,
1427
+ "predicted": 14
1428
+ },
1429
+ "T1219": {
1430
+ "precision": 1.0,
1431
+ "recall": 0.25,
1432
+ "f1": 0.4,
1433
+ "support": 4,
1434
+ "predicted": 1
1435
+ },
1436
+ "T1484.001": {
1437
+ "precision": 1.0,
1438
+ "recall": 0.25,
1439
+ "f1": 0.4,
1440
+ "support": 4,
1441
+ "predicted": 1
1442
+ },
1443
+ "T1518.001": {
1444
+ "precision": 0.0,
1445
+ "recall": 0.0,
1446
+ "f1": 0.0,
1447
+ "support": 4,
1448
+ "predicted": 0
1449
+ },
1450
+ "T1543.003": {
1451
+ "precision": 0.5,
1452
+ "recall": 0.5,
1453
+ "f1": 0.5,
1454
+ "support": 4,
1455
+ "predicted": 4
1456
+ },
1457
+ "T1547.001": {
1458
+ "precision": 0.0,
1459
+ "recall": 0.0,
1460
+ "f1": 0.0,
1461
+ "support": 10,
1462
+ "predicted": 2
1463
+ },
1464
+ "T1548.002": {
1465
+ "precision": 0.25,
1466
+ "recall": 0.1667,
1467
+ "f1": 0.2,
1468
+ "support": 6,
1469
+ "predicted": 4
1470
+ },
1471
+ "T1552.001": {
1472
+ "precision": 0.0,
1473
+ "recall": 0.0,
1474
+ "f1": 0.0,
1475
+ "support": 3,
1476
+ "predicted": 0
1477
+ },
1478
+ "T1562.001": {
1479
+ "precision": 0.0,
1480
+ "recall": 0.0,
1481
+ "f1": 0.0,
1482
+ "support": 13,
1483
+ "predicted": 0
1484
+ },
1485
+ "T1564.001": {
1486
+ "precision": 0.0,
1487
+ "recall": 0.0,
1488
+ "f1": 0.0,
1489
+ "support": 1,
1490
+ "predicted": 1
1491
+ },
1492
+ "T1566.001": {
1493
+ "precision": 0.2174,
1494
+ "recall": 0.6667,
1495
+ "f1": 0.3279,
1496
+ "support": 15,
1497
+ "predicted": 46
1498
+ },
1499
+ "T1569.002": {
1500
+ "precision": 0.0,
1501
+ "recall": 0.0,
1502
+ "f1": 0.0,
1503
+ "support": 2,
1504
+ "predicted": 3
1505
+ },
1506
+ "T1570": {
1507
+ "precision": 0.5,
1508
+ "recall": 0.0769,
1509
+ "f1": 0.1333,
1510
+ "support": 13,
1511
+ "predicted": 2
1512
+ },
1513
+ "T1573.001": {
1514
+ "precision": 0.4,
1515
+ "recall": 0.2222,
1516
+ "f1": 0.2857,
1517
+ "support": 9,
1518
+ "predicted": 5
1519
+ },
1520
+ "T1574.002": {
1521
+ "precision": 0.0,
1522
+ "recall": 0.0,
1523
+ "f1": 0.0,
1524
+ "support": 18,
1525
+ "predicted": 0
1526
+ }
1527
+ }
1528
+ }
1529
+ },
1530
+ "tfidf_lr": {
1531
+ "global_threshold": {
1532
+ "threshold": 0.35,
1533
+ "micro_f1": 0.5518,
1534
+ "macro_f1": 0.5149,
1535
+ "micro_precision": 0.4683,
1536
+ "micro_recall": 0.6716,
1537
+ "macro_precision": 0.4666,
1538
+ "macro_recall": 0.6088,
1539
+ "n_examples": 2767,
1540
+ "n_positive_labels": 737,
1541
+ "per_class": {
1542
+ "T1003.001": {
1543
+ "precision": 0.6,
1544
+ "recall": 0.8824,
1545
+ "f1": 0.7143,
1546
+ "support": 17,
1547
+ "predicted": 25
1548
+ },
1549
+ "T1005": {
1550
+ "precision": 0.2,
1551
+ "recall": 0.1667,
1552
+ "f1": 0.1818,
1553
+ "support": 12,
1554
+ "predicted": 10
1555
+ },
1556
+ "T1012": {
1557
+ "precision": 0.4,
1558
+ "recall": 0.5,
1559
+ "f1": 0.4444,
1560
+ "support": 4,
1561
+ "predicted": 5
1562
+ },
1563
+ "T1016": {
1564
+ "precision": 0.4286,
1565
+ "recall": 0.3333,
1566
+ "f1": 0.375,
1567
+ "support": 9,
1568
+ "predicted": 7
1569
+ },
1570
+ "T1021.001": {
1571
+ "precision": 0.6,
1572
+ "recall": 0.75,
1573
+ "f1": 0.6667,
1574
+ "support": 16,
1575
+ "predicted": 20
1576
+ },
1577
+ "T1027": {
1578
+ "precision": 0.3676,
1579
+ "recall": 0.7083,
1580
+ "f1": 0.484,
1581
+ "support": 96,
1582
+ "predicted": 185
1583
+ },
1584
+ "T1033": {
1585
+ "precision": 0.2,
1586
+ "recall": 0.1667,
1587
+ "f1": 0.1818,
1588
+ "support": 6,
1589
+ "predicted": 5
1590
+ },
1591
+ "T1036.005": {
1592
+ "precision": 0.4545,
1593
+ "recall": 0.625,
1594
+ "f1": 0.5263,
1595
+ "support": 8,
1596
+ "predicted": 11
1597
+ },
1598
+ "T1041": {
1599
+ "precision": 0.2727,
1600
+ "recall": 0.4615,
1601
+ "f1": 0.3429,
1602
+ "support": 13,
1603
+ "predicted": 22
1604
+ },
1605
+ "T1047": {
1606
+ "precision": 0.5455,
1607
+ "recall": 0.6,
1608
+ "f1": 0.5714,
1609
+ "support": 10,
1610
+ "predicted": 11
1611
+ },
1612
+ "T1053.005": {
1613
+ "precision": 0.6842,
1614
+ "recall": 0.9286,
1615
+ "f1": 0.7879,
1616
+ "support": 14,
1617
+ "predicted": 19
1618
+ },
1619
+ "T1055": {
1620
+ "precision": 0.5741,
1621
+ "recall": 0.7209,
1622
+ "f1": 0.6392,
1623
+ "support": 43,
1624
+ "predicted": 54
1625
+ },
1626
+ "T1056.001": {
1627
+ "precision": 0.8571,
1628
+ "recall": 1.0,
1629
+ "f1": 0.9231,
1630
+ "support": 6,
1631
+ "predicted": 7
1632
+ },
1633
+ "T1057": {
1634
+ "precision": 0.375,
1635
+ "recall": 0.5,
1636
+ "f1": 0.4286,
1637
+ "support": 12,
1638
+ "predicted": 16
1639
+ },
1640
+ "T1059.003": {
1641
+ "precision": 0.4627,
1642
+ "recall": 0.6889,
1643
+ "f1": 0.5536,
1644
+ "support": 45,
1645
+ "predicted": 67
1646
+ },
1647
+ "T1068": {
1648
+ "precision": 0.0,
1649
+ "recall": 0.0,
1650
+ "f1": 0.0,
1651
+ "support": 1,
1652
+ "predicted": 0
1653
+ },
1654
+ "T1070.004": {
1655
+ "precision": 0.4231,
1656
+ "recall": 0.7333,
1657
+ "f1": 0.5366,
1658
+ "support": 15,
1659
+ "predicted": 26
1660
+ },
1661
+ "T1071.001": {
1662
+ "precision": 0.4688,
1663
+ "recall": 0.5357,
1664
+ "f1": 0.5,
1665
+ "support": 28,
1666
+ "predicted": 32
1667
+ },
1668
+ "T1072": {
1669
+ "precision": 0.0,
1670
+ "recall": 0.0,
1671
+ "f1": 0.0,
1672
+ "support": 2,
1673
+ "predicted": 2
1674
+ },
1675
+ "T1074.001": {
1676
+ "precision": 0.2,
1677
+ "recall": 0.3333,
1678
+ "f1": 0.25,
1679
+ "support": 3,
1680
+ "predicted": 5
1681
+ },
1682
+ "T1078": {
1683
+ "precision": 0.325,
1684
+ "recall": 0.619,
1685
+ "f1": 0.4262,
1686
+ "support": 21,
1687
+ "predicted": 40
1688
+ },
1689
+ "T1082": {
1690
+ "precision": 0.3158,
1691
+ "recall": 0.3529,
1692
+ "f1": 0.3333,
1693
+ "support": 17,
1694
+ "predicted": 19
1695
+ },
1696
+ "T1083": {
1697
+ "precision": 0.3077,
1698
+ "recall": 0.3077,
1699
+ "f1": 0.3077,
1700
+ "support": 13,
1701
+ "predicted": 13
1702
+ },
1703
+ "T1090": {
1704
+ "precision": 0.5217,
1705
+ "recall": 0.6667,
1706
+ "f1": 0.5854,
1707
+ "support": 18,
1708
+ "predicted": 23
1709
+ },
1710
+ "T1095": {
1711
+ "precision": 0.5714,
1712
+ "recall": 0.6667,
1713
+ "f1": 0.6154,
1714
+ "support": 12,
1715
+ "predicted": 14
1716
+ },
1717
+ "T1105": {
1718
+ "precision": 0.4,
1719
+ "recall": 0.6,
1720
+ "f1": 0.48,
1721
+ "support": 40,
1722
+ "predicted": 60
1723
+ },
1724
+ "T1106": {
1725
+ "precision": 0.4359,
1726
+ "recall": 0.8095,
1727
+ "f1": 0.5667,
1728
+ "support": 21,
1729
+ "predicted": 39
1730
+ },
1731
+ "T1110": {
1732
+ "precision": 0.5,
1733
+ "recall": 1.0,
1734
+ "f1": 0.6667,
1735
+ "support": 3,
1736
+ "predicted": 6
1737
+ },
1738
+ "T1112": {
1739
+ "precision": 0.4375,
1740
+ "recall": 0.9333,
1741
+ "f1": 0.5957,
1742
+ "support": 15,
1743
+ "predicted": 32
1744
+ },
1745
+ "T1113": {
1746
+ "precision": 0.7,
1747
+ "recall": 0.875,
1748
+ "f1": 0.7778,
1749
+ "support": 8,
1750
+ "predicted": 10
1751
+ },
1752
+ "T1140": {
1753
+ "precision": 0.6087,
1754
+ "recall": 0.8358,
1755
+ "f1": 0.7044,
1756
+ "support": 67,
1757
+ "predicted": 92
1758
+ },
1759
+ "T1190": {
1760
+ "precision": 0.6667,
1761
+ "recall": 0.6154,
1762
+ "f1": 0.64,
1763
+ "support": 13,
1764
+ "predicted": 12
1765
+ },
1766
+ "T1204.002": {
1767
+ "precision": 0.4118,
1768
+ "recall": 0.7,
1769
+ "f1": 0.5185,
1770
+ "support": 10,
1771
+ "predicted": 17
1772
+ },
1773
+ "T1210": {
1774
+ "precision": 0.0,
1775
+ "recall": 0.0,
1776
+ "f1": 0.0,
1777
+ "support": 2,
1778
+ "predicted": 0
1779
+ },
1780
+ "T1218.011": {
1781
+ "precision": 0.5,
1782
+ "recall": 0.5455,
1783
+ "f1": 0.5217,
1784
+ "support": 11,
1785
+ "predicted": 12
1786
+ },
1787
+ "T1219": {
1788
+ "precision": 0.3333,
1789
+ "recall": 0.75,
1790
+ "f1": 0.4615,
1791
+ "support": 4,
1792
+ "predicted": 9
1793
+ },
1794
+ "T1484.001": {
1795
+ "precision": 0.5,
1796
+ "recall": 0.75,
1797
+ "f1": 0.6,
1798
+ "support": 4,
1799
+ "predicted": 6
1800
+ },
1801
+ "T1518.001": {
1802
+ "precision": 1.0,
1803
+ "recall": 0.5,
1804
+ "f1": 0.6667,
1805
+ "support": 4,
1806
+ "predicted": 2
1807
+ },
1808
+ "T1543.003": {
1809
+ "precision": 0.6,
1810
+ "recall": 0.75,
1811
+ "f1": 0.6667,
1812
+ "support": 4,
1813
+ "predicted": 5
1814
+ },
1815
+ "T1547.001": {
1816
+ "precision": 0.4706,
1817
+ "recall": 0.8,
1818
+ "f1": 0.5926,
1819
+ "support": 10,
1820
+ "predicted": 17
1821
+ },
1822
+ "T1548.002": {
1823
+ "precision": 0.8,
1824
+ "recall": 0.6667,
1825
+ "f1": 0.7273,
1826
+ "support": 6,
1827
+ "predicted": 5
1828
+ },
1829
+ "T1552.001": {
1830
+ "precision": 0.5,
1831
+ "recall": 0.3333,
1832
+ "f1": 0.4,
1833
+ "support": 3,
1834
+ "predicted": 2
1835
+ },
1836
+ "T1562.001": {
1837
+ "precision": 0.5625,
1838
+ "recall": 0.6923,
1839
+ "f1": 0.6207,
1840
+ "support": 13,
1841
+ "predicted": 16
1842
+ },
1843
+ "T1564.001": {
1844
+ "precision": 0.5,
1845
+ "recall": 1.0,
1846
+ "f1": 0.6667,
1847
+ "support": 1,
1848
+ "predicted": 2
1849
+ },
1850
+ "T1566.001": {
1851
+ "precision": 0.5,
1852
+ "recall": 0.8,
1853
+ "f1": 0.6154,
1854
+ "support": 15,
1855
+ "predicted": 24
1856
+ },
1857
+ "T1569.002": {
1858
+ "precision": 0.5,
1859
+ "recall": 0.5,
1860
+ "f1": 0.5,
1861
+ "support": 2,
1862
+ "predicted": 2
1863
+ },
1864
+ "T1570": {
1865
+ "precision": 0.6667,
1866
+ "recall": 0.4615,
1867
+ "f1": 0.5455,
1868
+ "support": 13,
1869
+ "predicted": 9
1870
+ },
1871
+ "T1573.001": {
1872
+ "precision": 0.3889,
1873
+ "recall": 0.7778,
1874
+ "f1": 0.5185,
1875
+ "support": 9,
1876
+ "predicted": 18
1877
+ },
1878
+ "T1574.002": {
1879
+ "precision": 0.7273,
1880
+ "recall": 0.8889,
1881
+ "f1": 0.8,
1882
+ "support": 18,
1883
+ "predicted": 22
1884
+ }
1885
+ }
1886
+ },
1887
+ "per_class_threshold": {
1888
+ "thresholds": {
1889
+ "T1003.001": 0.7,
1890
+ "T1005": 0.75,
1891
+ "T1012": 0.25,
1892
+ "T1016": 0.3,
1893
+ "T1021.001": 0.3,
1894
+ "T1027": 0.65,
1895
+ "T1033": 0.1,
1896
+ "T1036.005": 0.35,
1897
+ "T1041": 0.8,
1898
+ "T1047": 0.35,
1899
+ "T1053.005": 0.35,
1900
+ "T1055": 0.6,
1901
+ "T1056.001": 0.2,
1902
+ "T1057": 0.35,
1903
+ "T1059.003": 0.4,
1904
+ "T1068": 0.2,
1905
+ "T1070.004": 0.7,
1906
+ "T1071.001": 0.45,
1907
+ "T1072": 0.05,
1908
+ "T1074.001": 0.05,
1909
+ "T1078": 0.9,
1910
+ "T1082": 0.55,
1911
+ "T1083": 0.6,
1912
+ "T1090": 0.5,
1913
+ "T1095": 0.3,
1914
+ "T1105": 0.7,
1915
+ "T1106": 0.4,
1916
+ "T1110": 0.45,
1917
+ "T1112": 0.65,
1918
+ "T1113": 0.6,
1919
+ "T1140": 0.5,
1920
+ "T1190": 0.55,
1921
+ "T1204.002": 0.8,
1922
+ "T1210": 0.05,
1923
+ "T1218.011": 0.4,
1924
+ "T1219": 0.6,
1925
+ "T1484.001": 0.35,
1926
+ "T1518.001": 0.3,
1927
+ "T1543.003": 0.55,
1928
+ "T1547.001": 0.65,
1929
+ "T1548.002": 0.05,
1930
+ "T1552.001": 0.35,
1931
+ "T1562.001": 0.85,
1932
+ "T1564.001": 0.3,
1933
+ "T1566.001": 0.35,
1934
+ "T1569.002": 0.25,
1935
+ "T1570": 0.55,
1936
+ "T1573.001": 0.35,
1937
+ "T1574.002": 0.6
1938
+ },
1939
+ "micro_f1": 0.5452,
1940
+ "macro_f1": 0.5065,
1941
+ "micro_precision": 0.5138,
1942
+ "micro_recall": 0.5807,
1943
+ "macro_precision": 0.5006,
1944
+ "macro_recall": 0.5664,
1945
+ "n_examples": 2767,
1946
+ "n_positive_labels": 737,
1947
+ "per_class": {
1948
+ "T1003.001": {
1949
+ "precision": 0.7692,
1950
+ "recall": 0.5882,
1951
+ "f1": 0.6667,
1952
+ "support": 17,
1953
+ "predicted": 13
1954
+ },
1955
+ "T1005": {
1956
+ "precision": 0.3333,
1957
+ "recall": 0.0833,
1958
+ "f1": 0.1333,
1959
+ "support": 12,
1960
+ "predicted": 3
1961
+ },
1962
+ "T1012": {
1963
+ "precision": 0.4,
1964
+ "recall": 0.5,
1965
+ "f1": 0.4444,
1966
+ "support": 4,
1967
+ "predicted": 5
1968
+ },
1969
+ "T1016": {
1970
+ "precision": 0.4286,
1971
+ "recall": 0.3333,
1972
+ "f1": 0.375,
1973
+ "support": 9,
1974
+ "predicted": 7
1975
+ },
1976
+ "T1021.001": {
1977
+ "precision": 0.6364,
1978
+ "recall": 0.875,
1979
+ "f1": 0.7368,
1980
+ "support": 16,
1981
+ "predicted": 22
1982
+ },
1983
+ "T1027": {
1984
+ "precision": 0.5213,
1985
+ "recall": 0.5104,
1986
+ "f1": 0.5158,
1987
+ "support": 96,
1988
+ "predicted": 94
1989
+ },
1990
+ "T1033": {
1991
+ "precision": 0.1875,
1992
+ "recall": 0.5,
1993
+ "f1": 0.2727,
1994
+ "support": 6,
1995
+ "predicted": 16
1996
+ },
1997
+ "T1036.005": {
1998
+ "precision": 0.4545,
1999
+ "recall": 0.625,
2000
+ "f1": 0.5263,
2001
+ "support": 8,
2002
+ "predicted": 11
2003
+ },
2004
+ "T1041": {
2005
+ "precision": 0.2857,
2006
+ "recall": 0.1538,
2007
+ "f1": 0.2,
2008
+ "support": 13,
2009
+ "predicted": 7
2010
+ },
2011
+ "T1047": {
2012
+ "precision": 0.5455,
2013
+ "recall": 0.6,
2014
+ "f1": 0.5714,
2015
+ "support": 10,
2016
+ "predicted": 11
2017
+ },
2018
+ "T1053.005": {
2019
+ "precision": 0.6842,
2020
+ "recall": 0.9286,
2021
+ "f1": 0.7879,
2022
+ "support": 14,
2023
+ "predicted": 19
2024
+ },
2025
+ "T1055": {
2026
+ "precision": 0.6591,
2027
+ "recall": 0.6744,
2028
+ "f1": 0.6667,
2029
+ "support": 43,
2030
+ "predicted": 44
2031
+ },
2032
+ "T1056.001": {
2033
+ "precision": 0.75,
2034
+ "recall": 1.0,
2035
+ "f1": 0.8571,
2036
+ "support": 6,
2037
+ "predicted": 8
2038
+ },
2039
+ "T1057": {
2040
+ "precision": 0.375,
2041
+ "recall": 0.5,
2042
+ "f1": 0.4286,
2043
+ "support": 12,
2044
+ "predicted": 16
2045
+ },
2046
+ "T1059.003": {
2047
+ "precision": 0.459,
2048
+ "recall": 0.6222,
2049
+ "f1": 0.5283,
2050
+ "support": 45,
2051
+ "predicted": 61
2052
+ },
2053
+ "T1068": {
2054
+ "precision": 0.0,
2055
+ "recall": 0.0,
2056
+ "f1": 0.0,
2057
+ "support": 1,
2058
+ "predicted": 2
2059
+ },
2060
+ "T1070.004": {
2061
+ "precision": 0.6,
2062
+ "recall": 0.6,
2063
+ "f1": 0.6,
2064
+ "support": 15,
2065
+ "predicted": 15
2066
+ },
2067
+ "T1071.001": {
2068
+ "precision": 0.5385,
2069
+ "recall": 0.5,
2070
+ "f1": 0.5185,
2071
+ "support": 28,
2072
+ "predicted": 26
2073
+ },
2074
+ "T1072": {
2075
+ "precision": 0.25,
2076
+ "recall": 0.5,
2077
+ "f1": 0.3333,
2078
+ "support": 2,
2079
+ "predicted": 4
2080
+ },
2081
+ "T1074.001": {
2082
+ "precision": 0.0286,
2083
+ "recall": 0.3333,
2084
+ "f1": 0.0526,
2085
+ "support": 3,
2086
+ "predicted": 35
2087
+ },
2088
+ "T1078": {
2089
+ "precision": 1.0,
2090
+ "recall": 0.3333,
2091
+ "f1": 0.5,
2092
+ "support": 21,
2093
+ "predicted": 7
2094
+ },
2095
+ "T1082": {
2096
+ "precision": 0.3846,
2097
+ "recall": 0.2941,
2098
+ "f1": 0.3333,
2099
+ "support": 17,
2100
+ "predicted": 13
2101
+ },
2102
+ "T1083": {
2103
+ "precision": 0.2857,
2104
+ "recall": 0.1538,
2105
+ "f1": 0.2,
2106
+ "support": 13,
2107
+ "predicted": 7
2108
+ },
2109
+ "T1090": {
2110
+ "precision": 0.5263,
2111
+ "recall": 0.5556,
2112
+ "f1": 0.5405,
2113
+ "support": 18,
2114
+ "predicted": 19
2115
+ },
2116
+ "T1095": {
2117
+ "precision": 0.6,
2118
+ "recall": 0.75,
2119
+ "f1": 0.6667,
2120
+ "support": 12,
2121
+ "predicted": 15
2122
+ },
2123
+ "T1105": {
2124
+ "precision": 0.4688,
2125
+ "recall": 0.375,
2126
+ "f1": 0.4167,
2127
+ "support": 40,
2128
+ "predicted": 32
2129
+ },
2130
+ "T1106": {
2131
+ "precision": 0.4848,
2132
+ "recall": 0.7619,
2133
+ "f1": 0.5926,
2134
+ "support": 21,
2135
+ "predicted": 33
2136
+ },
2137
+ "T1110": {
2138
+ "precision": 0.5,
2139
+ "recall": 1.0,
2140
+ "f1": 0.6667,
2141
+ "support": 3,
2142
+ "predicted": 6
2143
+ },
2144
+ "T1112": {
2145
+ "precision": 0.5652,
2146
+ "recall": 0.8667,
2147
+ "f1": 0.6842,
2148
+ "support": 15,
2149
+ "predicted": 23
2150
+ },
2151
+ "T1113": {
2152
+ "precision": 0.75,
2153
+ "recall": 0.75,
2154
+ "f1": 0.75,
2155
+ "support": 8,
2156
+ "predicted": 8
2157
+ },
2158
+ "T1140": {
2159
+ "precision": 0.6974,
2160
+ "recall": 0.791,
2161
+ "f1": 0.7413,
2162
+ "support": 67,
2163
+ "predicted": 76
2164
+ },
2165
+ "T1190": {
2166
+ "precision": 0.625,
2167
+ "recall": 0.3846,
2168
+ "f1": 0.4762,
2169
+ "support": 13,
2170
+ "predicted": 8
2171
+ },
2172
+ "T1204.002": {
2173
+ "precision": 0.3636,
2174
+ "recall": 0.4,
2175
+ "f1": 0.381,
2176
+ "support": 10,
2177
+ "predicted": 11
2178
+ },
2179
+ "T1210": {
2180
+ "precision": 0.0,
2181
+ "recall": 0.0,
2182
+ "f1": 0.0,
2183
+ "support": 2,
2184
+ "predicted": 16
2185
+ },
2186
+ "T1218.011": {
2187
+ "precision": 0.5,
2188
+ "recall": 0.5455,
2189
+ "f1": 0.5217,
2190
+ "support": 11,
2191
+ "predicted": 12
2192
+ },
2193
+ "T1219": {
2194
+ "precision": 0.2857,
2195
+ "recall": 0.5,
2196
+ "f1": 0.3636,
2197
+ "support": 4,
2198
+ "predicted": 7
2199
+ },
2200
+ "T1484.001": {
2201
+ "precision": 0.5,
2202
+ "recall": 0.75,
2203
+ "f1": 0.6,
2204
+ "support": 4,
2205
+ "predicted": 6
2206
+ },
2207
+ "T1518.001": {
2208
+ "precision": 0.6667,
2209
+ "recall": 0.5,
2210
+ "f1": 0.5714,
2211
+ "support": 4,
2212
+ "predicted": 3
2213
+ },
2214
+ "T1543.003": {
2215
+ "precision": 0.75,
2216
+ "recall": 0.75,
2217
+ "f1": 0.75,
2218
+ "support": 4,
2219
+ "predicted": 4
2220
+ },
2221
+ "T1547.001": {
2222
+ "precision": 0.5455,
2223
+ "recall": 0.6,
2224
+ "f1": 0.5714,
2225
+ "support": 10,
2226
+ "predicted": 11
2227
+ },
2228
+ "T1548.002": {
2229
+ "precision": 0.4286,
2230
+ "recall": 1.0,
2231
+ "f1": 0.6,
2232
+ "support": 6,
2233
+ "predicted": 14
2234
+ },
2235
+ "T1552.001": {
2236
+ "precision": 0.5,
2237
+ "recall": 0.3333,
2238
+ "f1": 0.4,
2239
+ "support": 3,
2240
+ "predicted": 2
2241
+ },
2242
+ "T1562.001": {
2243
+ "precision": 0.8182,
2244
+ "recall": 0.6923,
2245
+ "f1": 0.75,
2246
+ "support": 13,
2247
+ "predicted": 11
2248
+ },
2249
+ "T1564.001": {
2250
+ "precision": 0.5,
2251
+ "recall": 1.0,
2252
+ "f1": 0.6667,
2253
+ "support": 1,
2254
+ "predicted": 2
2255
+ },
2256
+ "T1566.001": {
2257
+ "precision": 0.5,
2258
+ "recall": 0.8,
2259
+ "f1": 0.6154,
2260
+ "support": 15,
2261
+ "predicted": 24
2262
+ },
2263
+ "T1569.002": {
2264
+ "precision": 0.3333,
2265
+ "recall": 0.5,
2266
+ "f1": 0.4,
2267
+ "support": 2,
2268
+ "predicted": 3
2269
+ },
2270
+ "T1570": {
2271
+ "precision": 0.8333,
2272
+ "recall": 0.3846,
2273
+ "f1": 0.5263,
2274
+ "support": 13,
2275
+ "predicted": 6
2276
+ },
2277
+ "T1573.001": {
2278
+ "precision": 0.3889,
2279
+ "recall": 0.7778,
2280
+ "f1": 0.5185,
2281
+ "support": 9,
2282
+ "predicted": 18
2283
+ },
2284
+ "T1574.002": {
2285
+ "precision": 0.8235,
2286
+ "recall": 0.7778,
2287
+ "f1": 0.8,
2288
+ "support": 18,
2289
+ "predicted": 17
2290
+ }
2291
+ }
2292
+ }
2293
+ }
2294
+ }
results/cv_5fold__modernbert.json ADDED
@@ -0,0 +1,154 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "folds": 5,
3
+ "model": "modernbert",
4
+ "include_single_label": false,
5
+ "n_records": 18437,
6
+ "n_documents": 151,
7
+ "per_fold": [
8
+ {
9
+ "fold": 1,
10
+ "n_train": 12217,
11
+ "n_dev": 2531,
12
+ "n_test": 3689,
13
+ "alpha": 0.65,
14
+ "tfidf_global": 0.46965437359234363,
15
+ "tfidf_perclass": 0.4462328501097996,
16
+ "modernbert_global": 0.47557457457038915,
17
+ "modernbert_perclass": 0.4671987334873316,
18
+ "ensemble_global": 0.4940021202369539,
19
+ "ensemble_perclass": 0.4901281436672364
20
+ },
21
+ {
22
+ "fold": 2,
23
+ "n_train": 11680,
24
+ "n_dev": 3070,
25
+ "n_test": 3687,
26
+ "alpha": 0.45,
27
+ "tfidf_global": 0.45555518141046525,
28
+ "tfidf_perclass": 0.43007794192384985,
29
+ "modernbert_global": 0.43093149426344785,
30
+ "modernbert_perclass": 0.42370811339004016,
31
+ "ensemble_global": 0.47739624781767503,
32
+ "ensemble_perclass": 0.47935636747537275
33
+ },
34
+ {
35
+ "fold": 3,
36
+ "n_train": 11485,
37
+ "n_dev": 3264,
38
+ "n_test": 3688,
39
+ "alpha": 0.6,
40
+ "tfidf_global": 0.3984827424083545,
41
+ "tfidf_perclass": 0.38690020936331676,
42
+ "modernbert_global": 0.3973866807983229,
43
+ "modernbert_perclass": 0.4004321602008274,
44
+ "ensemble_global": 0.43007618921982177,
45
+ "ensemble_perclass": 0.4383313673266281
46
+ },
47
+ {
48
+ "fold": 4,
49
+ "n_train": 11686,
50
+ "n_dev": 3066,
51
+ "n_test": 3685,
52
+ "alpha": 0.6,
53
+ "tfidf_global": 0.42770230210183274,
54
+ "tfidf_perclass": 0.41853640175810486,
55
+ "modernbert_global": 0.4411680228605778,
56
+ "modernbert_perclass": 0.4301876533459539,
57
+ "ensemble_global": 0.4470604882595841,
58
+ "ensemble_perclass": 0.46755732260095656
59
+ },
60
+ {
61
+ "fold": 5,
62
+ "n_train": 11736,
63
+ "n_dev": 3013,
64
+ "n_test": 3688,
65
+ "alpha": 0.4,
66
+ "tfidf_global": 0.5093374796188858,
67
+ "tfidf_perclass": 0.4814111877991654,
68
+ "modernbert_global": 0.4368041684081253,
69
+ "modernbert_perclass": 0.4099604272134864,
70
+ "ensemble_global": 0.505494951471835,
71
+ "ensemble_perclass": 0.49363457901388585
72
+ }
73
+ ],
74
+ "summary": {
75
+ "tfidf_global": {
76
+ "mean": 0.4521,
77
+ "std": 0.042,
78
+ "folds": [
79
+ 0.4697,
80
+ 0.4556,
81
+ 0.3985,
82
+ 0.4277,
83
+ 0.5093
84
+ ]
85
+ },
86
+ "tfidf_perclass": {
87
+ "mean": 0.4326,
88
+ "std": 0.0349,
89
+ "folds": [
90
+ 0.4462,
91
+ 0.4301,
92
+ 0.3869,
93
+ 0.4185,
94
+ 0.4814
95
+ ]
96
+ },
97
+ "modernbert_global": {
98
+ "mean": 0.4364,
99
+ "std": 0.0279,
100
+ "folds": [
101
+ 0.4756,
102
+ 0.4309,
103
+ 0.3974,
104
+ 0.4412,
105
+ 0.4368
106
+ ]
107
+ },
108
+ "modernbert_perclass": {
109
+ "mean": 0.4263,
110
+ "std": 0.0256,
111
+ "folds": [
112
+ 0.4672,
113
+ 0.4237,
114
+ 0.4004,
115
+ 0.4302,
116
+ 0.41
117
+ ]
118
+ },
119
+ "ensemble_global": {
120
+ "mean": 0.4708,
121
+ "std": 0.0317,
122
+ "folds": [
123
+ 0.494,
124
+ 0.4774,
125
+ 0.4301,
126
+ 0.4471,
127
+ 0.5055
128
+ ]
129
+ },
130
+ "ensemble_perclass": {
131
+ "mean": 0.4738,
132
+ "std": 0.0223,
133
+ "folds": [
134
+ 0.4901,
135
+ 0.4794,
136
+ 0.4383,
137
+ 0.4676,
138
+ 0.4936
139
+ ]
140
+ }
141
+ },
142
+ "ensemble_vs_tfidf_gap": {
143
+ "per_fold": [
144
+ 0.0205,
145
+ 0.0238,
146
+ 0.0398,
147
+ 0.0399,
148
+ -0.0157
149
+ ],
150
+ "mean": 0.0217,
151
+ "std": 0.0227,
152
+ "min": -0.0157
153
+ }
154
+ }
results/ensemble_tfidf_modernbert__document.json ADDED
@@ -0,0 +1,779 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "run": "ensemble_tfidf_modernbert",
3
+ "split_scheme": "document",
4
+ "alpha": 0.7,
5
+ "alpha_meaning": "scores = alpha*tfidf + (1-alpha)*bert",
6
+ "dev_macro_f1_at_best_alpha": 0.4745,
7
+ "global_threshold": {
8
+ "threshold": 0.2,
9
+ "micro_f1": 0.4696,
10
+ "macro_f1": 0.4504,
11
+ "micro_precision": 0.3394,
12
+ "micro_recall": 0.7618,
13
+ "macro_precision": 0.3711,
14
+ "macro_recall": 0.6685,
15
+ "n_examples": 3635,
16
+ "n_positive_labels": 911,
17
+ "per_class": {
18
+ "T1003.001": {
19
+ "precision": 0.4048,
20
+ "recall": 0.8095,
21
+ "f1": 0.5397,
22
+ "support": 21,
23
+ "predicted": 42
24
+ },
25
+ "T1005": {
26
+ "precision": 0.0563,
27
+ "recall": 0.5,
28
+ "f1": 0.1013,
29
+ "support": 8,
30
+ "predicted": 71
31
+ },
32
+ "T1012": {
33
+ "precision": 0.1875,
34
+ "recall": 0.75,
35
+ "f1": 0.3,
36
+ "support": 4,
37
+ "predicted": 16
38
+ },
39
+ "T1016": {
40
+ "precision": 0.2778,
41
+ "recall": 0.4545,
42
+ "f1": 0.3448,
43
+ "support": 11,
44
+ "predicted": 18
45
+ },
46
+ "T1021.001": {
47
+ "precision": 0.3077,
48
+ "recall": 1.0,
49
+ "f1": 0.4706,
50
+ "support": 12,
51
+ "predicted": 39
52
+ },
53
+ "T1027": {
54
+ "precision": 0.2917,
55
+ "recall": 0.8235,
56
+ "f1": 0.4308,
57
+ "support": 102,
58
+ "predicted": 288
59
+ },
60
+ "T1033": {
61
+ "precision": 0.5,
62
+ "recall": 0.9231,
63
+ "f1": 0.6486,
64
+ "support": 13,
65
+ "predicted": 24
66
+ },
67
+ "T1036.005": {
68
+ "precision": 0.2308,
69
+ "recall": 0.6,
70
+ "f1": 0.3333,
71
+ "support": 10,
72
+ "predicted": 26
73
+ },
74
+ "T1041": {
75
+ "precision": 0.2105,
76
+ "recall": 0.7059,
77
+ "f1": 0.3243,
78
+ "support": 17,
79
+ "predicted": 57
80
+ },
81
+ "T1047": {
82
+ "precision": 0.6579,
83
+ "recall": 0.8929,
84
+ "f1": 0.7576,
85
+ "support": 28,
86
+ "predicted": 38
87
+ },
88
+ "T1053.005": {
89
+ "precision": 0.5319,
90
+ "recall": 0.9615,
91
+ "f1": 0.6849,
92
+ "support": 26,
93
+ "predicted": 47
94
+ },
95
+ "T1055": {
96
+ "precision": 0.4459,
97
+ "recall": 0.825,
98
+ "f1": 0.5789,
99
+ "support": 40,
100
+ "predicted": 74
101
+ },
102
+ "T1056.001": {
103
+ "precision": 0.7143,
104
+ "recall": 1.0,
105
+ "f1": 0.8333,
106
+ "support": 15,
107
+ "predicted": 21
108
+ },
109
+ "T1057": {
110
+ "precision": 0.3448,
111
+ "recall": 0.6667,
112
+ "f1": 0.4545,
113
+ "support": 15,
114
+ "predicted": 29
115
+ },
116
+ "T1059.003": {
117
+ "precision": 0.3154,
118
+ "recall": 0.8039,
119
+ "f1": 0.453,
120
+ "support": 51,
121
+ "predicted": 130
122
+ },
123
+ "T1068": {
124
+ "precision": 1.0,
125
+ "recall": 0.3333,
126
+ "f1": 0.5,
127
+ "support": 3,
128
+ "predicted": 1
129
+ },
130
+ "T1070.004": {
131
+ "precision": 0.3103,
132
+ "recall": 0.75,
133
+ "f1": 0.439,
134
+ "support": 12,
135
+ "predicted": 29
136
+ },
137
+ "T1071.001": {
138
+ "precision": 0.2466,
139
+ "recall": 0.8571,
140
+ "f1": 0.383,
141
+ "support": 21,
142
+ "predicted": 73
143
+ },
144
+ "T1072": {
145
+ "precision": 0.0,
146
+ "recall": 0.0,
147
+ "f1": 0.0,
148
+ "support": 3,
149
+ "predicted": 0
150
+ },
151
+ "T1074.001": {
152
+ "precision": 0.1333,
153
+ "recall": 0.6667,
154
+ "f1": 0.2222,
155
+ "support": 3,
156
+ "predicted": 15
157
+ },
158
+ "T1078": {
159
+ "precision": 0.1619,
160
+ "recall": 0.7391,
161
+ "f1": 0.2656,
162
+ "support": 23,
163
+ "predicted": 105
164
+ },
165
+ "T1082": {
166
+ "precision": 0.2203,
167
+ "recall": 0.5417,
168
+ "f1": 0.3133,
169
+ "support": 24,
170
+ "predicted": 59
171
+ },
172
+ "T1083": {
173
+ "precision": 0.2128,
174
+ "recall": 0.6667,
175
+ "f1": 0.3226,
176
+ "support": 15,
177
+ "predicted": 47
178
+ },
179
+ "T1090": {
180
+ "precision": 0.3261,
181
+ "recall": 0.6522,
182
+ "f1": 0.4348,
183
+ "support": 23,
184
+ "predicted": 46
185
+ },
186
+ "T1095": {
187
+ "precision": 0.3333,
188
+ "recall": 0.3333,
189
+ "f1": 0.3333,
190
+ "support": 3,
191
+ "predicted": 3
192
+ },
193
+ "T1105": {
194
+ "precision": 0.3,
195
+ "recall": 0.7895,
196
+ "f1": 0.4348,
197
+ "support": 57,
198
+ "predicted": 150
199
+ },
200
+ "T1106": {
201
+ "precision": 0.375,
202
+ "recall": 0.7059,
203
+ "f1": 0.4898,
204
+ "support": 34,
205
+ "predicted": 64
206
+ },
207
+ "T1110": {
208
+ "precision": 0.5517,
209
+ "recall": 0.8421,
210
+ "f1": 0.6667,
211
+ "support": 19,
212
+ "predicted": 29
213
+ },
214
+ "T1112": {
215
+ "precision": 0.2553,
216
+ "recall": 0.8571,
217
+ "f1": 0.3934,
218
+ "support": 14,
219
+ "predicted": 47
220
+ },
221
+ "T1113": {
222
+ "precision": 0.4737,
223
+ "recall": 1.0,
224
+ "f1": 0.6429,
225
+ "support": 9,
226
+ "predicted": 19
227
+ },
228
+ "T1140": {
229
+ "precision": 0.5328,
230
+ "recall": 0.9559,
231
+ "f1": 0.6842,
232
+ "support": 68,
233
+ "predicted": 122
234
+ },
235
+ "T1190": {
236
+ "precision": 0.2903,
237
+ "recall": 0.6,
238
+ "f1": 0.3913,
239
+ "support": 15,
240
+ "predicted": 31
241
+ },
242
+ "T1204.002": {
243
+ "precision": 0.3,
244
+ "recall": 0.6,
245
+ "f1": 0.4,
246
+ "support": 15,
247
+ "predicted": 30
248
+ },
249
+ "T1210": {
250
+ "precision": 0.0,
251
+ "recall": 0.0,
252
+ "f1": 0.0,
253
+ "support": 2,
254
+ "predicted": 2
255
+ },
256
+ "T1218.011": {
257
+ "precision": 0.6,
258
+ "recall": 0.6429,
259
+ "f1": 0.6207,
260
+ "support": 14,
261
+ "predicted": 15
262
+ },
263
+ "T1219": {
264
+ "precision": 0.4375,
265
+ "recall": 0.3684,
266
+ "f1": 0.4,
267
+ "support": 19,
268
+ "predicted": 16
269
+ },
270
+ "T1484.001": {
271
+ "precision": 0.5,
272
+ "recall": 0.4,
273
+ "f1": 0.4444,
274
+ "support": 5,
275
+ "predicted": 4
276
+ },
277
+ "T1518.001": {
278
+ "precision": 0.2857,
279
+ "recall": 0.5714,
280
+ "f1": 0.381,
281
+ "support": 7,
282
+ "predicted": 14
283
+ },
284
+ "T1543.003": {
285
+ "precision": 0.6667,
286
+ "recall": 0.7692,
287
+ "f1": 0.7143,
288
+ "support": 13,
289
+ "predicted": 15
290
+ },
291
+ "T1547.001": {
292
+ "precision": 0.4333,
293
+ "recall": 0.8125,
294
+ "f1": 0.5652,
295
+ "support": 16,
296
+ "predicted": 30
297
+ },
298
+ "T1548.002": {
299
+ "precision": 0.6667,
300
+ "recall": 1.0,
301
+ "f1": 0.8,
302
+ "support": 2,
303
+ "predicted": 3
304
+ },
305
+ "T1552.001": {
306
+ "precision": 0.1667,
307
+ "recall": 0.25,
308
+ "f1": 0.2,
309
+ "support": 4,
310
+ "predicted": 6
311
+ },
312
+ "T1562.001": {
313
+ "precision": 0.6176,
314
+ "recall": 0.8077,
315
+ "f1": 0.7,
316
+ "support": 26,
317
+ "predicted": 34
318
+ },
319
+ "T1564.001": {
320
+ "precision": 0.5,
321
+ "recall": 0.6667,
322
+ "f1": 0.5714,
323
+ "support": 3,
324
+ "predicted": 4
325
+ },
326
+ "T1566.001": {
327
+ "precision": 0.24,
328
+ "recall": 0.4615,
329
+ "f1": 0.3158,
330
+ "support": 13,
331
+ "predicted": 25
332
+ },
333
+ "T1569.002": {
334
+ "precision": 0.2,
335
+ "recall": 0.2,
336
+ "f1": 0.2,
337
+ "support": 5,
338
+ "predicted": 5
339
+ },
340
+ "T1570": {
341
+ "precision": 0.1333,
342
+ "recall": 0.6667,
343
+ "f1": 0.2222,
344
+ "support": 6,
345
+ "predicted": 30
346
+ },
347
+ "T1573.001": {
348
+ "precision": 0.4783,
349
+ "recall": 0.6875,
350
+ "f1": 0.5641,
351
+ "support": 16,
352
+ "predicted": 23
353
+ },
354
+ "T1574.002": {
355
+ "precision": 0.7586,
356
+ "recall": 0.8462,
357
+ "f1": 0.8,
358
+ "support": 26,
359
+ "predicted": 29
360
+ }
361
+ }
362
+ },
363
+ "per_class_threshold": {
364
+ "thresholds": {
365
+ "T1003.001": 0.7,
366
+ "T1005": 0.7,
367
+ "T1012": 0.2,
368
+ "T1016": 0.25,
369
+ "T1021.001": 0.55,
370
+ "T1027": 0.4,
371
+ "T1033": 0.35,
372
+ "T1036.005": 0.15,
373
+ "T1041": 0.5,
374
+ "T1047": 0.15,
375
+ "T1053.005": 0.4,
376
+ "T1055": 0.55,
377
+ "T1056.001": 0.35,
378
+ "T1057": 0.5,
379
+ "T1059.003": 0.7,
380
+ "T1068": 0.2,
381
+ "T1070.004": 0.2,
382
+ "T1071.001": 0.6,
383
+ "T1072": 0.05,
384
+ "T1074.001": 0.05,
385
+ "T1078": 0.4,
386
+ "T1082": 0.8,
387
+ "T1083": 0.35,
388
+ "T1090": 0.3,
389
+ "T1095": 0.2,
390
+ "T1105": 0.65,
391
+ "T1106": 0.5,
392
+ "T1110": 0.3,
393
+ "T1112": 0.9,
394
+ "T1113": 0.1,
395
+ "T1140": 0.45,
396
+ "T1190": 0.3,
397
+ "T1204.002": 0.3,
398
+ "T1210": 0.05,
399
+ "T1218.011": 0.15,
400
+ "T1219": 0.15,
401
+ "T1484.001": 0.1,
402
+ "T1518.001": 0.2,
403
+ "T1543.003": 0.4,
404
+ "T1547.001": 0.55,
405
+ "T1548.002": 0.1,
406
+ "T1552.001": 0.05,
407
+ "T1562.001": 0.4,
408
+ "T1564.001": 0.2,
409
+ "T1566.001": 0.4,
410
+ "T1569.002": 0.1,
411
+ "T1570": 0.65,
412
+ "T1573.001": 0.2,
413
+ "T1574.002": 0.2
414
+ },
415
+ "micro_f1": 0.5243,
416
+ "macro_f1": 0.4965,
417
+ "micro_precision": 0.45,
418
+ "micro_recall": 0.6279,
419
+ "macro_precision": 0.4821,
420
+ "macro_recall": 0.6089,
421
+ "n_examples": 3635,
422
+ "n_positive_labels": 911,
423
+ "per_class": {
424
+ "T1003.001": {
425
+ "precision": 0.7143,
426
+ "recall": 0.7143,
427
+ "f1": 0.7143,
428
+ "support": 21,
429
+ "predicted": 21
430
+ },
431
+ "T1005": {
432
+ "precision": 0.375,
433
+ "recall": 0.375,
434
+ "f1": 0.375,
435
+ "support": 8,
436
+ "predicted": 8
437
+ },
438
+ "T1012": {
439
+ "precision": 0.1875,
440
+ "recall": 0.75,
441
+ "f1": 0.3,
442
+ "support": 4,
443
+ "predicted": 16
444
+ },
445
+ "T1016": {
446
+ "precision": 0.3571,
447
+ "recall": 0.4545,
448
+ "f1": 0.4,
449
+ "support": 11,
450
+ "predicted": 14
451
+ },
452
+ "T1021.001": {
453
+ "precision": 0.5263,
454
+ "recall": 0.8333,
455
+ "f1": 0.6452,
456
+ "support": 12,
457
+ "predicted": 19
458
+ },
459
+ "T1027": {
460
+ "precision": 0.4438,
461
+ "recall": 0.6961,
462
+ "f1": 0.542,
463
+ "support": 102,
464
+ "predicted": 160
465
+ },
466
+ "T1033": {
467
+ "precision": 0.625,
468
+ "recall": 0.7692,
469
+ "f1": 0.6897,
470
+ "support": 13,
471
+ "predicted": 16
472
+ },
473
+ "T1036.005": {
474
+ "precision": 0.1579,
475
+ "recall": 0.6,
476
+ "f1": 0.25,
477
+ "support": 10,
478
+ "predicted": 38
479
+ },
480
+ "T1041": {
481
+ "precision": 0.28,
482
+ "recall": 0.4118,
483
+ "f1": 0.3333,
484
+ "support": 17,
485
+ "predicted": 25
486
+ },
487
+ "T1047": {
488
+ "precision": 0.6098,
489
+ "recall": 0.8929,
490
+ "f1": 0.7246,
491
+ "support": 28,
492
+ "predicted": 41
493
+ },
494
+ "T1053.005": {
495
+ "precision": 0.625,
496
+ "recall": 0.9615,
497
+ "f1": 0.7576,
498
+ "support": 26,
499
+ "predicted": 40
500
+ },
501
+ "T1055": {
502
+ "precision": 0.6136,
503
+ "recall": 0.675,
504
+ "f1": 0.6429,
505
+ "support": 40,
506
+ "predicted": 44
507
+ },
508
+ "T1056.001": {
509
+ "precision": 0.7778,
510
+ "recall": 0.9333,
511
+ "f1": 0.8485,
512
+ "support": 15,
513
+ "predicted": 18
514
+ },
515
+ "T1057": {
516
+ "precision": 0.8182,
517
+ "recall": 0.6,
518
+ "f1": 0.6923,
519
+ "support": 15,
520
+ "predicted": 11
521
+ },
522
+ "T1059.003": {
523
+ "precision": 0.6667,
524
+ "recall": 0.3922,
525
+ "f1": 0.4938,
526
+ "support": 51,
527
+ "predicted": 30
528
+ },
529
+ "T1068": {
530
+ "precision": 1.0,
531
+ "recall": 0.3333,
532
+ "f1": 0.5,
533
+ "support": 3,
534
+ "predicted": 1
535
+ },
536
+ "T1070.004": {
537
+ "precision": 0.3103,
538
+ "recall": 0.75,
539
+ "f1": 0.439,
540
+ "support": 12,
541
+ "predicted": 29
542
+ },
543
+ "T1071.001": {
544
+ "precision": 0.4118,
545
+ "recall": 0.6667,
546
+ "f1": 0.5091,
547
+ "support": 21,
548
+ "predicted": 34
549
+ },
550
+ "T1072": {
551
+ "precision": 0.0,
552
+ "recall": 0.0,
553
+ "f1": 0.0,
554
+ "support": 3,
555
+ "predicted": 0
556
+ },
557
+ "T1074.001": {
558
+ "precision": 0.0303,
559
+ "recall": 0.6667,
560
+ "f1": 0.058,
561
+ "support": 3,
562
+ "predicted": 66
563
+ },
564
+ "T1078": {
565
+ "precision": 0.25,
566
+ "recall": 0.5652,
567
+ "f1": 0.3467,
568
+ "support": 23,
569
+ "predicted": 52
570
+ },
571
+ "T1082": {
572
+ "precision": 0.8,
573
+ "recall": 0.3333,
574
+ "f1": 0.4706,
575
+ "support": 24,
576
+ "predicted": 10
577
+ },
578
+ "T1083": {
579
+ "precision": 0.3333,
580
+ "recall": 0.6667,
581
+ "f1": 0.4444,
582
+ "support": 15,
583
+ "predicted": 30
584
+ },
585
+ "T1090": {
586
+ "precision": 0.4333,
587
+ "recall": 0.5652,
588
+ "f1": 0.4906,
589
+ "support": 23,
590
+ "predicted": 30
591
+ },
592
+ "T1095": {
593
+ "precision": 0.3333,
594
+ "recall": 0.3333,
595
+ "f1": 0.3333,
596
+ "support": 3,
597
+ "predicted": 3
598
+ },
599
+ "T1105": {
600
+ "precision": 0.6364,
601
+ "recall": 0.3684,
602
+ "f1": 0.4667,
603
+ "support": 57,
604
+ "predicted": 33
605
+ },
606
+ "T1106": {
607
+ "precision": 0.5556,
608
+ "recall": 0.4412,
609
+ "f1": 0.4918,
610
+ "support": 34,
611
+ "predicted": 27
612
+ },
613
+ "T1110": {
614
+ "precision": 0.6364,
615
+ "recall": 0.7368,
616
+ "f1": 0.6829,
617
+ "support": 19,
618
+ "predicted": 22
619
+ },
620
+ "T1112": {
621
+ "precision": 0.6667,
622
+ "recall": 0.2857,
623
+ "f1": 0.4,
624
+ "support": 14,
625
+ "predicted": 6
626
+ },
627
+ "T1113": {
628
+ "precision": 0.36,
629
+ "recall": 1.0,
630
+ "f1": 0.5294,
631
+ "support": 9,
632
+ "predicted": 25
633
+ },
634
+ "T1140": {
635
+ "precision": 0.7143,
636
+ "recall": 0.8824,
637
+ "f1": 0.7895,
638
+ "support": 68,
639
+ "predicted": 84
640
+ },
641
+ "T1190": {
642
+ "precision": 0.3684,
643
+ "recall": 0.4667,
644
+ "f1": 0.4118,
645
+ "support": 15,
646
+ "predicted": 19
647
+ },
648
+ "T1204.002": {
649
+ "precision": 0.375,
650
+ "recall": 0.6,
651
+ "f1": 0.4615,
652
+ "support": 15,
653
+ "predicted": 24
654
+ },
655
+ "T1210": {
656
+ "precision": 0.0227,
657
+ "recall": 0.5,
658
+ "f1": 0.0435,
659
+ "support": 2,
660
+ "predicted": 44
661
+ },
662
+ "T1218.011": {
663
+ "precision": 0.5882,
664
+ "recall": 0.7143,
665
+ "f1": 0.6452,
666
+ "support": 14,
667
+ "predicted": 17
668
+ },
669
+ "T1219": {
670
+ "precision": 0.4211,
671
+ "recall": 0.4211,
672
+ "f1": 0.4211,
673
+ "support": 19,
674
+ "predicted": 19
675
+ },
676
+ "T1484.001": {
677
+ "precision": 0.4167,
678
+ "recall": 1.0,
679
+ "f1": 0.5882,
680
+ "support": 5,
681
+ "predicted": 12
682
+ },
683
+ "T1518.001": {
684
+ "precision": 0.2857,
685
+ "recall": 0.5714,
686
+ "f1": 0.381,
687
+ "support": 7,
688
+ "predicted": 14
689
+ },
690
+ "T1543.003": {
691
+ "precision": 0.8333,
692
+ "recall": 0.3846,
693
+ "f1": 0.5263,
694
+ "support": 13,
695
+ "predicted": 6
696
+ },
697
+ "T1547.001": {
698
+ "precision": 0.7857,
699
+ "recall": 0.6875,
700
+ "f1": 0.7333,
701
+ "support": 16,
702
+ "predicted": 14
703
+ },
704
+ "T1548.002": {
705
+ "precision": 0.6667,
706
+ "recall": 1.0,
707
+ "f1": 0.8,
708
+ "support": 2,
709
+ "predicted": 3
710
+ },
711
+ "T1552.001": {
712
+ "precision": 0.0606,
713
+ "recall": 0.5,
714
+ "f1": 0.1081,
715
+ "support": 4,
716
+ "predicted": 33
717
+ },
718
+ "T1562.001": {
719
+ "precision": 0.7917,
720
+ "recall": 0.7308,
721
+ "f1": 0.76,
722
+ "support": 26,
723
+ "predicted": 24
724
+ },
725
+ "T1564.001": {
726
+ "precision": 0.5,
727
+ "recall": 0.6667,
728
+ "f1": 0.5714,
729
+ "support": 3,
730
+ "predicted": 4
731
+ },
732
+ "T1566.001": {
733
+ "precision": 0.25,
734
+ "recall": 0.3077,
735
+ "f1": 0.2759,
736
+ "support": 13,
737
+ "predicted": 16
738
+ },
739
+ "T1569.002": {
740
+ "precision": 0.2727,
741
+ "recall": 0.6,
742
+ "f1": 0.375,
743
+ "support": 5,
744
+ "predicted": 11
745
+ },
746
+ "T1570": {
747
+ "precision": 0.5,
748
+ "recall": 0.5,
749
+ "f1": 0.5,
750
+ "support": 6,
751
+ "predicted": 6
752
+ },
753
+ "T1573.001": {
754
+ "precision": 0.4783,
755
+ "recall": 0.6875,
756
+ "f1": 0.5641,
757
+ "support": 16,
758
+ "predicted": 23
759
+ },
760
+ "T1574.002": {
761
+ "precision": 0.7586,
762
+ "recall": 0.8462,
763
+ "f1": 0.8,
764
+ "support": 26,
765
+ "predicted": 29
766
+ }
767
+ }
768
+ },
769
+ "components_test_macro_f1": {
770
+ "tfidf_lr": {
771
+ "global": 0.4492,
772
+ "per_class": 0.4536
773
+ },
774
+ "modernbert": {
775
+ "global": 0.4253,
776
+ "per_class": 0.4454
777
+ }
778
+ }
779
+ }
results/modernbert__document.json ADDED
@@ -0,0 +1,769 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "run": "modernbert__document",
3
+ "base_model": "answerdotai/ModernBERT-base",
4
+ "split_scheme": "document",
5
+ "best_epoch": 5,
6
+ "dev_macro_f1": 0.4139,
7
+ "global_threshold": {
8
+ "threshold": 0.45,
9
+ "micro_f1": 0.4936,
10
+ "macro_f1": 0.4253,
11
+ "micro_precision": 0.4119,
12
+ "micro_recall": 0.6158,
13
+ "macro_precision": 0.3983,
14
+ "macro_recall": 0.5201,
15
+ "n_examples": 3635,
16
+ "n_positive_labels": 911,
17
+ "per_class": {
18
+ "T1003.001": {
19
+ "precision": 0.4839,
20
+ "recall": 0.7143,
21
+ "f1": 0.5769,
22
+ "support": 21,
23
+ "predicted": 31
24
+ },
25
+ "T1005": {
26
+ "precision": 0.087,
27
+ "recall": 0.5,
28
+ "f1": 0.1481,
29
+ "support": 8,
30
+ "predicted": 46
31
+ },
32
+ "T1012": {
33
+ "precision": 0.25,
34
+ "recall": 0.5,
35
+ "f1": 0.3333,
36
+ "support": 4,
37
+ "predicted": 8
38
+ },
39
+ "T1016": {
40
+ "precision": 0.5,
41
+ "recall": 0.2727,
42
+ "f1": 0.3529,
43
+ "support": 11,
44
+ "predicted": 6
45
+ },
46
+ "T1021.001": {
47
+ "precision": 0.44,
48
+ "recall": 0.9167,
49
+ "f1": 0.5946,
50
+ "support": 12,
51
+ "predicted": 25
52
+ },
53
+ "T1027": {
54
+ "precision": 0.3946,
55
+ "recall": 0.7157,
56
+ "f1": 0.5087,
57
+ "support": 102,
58
+ "predicted": 185
59
+ },
60
+ "T1033": {
61
+ "precision": 0.3333,
62
+ "recall": 0.2308,
63
+ "f1": 0.2727,
64
+ "support": 13,
65
+ "predicted": 9
66
+ },
67
+ "T1036.005": {
68
+ "precision": 0.2308,
69
+ "recall": 0.3,
70
+ "f1": 0.2609,
71
+ "support": 10,
72
+ "predicted": 13
73
+ },
74
+ "T1041": {
75
+ "precision": 0.3448,
76
+ "recall": 0.5882,
77
+ "f1": 0.4348,
78
+ "support": 17,
79
+ "predicted": 29
80
+ },
81
+ "T1047": {
82
+ "precision": 0.6923,
83
+ "recall": 0.6429,
84
+ "f1": 0.6667,
85
+ "support": 28,
86
+ "predicted": 26
87
+ },
88
+ "T1053.005": {
89
+ "precision": 0.6154,
90
+ "recall": 0.9231,
91
+ "f1": 0.7385,
92
+ "support": 26,
93
+ "predicted": 39
94
+ },
95
+ "T1055": {
96
+ "precision": 0.5082,
97
+ "recall": 0.775,
98
+ "f1": 0.6139,
99
+ "support": 40,
100
+ "predicted": 61
101
+ },
102
+ "T1056.001": {
103
+ "precision": 0.9333,
104
+ "recall": 0.9333,
105
+ "f1": 0.9333,
106
+ "support": 15,
107
+ "predicted": 15
108
+ },
109
+ "T1057": {
110
+ "precision": 0.3182,
111
+ "recall": 0.4667,
112
+ "f1": 0.3784,
113
+ "support": 15,
114
+ "predicted": 22
115
+ },
116
+ "T1059.003": {
117
+ "precision": 0.3407,
118
+ "recall": 0.6078,
119
+ "f1": 0.4366,
120
+ "support": 51,
121
+ "predicted": 91
122
+ },
123
+ "T1068": {
124
+ "precision": 0.0,
125
+ "recall": 0.0,
126
+ "f1": 0.0,
127
+ "support": 3,
128
+ "predicted": 0
129
+ },
130
+ "T1070.004": {
131
+ "precision": 0.3158,
132
+ "recall": 0.5,
133
+ "f1": 0.3871,
134
+ "support": 12,
135
+ "predicted": 19
136
+ },
137
+ "T1071.001": {
138
+ "precision": 0.2951,
139
+ "recall": 0.8571,
140
+ "f1": 0.439,
141
+ "support": 21,
142
+ "predicted": 61
143
+ },
144
+ "T1072": {
145
+ "precision": 0.0,
146
+ "recall": 0.0,
147
+ "f1": 0.0,
148
+ "support": 3,
149
+ "predicted": 0
150
+ },
151
+ "T1074.001": {
152
+ "precision": 0.1333,
153
+ "recall": 0.6667,
154
+ "f1": 0.2222,
155
+ "support": 3,
156
+ "predicted": 15
157
+ },
158
+ "T1078": {
159
+ "precision": 0.1642,
160
+ "recall": 0.4783,
161
+ "f1": 0.2444,
162
+ "support": 23,
163
+ "predicted": 67
164
+ },
165
+ "T1082": {
166
+ "precision": 0.3448,
167
+ "recall": 0.4167,
168
+ "f1": 0.3774,
169
+ "support": 24,
170
+ "predicted": 29
171
+ },
172
+ "T1083": {
173
+ "precision": 0.3214,
174
+ "recall": 0.6,
175
+ "f1": 0.4186,
176
+ "support": 15,
177
+ "predicted": 28
178
+ },
179
+ "T1090": {
180
+ "precision": 0.3784,
181
+ "recall": 0.6087,
182
+ "f1": 0.4667,
183
+ "support": 23,
184
+ "predicted": 37
185
+ },
186
+ "T1095": {
187
+ "precision": 0.3333,
188
+ "recall": 0.3333,
189
+ "f1": 0.3333,
190
+ "support": 3,
191
+ "predicted": 3
192
+ },
193
+ "T1105": {
194
+ "precision": 0.3824,
195
+ "recall": 0.6842,
196
+ "f1": 0.4906,
197
+ "support": 57,
198
+ "predicted": 102
199
+ },
200
+ "T1106": {
201
+ "precision": 0.5405,
202
+ "recall": 0.5882,
203
+ "f1": 0.5634,
204
+ "support": 34,
205
+ "predicted": 37
206
+ },
207
+ "T1110": {
208
+ "precision": 0.7647,
209
+ "recall": 0.6842,
210
+ "f1": 0.7222,
211
+ "support": 19,
212
+ "predicted": 17
213
+ },
214
+ "T1112": {
215
+ "precision": 0.2821,
216
+ "recall": 0.7857,
217
+ "f1": 0.4151,
218
+ "support": 14,
219
+ "predicted": 39
220
+ },
221
+ "T1113": {
222
+ "precision": 0.5,
223
+ "recall": 0.7778,
224
+ "f1": 0.6087,
225
+ "support": 9,
226
+ "predicted": 14
227
+ },
228
+ "T1140": {
229
+ "precision": 0.6591,
230
+ "recall": 0.8529,
231
+ "f1": 0.7436,
232
+ "support": 68,
233
+ "predicted": 88
234
+ },
235
+ "T1190": {
236
+ "precision": 0.1739,
237
+ "recall": 0.2667,
238
+ "f1": 0.2105,
239
+ "support": 15,
240
+ "predicted": 23
241
+ },
242
+ "T1204.002": {
243
+ "precision": 0.3889,
244
+ "recall": 0.4667,
245
+ "f1": 0.4242,
246
+ "support": 15,
247
+ "predicted": 18
248
+ },
249
+ "T1210": {
250
+ "precision": 0.3333,
251
+ "recall": 0.5,
252
+ "f1": 0.4,
253
+ "support": 2,
254
+ "predicted": 3
255
+ },
256
+ "T1218.011": {
257
+ "precision": 0.6364,
258
+ "recall": 0.5,
259
+ "f1": 0.56,
260
+ "support": 14,
261
+ "predicted": 11
262
+ },
263
+ "T1219": {
264
+ "precision": 0.5714,
265
+ "recall": 0.2105,
266
+ "f1": 0.3077,
267
+ "support": 19,
268
+ "predicted": 7
269
+ },
270
+ "T1484.001": {
271
+ "precision": 0.5,
272
+ "recall": 0.2,
273
+ "f1": 0.2857,
274
+ "support": 5,
275
+ "predicted": 2
276
+ },
277
+ "T1518.001": {
278
+ "precision": 0.3,
279
+ "recall": 0.4286,
280
+ "f1": 0.3529,
281
+ "support": 7,
282
+ "predicted": 10
283
+ },
284
+ "T1543.003": {
285
+ "precision": 0.7778,
286
+ "recall": 0.5385,
287
+ "f1": 0.6364,
288
+ "support": 13,
289
+ "predicted": 9
290
+ },
291
+ "T1547.001": {
292
+ "precision": 0.5455,
293
+ "recall": 0.75,
294
+ "f1": 0.6316,
295
+ "support": 16,
296
+ "predicted": 22
297
+ },
298
+ "T1548.002": {
299
+ "precision": 0.6667,
300
+ "recall": 1.0,
301
+ "f1": 0.8,
302
+ "support": 2,
303
+ "predicted": 3
304
+ },
305
+ "T1552.001": {
306
+ "precision": 0.0,
307
+ "recall": 0.0,
308
+ "f1": 0.0,
309
+ "support": 4,
310
+ "predicted": 0
311
+ },
312
+ "T1562.001": {
313
+ "precision": 0.8095,
314
+ "recall": 0.6538,
315
+ "f1": 0.7234,
316
+ "support": 26,
317
+ "predicted": 21
318
+ },
319
+ "T1564.001": {
320
+ "precision": 0.0,
321
+ "recall": 0.0,
322
+ "f1": 0.0,
323
+ "support": 3,
324
+ "predicted": 0
325
+ },
326
+ "T1566.001": {
327
+ "precision": 0.2667,
328
+ "recall": 0.3077,
329
+ "f1": 0.2857,
330
+ "support": 13,
331
+ "predicted": 15
332
+ },
333
+ "T1569.002": {
334
+ "precision": 0.3333,
335
+ "recall": 0.2,
336
+ "f1": 0.25,
337
+ "support": 5,
338
+ "predicted": 3
339
+ },
340
+ "T1570": {
341
+ "precision": 0.2105,
342
+ "recall": 0.6667,
343
+ "f1": 0.32,
344
+ "support": 6,
345
+ "predicted": 19
346
+ },
347
+ "T1573.001": {
348
+ "precision": 0.3529,
349
+ "recall": 0.375,
350
+ "f1": 0.3636,
351
+ "support": 16,
352
+ "predicted": 17
353
+ },
354
+ "T1574.002": {
355
+ "precision": 0.7647,
356
+ "recall": 0.5,
357
+ "f1": 0.6047,
358
+ "support": 26,
359
+ "predicted": 17
360
+ }
361
+ }
362
+ },
363
+ "per_class_threshold": {
364
+ "thresholds": {
365
+ "T1003.001": 0.7,
366
+ "T1005": 0.85,
367
+ "T1012": 0.35,
368
+ "T1016": 0.25,
369
+ "T1021.001": 0.7,
370
+ "T1027": 0.8,
371
+ "T1033": 0.25,
372
+ "T1036.005": 0.15,
373
+ "T1041": 0.4,
374
+ "T1047": 0.45,
375
+ "T1053.005": 0.6,
376
+ "T1055": 0.85,
377
+ "T1056.001": 0.6,
378
+ "T1057": 0.4,
379
+ "T1059.003": 0.6,
380
+ "T1068": 0.15,
381
+ "T1070.004": 0.65,
382
+ "T1071.001": 0.8,
383
+ "T1072": 0.05,
384
+ "T1074.001": 0.2,
385
+ "T1078": 0.65,
386
+ "T1082": 0.95,
387
+ "T1083": 0.95,
388
+ "T1090": 0.85,
389
+ "T1095": 0.6,
390
+ "T1105": 0.85,
391
+ "T1106": 0.7,
392
+ "T1110": 0.35,
393
+ "T1112": 0.9,
394
+ "T1113": 0.15,
395
+ "T1140": 0.75,
396
+ "T1190": 0.3,
397
+ "T1204.002": 0.5,
398
+ "T1210": 0.05,
399
+ "T1218.011": 0.4,
400
+ "T1219": 0.35,
401
+ "T1484.001": 0.2,
402
+ "T1518.001": 0.4,
403
+ "T1543.003": 0.1,
404
+ "T1547.001": 0.55,
405
+ "T1548.002": 0.2,
406
+ "T1552.001": 0.05,
407
+ "T1562.001": 0.75,
408
+ "T1564.001": 0.15,
409
+ "T1566.001": 0.2,
410
+ "T1569.002": 0.25,
411
+ "T1570": 0.8,
412
+ "T1573.001": 0.6,
413
+ "T1574.002": 0.65
414
+ },
415
+ "micro_f1": 0.4784,
416
+ "macro_f1": 0.4454,
417
+ "micro_precision": 0.4182,
418
+ "micro_recall": 0.5587,
419
+ "macro_precision": 0.4397,
420
+ "macro_recall": 0.5491,
421
+ "n_examples": 3635,
422
+ "n_positive_labels": 911,
423
+ "per_class": {
424
+ "T1003.001": {
425
+ "precision": 0.6818,
426
+ "recall": 0.7143,
427
+ "f1": 0.6977,
428
+ "support": 21,
429
+ "predicted": 22
430
+ },
431
+ "T1005": {
432
+ "precision": 0.25,
433
+ "recall": 0.375,
434
+ "f1": 0.3,
435
+ "support": 8,
436
+ "predicted": 12
437
+ },
438
+ "T1012": {
439
+ "precision": 0.1429,
440
+ "recall": 0.5,
441
+ "f1": 0.2222,
442
+ "support": 4,
443
+ "predicted": 14
444
+ },
445
+ "T1016": {
446
+ "precision": 0.2857,
447
+ "recall": 0.3636,
448
+ "f1": 0.32,
449
+ "support": 11,
450
+ "predicted": 14
451
+ },
452
+ "T1021.001": {
453
+ "precision": 0.5789,
454
+ "recall": 0.9167,
455
+ "f1": 0.7097,
456
+ "support": 12,
457
+ "predicted": 19
458
+ },
459
+ "T1027": {
460
+ "precision": 0.567,
461
+ "recall": 0.5392,
462
+ "f1": 0.5528,
463
+ "support": 102,
464
+ "predicted": 97
465
+ },
466
+ "T1033": {
467
+ "precision": 0.4375,
468
+ "recall": 0.5385,
469
+ "f1": 0.4828,
470
+ "support": 13,
471
+ "predicted": 16
472
+ },
473
+ "T1036.005": {
474
+ "precision": 0.0816,
475
+ "recall": 0.4,
476
+ "f1": 0.1356,
477
+ "support": 10,
478
+ "predicted": 49
479
+ },
480
+ "T1041": {
481
+ "precision": 0.3125,
482
+ "recall": 0.5882,
483
+ "f1": 0.4082,
484
+ "support": 17,
485
+ "predicted": 32
486
+ },
487
+ "T1047": {
488
+ "precision": 0.6923,
489
+ "recall": 0.6429,
490
+ "f1": 0.6667,
491
+ "support": 28,
492
+ "predicted": 26
493
+ },
494
+ "T1053.005": {
495
+ "precision": 0.7059,
496
+ "recall": 0.9231,
497
+ "f1": 0.8,
498
+ "support": 26,
499
+ "predicted": 34
500
+ },
501
+ "T1055": {
502
+ "precision": 0.6774,
503
+ "recall": 0.525,
504
+ "f1": 0.5915,
505
+ "support": 40,
506
+ "predicted": 31
507
+ },
508
+ "T1056.001": {
509
+ "precision": 0.9333,
510
+ "recall": 0.9333,
511
+ "f1": 0.9333,
512
+ "support": 15,
513
+ "predicted": 15
514
+ },
515
+ "T1057": {
516
+ "precision": 0.3182,
517
+ "recall": 0.4667,
518
+ "f1": 0.3784,
519
+ "support": 15,
520
+ "predicted": 22
521
+ },
522
+ "T1059.003": {
523
+ "precision": 0.3896,
524
+ "recall": 0.5882,
525
+ "f1": 0.4688,
526
+ "support": 51,
527
+ "predicted": 77
528
+ },
529
+ "T1068": {
530
+ "precision": 0.0,
531
+ "recall": 0.0,
532
+ "f1": 0.0,
533
+ "support": 3,
534
+ "predicted": 1
535
+ },
536
+ "T1070.004": {
537
+ "precision": 0.3333,
538
+ "recall": 0.4167,
539
+ "f1": 0.3704,
540
+ "support": 12,
541
+ "predicted": 15
542
+ },
543
+ "T1071.001": {
544
+ "precision": 0.4286,
545
+ "recall": 0.8571,
546
+ "f1": 0.5714,
547
+ "support": 21,
548
+ "predicted": 42
549
+ },
550
+ "T1072": {
551
+ "precision": 0.0,
552
+ "recall": 0.0,
553
+ "f1": 0.0,
554
+ "support": 3,
555
+ "predicted": 9
556
+ },
557
+ "T1074.001": {
558
+ "precision": 0.0526,
559
+ "recall": 0.6667,
560
+ "f1": 0.0976,
561
+ "support": 3,
562
+ "predicted": 38
563
+ },
564
+ "T1078": {
565
+ "precision": 0.2292,
566
+ "recall": 0.4783,
567
+ "f1": 0.3099,
568
+ "support": 23,
569
+ "predicted": 48
570
+ },
571
+ "T1082": {
572
+ "precision": 0.8333,
573
+ "recall": 0.2083,
574
+ "f1": 0.3333,
575
+ "support": 24,
576
+ "predicted": 6
577
+ },
578
+ "T1083": {
579
+ "precision": 0.4444,
580
+ "recall": 0.2667,
581
+ "f1": 0.3333,
582
+ "support": 15,
583
+ "predicted": 9
584
+ },
585
+ "T1090": {
586
+ "precision": 0.75,
587
+ "recall": 0.5217,
588
+ "f1": 0.6154,
589
+ "support": 23,
590
+ "predicted": 16
591
+ },
592
+ "T1095": {
593
+ "precision": 0.3333,
594
+ "recall": 0.3333,
595
+ "f1": 0.3333,
596
+ "support": 3,
597
+ "predicted": 3
598
+ },
599
+ "T1105": {
600
+ "precision": 0.5102,
601
+ "recall": 0.4386,
602
+ "f1": 0.4717,
603
+ "support": 57,
604
+ "predicted": 49
605
+ },
606
+ "T1106": {
607
+ "precision": 0.64,
608
+ "recall": 0.4706,
609
+ "f1": 0.5424,
610
+ "support": 34,
611
+ "predicted": 25
612
+ },
613
+ "T1110": {
614
+ "precision": 0.6842,
615
+ "recall": 0.6842,
616
+ "f1": 0.6842,
617
+ "support": 19,
618
+ "predicted": 19
619
+ },
620
+ "T1112": {
621
+ "precision": 0.2941,
622
+ "recall": 0.3571,
623
+ "f1": 0.3226,
624
+ "support": 14,
625
+ "predicted": 17
626
+ },
627
+ "T1113": {
628
+ "precision": 0.3478,
629
+ "recall": 0.8889,
630
+ "f1": 0.5,
631
+ "support": 9,
632
+ "predicted": 23
633
+ },
634
+ "T1140": {
635
+ "precision": 0.7308,
636
+ "recall": 0.8382,
637
+ "f1": 0.7808,
638
+ "support": 68,
639
+ "predicted": 78
640
+ },
641
+ "T1190": {
642
+ "precision": 0.1875,
643
+ "recall": 0.4,
644
+ "f1": 0.2553,
645
+ "support": 15,
646
+ "predicted": 32
647
+ },
648
+ "T1204.002": {
649
+ "precision": 0.3889,
650
+ "recall": 0.4667,
651
+ "f1": 0.4242,
652
+ "support": 15,
653
+ "predicted": 18
654
+ },
655
+ "T1210": {
656
+ "precision": 0.0377,
657
+ "recall": 1.0,
658
+ "f1": 0.0727,
659
+ "support": 2,
660
+ "predicted": 53
661
+ },
662
+ "T1218.011": {
663
+ "precision": 0.6364,
664
+ "recall": 0.5,
665
+ "f1": 0.56,
666
+ "support": 14,
667
+ "predicted": 11
668
+ },
669
+ "T1219": {
670
+ "precision": 0.4545,
671
+ "recall": 0.2632,
672
+ "f1": 0.3333,
673
+ "support": 19,
674
+ "predicted": 11
675
+ },
676
+ "T1484.001": {
677
+ "precision": 0.625,
678
+ "recall": 1.0,
679
+ "f1": 0.7692,
680
+ "support": 5,
681
+ "predicted": 8
682
+ },
683
+ "T1518.001": {
684
+ "precision": 0.2143,
685
+ "recall": 0.4286,
686
+ "f1": 0.2857,
687
+ "support": 7,
688
+ "predicted": 14
689
+ },
690
+ "T1543.003": {
691
+ "precision": 0.303,
692
+ "recall": 0.7692,
693
+ "f1": 0.4348,
694
+ "support": 13,
695
+ "predicted": 33
696
+ },
697
+ "T1547.001": {
698
+ "precision": 0.7059,
699
+ "recall": 0.75,
700
+ "f1": 0.7273,
701
+ "support": 16,
702
+ "predicted": 17
703
+ },
704
+ "T1548.002": {
705
+ "precision": 0.5,
706
+ "recall": 1.0,
707
+ "f1": 0.6667,
708
+ "support": 2,
709
+ "predicted": 4
710
+ },
711
+ "T1552.001": {
712
+ "precision": 0.0364,
713
+ "recall": 0.5,
714
+ "f1": 0.0678,
715
+ "support": 4,
716
+ "predicted": 55
717
+ },
718
+ "T1562.001": {
719
+ "precision": 0.9231,
720
+ "recall": 0.4615,
721
+ "f1": 0.6154,
722
+ "support": 26,
723
+ "predicted": 13
724
+ },
725
+ "T1564.001": {
726
+ "precision": 0.3333,
727
+ "recall": 0.6667,
728
+ "f1": 0.4444,
729
+ "support": 3,
730
+ "predicted": 6
731
+ },
732
+ "T1566.001": {
733
+ "precision": 0.2,
734
+ "recall": 0.4615,
735
+ "f1": 0.2791,
736
+ "support": 13,
737
+ "predicted": 30
738
+ },
739
+ "T1569.002": {
740
+ "precision": 0.5,
741
+ "recall": 0.6,
742
+ "f1": 0.5455,
743
+ "support": 5,
744
+ "predicted": 6
745
+ },
746
+ "T1570": {
747
+ "precision": 0.5,
748
+ "recall": 0.5,
749
+ "f1": 0.5,
750
+ "support": 6,
751
+ "predicted": 6
752
+ },
753
+ "T1573.001": {
754
+ "precision": 0.5,
755
+ "recall": 0.3125,
756
+ "f1": 0.3846,
757
+ "support": 16,
758
+ "predicted": 10
759
+ },
760
+ "T1574.002": {
761
+ "precision": 0.8333,
762
+ "recall": 0.3846,
763
+ "f1": 0.5263,
764
+ "support": 26,
765
+ "predicted": 12
766
+ }
767
+ }
768
+ }
769
+ }
results/modernbert__random.json ADDED
@@ -0,0 +1,769 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "run": "modernbert__random",
3
+ "base_model": "answerdotai/ModernBERT-base",
4
+ "split_scheme": "random",
5
+ "best_epoch": 5,
6
+ "dev_macro_f1": 0.5042,
7
+ "global_threshold": {
8
+ "threshold": 0.55,
9
+ "micro_f1": 0.557,
10
+ "macro_f1": 0.5045,
11
+ "micro_precision": 0.5038,
12
+ "micro_recall": 0.6228,
13
+ "macro_precision": 0.5067,
14
+ "macro_recall": 0.5544,
15
+ "n_examples": 2767,
16
+ "n_positive_labels": 737,
17
+ "per_class": {
18
+ "T1003.001": {
19
+ "precision": 0.5294,
20
+ "recall": 0.5294,
21
+ "f1": 0.5294,
22
+ "support": 17,
23
+ "predicted": 17
24
+ },
25
+ "T1005": {
26
+ "precision": 0.25,
27
+ "recall": 0.5,
28
+ "f1": 0.3333,
29
+ "support": 12,
30
+ "predicted": 24
31
+ },
32
+ "T1012": {
33
+ "precision": 0.4,
34
+ "recall": 0.5,
35
+ "f1": 0.4444,
36
+ "support": 4,
37
+ "predicted": 5
38
+ },
39
+ "T1016": {
40
+ "precision": 0.5,
41
+ "recall": 0.3333,
42
+ "f1": 0.4,
43
+ "support": 9,
44
+ "predicted": 6
45
+ },
46
+ "T1021.001": {
47
+ "precision": 0.5625,
48
+ "recall": 0.5625,
49
+ "f1": 0.5625,
50
+ "support": 16,
51
+ "predicted": 16
52
+ },
53
+ "T1027": {
54
+ "precision": 0.437,
55
+ "recall": 0.6146,
56
+ "f1": 0.5108,
57
+ "support": 96,
58
+ "predicted": 135
59
+ },
60
+ "T1033": {
61
+ "precision": 0.0,
62
+ "recall": 0.0,
63
+ "f1": 0.0,
64
+ "support": 6,
65
+ "predicted": 2
66
+ },
67
+ "T1036.005": {
68
+ "precision": 0.25,
69
+ "recall": 0.25,
70
+ "f1": 0.25,
71
+ "support": 8,
72
+ "predicted": 8
73
+ },
74
+ "T1041": {
75
+ "precision": 0.3158,
76
+ "recall": 0.4615,
77
+ "f1": 0.375,
78
+ "support": 13,
79
+ "predicted": 19
80
+ },
81
+ "T1047": {
82
+ "precision": 0.5556,
83
+ "recall": 0.5,
84
+ "f1": 0.5263,
85
+ "support": 10,
86
+ "predicted": 9
87
+ },
88
+ "T1053.005": {
89
+ "precision": 0.6471,
90
+ "recall": 0.7857,
91
+ "f1": 0.7097,
92
+ "support": 14,
93
+ "predicted": 17
94
+ },
95
+ "T1055": {
96
+ "precision": 0.5556,
97
+ "recall": 0.6977,
98
+ "f1": 0.6186,
99
+ "support": 43,
100
+ "predicted": 54
101
+ },
102
+ "T1056.001": {
103
+ "precision": 0.8571,
104
+ "recall": 1.0,
105
+ "f1": 0.9231,
106
+ "support": 6,
107
+ "predicted": 7
108
+ },
109
+ "T1057": {
110
+ "precision": 0.6364,
111
+ "recall": 0.5833,
112
+ "f1": 0.6087,
113
+ "support": 12,
114
+ "predicted": 11
115
+ },
116
+ "T1059.003": {
117
+ "precision": 0.4412,
118
+ "recall": 0.6667,
119
+ "f1": 0.531,
120
+ "support": 45,
121
+ "predicted": 68
122
+ },
123
+ "T1068": {
124
+ "precision": 0.0,
125
+ "recall": 0.0,
126
+ "f1": 0.0,
127
+ "support": 1,
128
+ "predicted": 0
129
+ },
130
+ "T1070.004": {
131
+ "precision": 0.3667,
132
+ "recall": 0.7333,
133
+ "f1": 0.4889,
134
+ "support": 15,
135
+ "predicted": 30
136
+ },
137
+ "T1071.001": {
138
+ "precision": 0.6333,
139
+ "recall": 0.6786,
140
+ "f1": 0.6552,
141
+ "support": 28,
142
+ "predicted": 30
143
+ },
144
+ "T1072": {
145
+ "precision": 0.0,
146
+ "recall": 0.0,
147
+ "f1": 0.0,
148
+ "support": 2,
149
+ "predicted": 1
150
+ },
151
+ "T1074.001": {
152
+ "precision": 0.5,
153
+ "recall": 0.3333,
154
+ "f1": 0.4,
155
+ "support": 3,
156
+ "predicted": 2
157
+ },
158
+ "T1078": {
159
+ "precision": 0.2812,
160
+ "recall": 0.4286,
161
+ "f1": 0.3396,
162
+ "support": 21,
163
+ "predicted": 32
164
+ },
165
+ "T1082": {
166
+ "precision": 0.4667,
167
+ "recall": 0.4118,
168
+ "f1": 0.4375,
169
+ "support": 17,
170
+ "predicted": 15
171
+ },
172
+ "T1083": {
173
+ "precision": 0.15,
174
+ "recall": 0.2308,
175
+ "f1": 0.1818,
176
+ "support": 13,
177
+ "predicted": 20
178
+ },
179
+ "T1090": {
180
+ "precision": 0.7059,
181
+ "recall": 0.6667,
182
+ "f1": 0.6857,
183
+ "support": 18,
184
+ "predicted": 17
185
+ },
186
+ "T1095": {
187
+ "precision": 0.5714,
188
+ "recall": 0.6667,
189
+ "f1": 0.6154,
190
+ "support": 12,
191
+ "predicted": 14
192
+ },
193
+ "T1105": {
194
+ "precision": 0.4545,
195
+ "recall": 0.625,
196
+ "f1": 0.5263,
197
+ "support": 40,
198
+ "predicted": 55
199
+ },
200
+ "T1106": {
201
+ "precision": 0.4444,
202
+ "recall": 0.7619,
203
+ "f1": 0.5614,
204
+ "support": 21,
205
+ "predicted": 36
206
+ },
207
+ "T1110": {
208
+ "precision": 0.75,
209
+ "recall": 1.0,
210
+ "f1": 0.8571,
211
+ "support": 3,
212
+ "predicted": 4
213
+ },
214
+ "T1112": {
215
+ "precision": 0.4242,
216
+ "recall": 0.9333,
217
+ "f1": 0.5833,
218
+ "support": 15,
219
+ "predicted": 33
220
+ },
221
+ "T1113": {
222
+ "precision": 0.75,
223
+ "recall": 0.75,
224
+ "f1": 0.75,
225
+ "support": 8,
226
+ "predicted": 8
227
+ },
228
+ "T1140": {
229
+ "precision": 0.7297,
230
+ "recall": 0.806,
231
+ "f1": 0.766,
232
+ "support": 67,
233
+ "predicted": 74
234
+ },
235
+ "T1190": {
236
+ "precision": 0.6,
237
+ "recall": 0.2308,
238
+ "f1": 0.3333,
239
+ "support": 13,
240
+ "predicted": 5
241
+ },
242
+ "T1204.002": {
243
+ "precision": 0.3333,
244
+ "recall": 0.4,
245
+ "f1": 0.3636,
246
+ "support": 10,
247
+ "predicted": 12
248
+ },
249
+ "T1210": {
250
+ "precision": 0.0,
251
+ "recall": 0.0,
252
+ "f1": 0.0,
253
+ "support": 2,
254
+ "predicted": 2
255
+ },
256
+ "T1218.011": {
257
+ "precision": 0.7778,
258
+ "recall": 0.6364,
259
+ "f1": 0.7,
260
+ "support": 11,
261
+ "predicted": 9
262
+ },
263
+ "T1219": {
264
+ "precision": 0.5714,
265
+ "recall": 1.0,
266
+ "f1": 0.7273,
267
+ "support": 4,
268
+ "predicted": 7
269
+ },
270
+ "T1484.001": {
271
+ "precision": 1.0,
272
+ "recall": 0.5,
273
+ "f1": 0.6667,
274
+ "support": 4,
275
+ "predicted": 2
276
+ },
277
+ "T1518.001": {
278
+ "precision": 1.0,
279
+ "recall": 0.25,
280
+ "f1": 0.4,
281
+ "support": 4,
282
+ "predicted": 1
283
+ },
284
+ "T1543.003": {
285
+ "precision": 0.5,
286
+ "recall": 0.75,
287
+ "f1": 0.6,
288
+ "support": 4,
289
+ "predicted": 6
290
+ },
291
+ "T1547.001": {
292
+ "precision": 0.75,
293
+ "recall": 0.6,
294
+ "f1": 0.6667,
295
+ "support": 10,
296
+ "predicted": 8
297
+ },
298
+ "T1548.002": {
299
+ "precision": 1.0,
300
+ "recall": 0.6667,
301
+ "f1": 0.8,
302
+ "support": 6,
303
+ "predicted": 4
304
+ },
305
+ "T1552.001": {
306
+ "precision": 0.0,
307
+ "recall": 0.0,
308
+ "f1": 0.0,
309
+ "support": 3,
310
+ "predicted": 0
311
+ },
312
+ "T1562.001": {
313
+ "precision": 0.6471,
314
+ "recall": 0.8462,
315
+ "f1": 0.7333,
316
+ "support": 13,
317
+ "predicted": 17
318
+ },
319
+ "T1564.001": {
320
+ "precision": 0.3333,
321
+ "recall": 1.0,
322
+ "f1": 0.5,
323
+ "support": 1,
324
+ "predicted": 3
325
+ },
326
+ "T1566.001": {
327
+ "precision": 0.619,
328
+ "recall": 0.8667,
329
+ "f1": 0.7222,
330
+ "support": 15,
331
+ "predicted": 21
332
+ },
333
+ "T1569.002": {
334
+ "precision": 0.5,
335
+ "recall": 0.5,
336
+ "f1": 0.5,
337
+ "support": 2,
338
+ "predicted": 2
339
+ },
340
+ "T1570": {
341
+ "precision": 0.8571,
342
+ "recall": 0.4615,
343
+ "f1": 0.6,
344
+ "support": 13,
345
+ "predicted": 7
346
+ },
347
+ "T1573.001": {
348
+ "precision": 0.3,
349
+ "recall": 0.6667,
350
+ "f1": 0.4138,
351
+ "support": 9,
352
+ "predicted": 20
353
+ },
354
+ "T1574.002": {
355
+ "precision": 0.875,
356
+ "recall": 0.7778,
357
+ "f1": 0.8235,
358
+ "support": 18,
359
+ "predicted": 16
360
+ }
361
+ }
362
+ },
363
+ "per_class_threshold": {
364
+ "thresholds": {
365
+ "T1003.001": 0.6,
366
+ "T1005": 0.35,
367
+ "T1012": 0.45,
368
+ "T1016": 0.2,
369
+ "T1021.001": 0.65,
370
+ "T1027": 0.65,
371
+ "T1033": 0.3,
372
+ "T1036.005": 0.9,
373
+ "T1041": 0.45,
374
+ "T1047": 0.3,
375
+ "T1053.005": 0.9,
376
+ "T1055": 0.75,
377
+ "T1056.001": 0.8,
378
+ "T1057": 0.5,
379
+ "T1059.003": 0.55,
380
+ "T1068": 0.05,
381
+ "T1070.004": 0.75,
382
+ "T1071.001": 0.85,
383
+ "T1072": 0.05,
384
+ "T1074.001": 0.15,
385
+ "T1078": 0.8,
386
+ "T1082": 0.25,
387
+ "T1083": 0.9,
388
+ "T1090": 0.55,
389
+ "T1095": 0.35,
390
+ "T1105": 0.65,
391
+ "T1106": 0.7,
392
+ "T1110": 0.55,
393
+ "T1112": 0.55,
394
+ "T1113": 0.7,
395
+ "T1140": 0.7,
396
+ "T1190": 0.2,
397
+ "T1204.002": 0.7,
398
+ "T1210": 0.15,
399
+ "T1218.011": 0.4,
400
+ "T1219": 0.5,
401
+ "T1484.001": 0.5,
402
+ "T1518.001": 0.85,
403
+ "T1543.003": 0.6,
404
+ "T1547.001": 0.8,
405
+ "T1548.002": 0.3,
406
+ "T1552.001": 0.5,
407
+ "T1562.001": 0.8,
408
+ "T1564.001": 0.45,
409
+ "T1566.001": 0.9,
410
+ "T1569.002": 0.25,
411
+ "T1570": 0.75,
412
+ "T1573.001": 0.6,
413
+ "T1574.002": 0.6
414
+ },
415
+ "micro_f1": 0.5399,
416
+ "macro_f1": 0.4822,
417
+ "micro_precision": 0.4994,
418
+ "micro_recall": 0.5875,
419
+ "macro_precision": 0.4959,
420
+ "macro_recall": 0.5294,
421
+ "n_examples": 2767,
422
+ "n_positive_labels": 737,
423
+ "per_class": {
424
+ "T1003.001": {
425
+ "precision": 0.5,
426
+ "recall": 0.4706,
427
+ "f1": 0.4848,
428
+ "support": 17,
429
+ "predicted": 16
430
+ },
431
+ "T1005": {
432
+ "precision": 0.2188,
433
+ "recall": 0.5833,
434
+ "f1": 0.3182,
435
+ "support": 12,
436
+ "predicted": 32
437
+ },
438
+ "T1012": {
439
+ "precision": 0.3333,
440
+ "recall": 0.5,
441
+ "f1": 0.4,
442
+ "support": 4,
443
+ "predicted": 6
444
+ },
445
+ "T1016": {
446
+ "precision": 0.4,
447
+ "recall": 0.4444,
448
+ "f1": 0.4211,
449
+ "support": 9,
450
+ "predicted": 10
451
+ },
452
+ "T1021.001": {
453
+ "precision": 0.6,
454
+ "recall": 0.5625,
455
+ "f1": 0.5806,
456
+ "support": 16,
457
+ "predicted": 15
458
+ },
459
+ "T1027": {
460
+ "precision": 0.4914,
461
+ "recall": 0.5938,
462
+ "f1": 0.5377,
463
+ "support": 96,
464
+ "predicted": 116
465
+ },
466
+ "T1033": {
467
+ "precision": 0.25,
468
+ "recall": 0.1667,
469
+ "f1": 0.2,
470
+ "support": 6,
471
+ "predicted": 4
472
+ },
473
+ "T1036.005": {
474
+ "precision": 0.0,
475
+ "recall": 0.0,
476
+ "f1": 0.0,
477
+ "support": 8,
478
+ "predicted": 1
479
+ },
480
+ "T1041": {
481
+ "precision": 0.2692,
482
+ "recall": 0.5385,
483
+ "f1": 0.359,
484
+ "support": 13,
485
+ "predicted": 26
486
+ },
487
+ "T1047": {
488
+ "precision": 0.5556,
489
+ "recall": 0.5,
490
+ "f1": 0.5263,
491
+ "support": 10,
492
+ "predicted": 9
493
+ },
494
+ "T1053.005": {
495
+ "precision": 0.7692,
496
+ "recall": 0.7143,
497
+ "f1": 0.7407,
498
+ "support": 14,
499
+ "predicted": 13
500
+ },
501
+ "T1055": {
502
+ "precision": 0.617,
503
+ "recall": 0.6744,
504
+ "f1": 0.6444,
505
+ "support": 43,
506
+ "predicted": 47
507
+ },
508
+ "T1056.001": {
509
+ "precision": 1.0,
510
+ "recall": 0.8333,
511
+ "f1": 0.9091,
512
+ "support": 6,
513
+ "predicted": 5
514
+ },
515
+ "T1057": {
516
+ "precision": 0.5,
517
+ "recall": 0.5833,
518
+ "f1": 0.5385,
519
+ "support": 12,
520
+ "predicted": 14
521
+ },
522
+ "T1059.003": {
523
+ "precision": 0.4412,
524
+ "recall": 0.6667,
525
+ "f1": 0.531,
526
+ "support": 45,
527
+ "predicted": 68
528
+ },
529
+ "T1068": {
530
+ "precision": 0.0,
531
+ "recall": 0.0,
532
+ "f1": 0.0,
533
+ "support": 1,
534
+ "predicted": 13
535
+ },
536
+ "T1070.004": {
537
+ "precision": 0.4737,
538
+ "recall": 0.6,
539
+ "f1": 0.5294,
540
+ "support": 15,
541
+ "predicted": 19
542
+ },
543
+ "T1071.001": {
544
+ "precision": 0.7368,
545
+ "recall": 0.5,
546
+ "f1": 0.5957,
547
+ "support": 28,
548
+ "predicted": 19
549
+ },
550
+ "T1072": {
551
+ "precision": 0.0,
552
+ "recall": 0.0,
553
+ "f1": 0.0,
554
+ "support": 2,
555
+ "predicted": 13
556
+ },
557
+ "T1074.001": {
558
+ "precision": 0.0588,
559
+ "recall": 0.3333,
560
+ "f1": 0.1,
561
+ "support": 3,
562
+ "predicted": 17
563
+ },
564
+ "T1078": {
565
+ "precision": 0.4375,
566
+ "recall": 0.3333,
567
+ "f1": 0.3784,
568
+ "support": 21,
569
+ "predicted": 16
570
+ },
571
+ "T1082": {
572
+ "precision": 0.3103,
573
+ "recall": 0.5294,
574
+ "f1": 0.3913,
575
+ "support": 17,
576
+ "predicted": 29
577
+ },
578
+ "T1083": {
579
+ "precision": 0.2,
580
+ "recall": 0.0769,
581
+ "f1": 0.1111,
582
+ "support": 13,
583
+ "predicted": 5
584
+ },
585
+ "T1090": {
586
+ "precision": 0.7059,
587
+ "recall": 0.6667,
588
+ "f1": 0.6857,
589
+ "support": 18,
590
+ "predicted": 17
591
+ },
592
+ "T1095": {
593
+ "precision": 0.5,
594
+ "recall": 0.75,
595
+ "f1": 0.6,
596
+ "support": 12,
597
+ "predicted": 18
598
+ },
599
+ "T1105": {
600
+ "precision": 0.5,
601
+ "recall": 0.625,
602
+ "f1": 0.5556,
603
+ "support": 40,
604
+ "predicted": 50
605
+ },
606
+ "T1106": {
607
+ "precision": 0.5185,
608
+ "recall": 0.6667,
609
+ "f1": 0.5833,
610
+ "support": 21,
611
+ "predicted": 27
612
+ },
613
+ "T1110": {
614
+ "precision": 0.75,
615
+ "recall": 1.0,
616
+ "f1": 0.8571,
617
+ "support": 3,
618
+ "predicted": 4
619
+ },
620
+ "T1112": {
621
+ "precision": 0.4242,
622
+ "recall": 0.9333,
623
+ "f1": 0.5833,
624
+ "support": 15,
625
+ "predicted": 33
626
+ },
627
+ "T1113": {
628
+ "precision": 1.0,
629
+ "recall": 0.75,
630
+ "f1": 0.8571,
631
+ "support": 8,
632
+ "predicted": 6
633
+ },
634
+ "T1140": {
635
+ "precision": 0.7941,
636
+ "recall": 0.806,
637
+ "f1": 0.8,
638
+ "support": 67,
639
+ "predicted": 68
640
+ },
641
+ "T1190": {
642
+ "precision": 0.3571,
643
+ "recall": 0.3846,
644
+ "f1": 0.3704,
645
+ "support": 13,
646
+ "predicted": 14
647
+ },
648
+ "T1204.002": {
649
+ "precision": 0.4444,
650
+ "recall": 0.4,
651
+ "f1": 0.4211,
652
+ "support": 10,
653
+ "predicted": 9
654
+ },
655
+ "T1210": {
656
+ "precision": 0.0,
657
+ "recall": 0.0,
658
+ "f1": 0.0,
659
+ "support": 2,
660
+ "predicted": 10
661
+ },
662
+ "T1218.011": {
663
+ "precision": 0.7,
664
+ "recall": 0.6364,
665
+ "f1": 0.6667,
666
+ "support": 11,
667
+ "predicted": 10
668
+ },
669
+ "T1219": {
670
+ "precision": 0.5,
671
+ "recall": 1.0,
672
+ "f1": 0.6667,
673
+ "support": 4,
674
+ "predicted": 8
675
+ },
676
+ "T1484.001": {
677
+ "precision": 1.0,
678
+ "recall": 0.5,
679
+ "f1": 0.6667,
680
+ "support": 4,
681
+ "predicted": 2
682
+ },
683
+ "T1518.001": {
684
+ "precision": 0.0,
685
+ "recall": 0.0,
686
+ "f1": 0.0,
687
+ "support": 4,
688
+ "predicted": 0
689
+ },
690
+ "T1543.003": {
691
+ "precision": 0.5,
692
+ "recall": 0.75,
693
+ "f1": 0.6,
694
+ "support": 4,
695
+ "predicted": 6
696
+ },
697
+ "T1547.001": {
698
+ "precision": 1.0,
699
+ "recall": 0.4,
700
+ "f1": 0.5714,
701
+ "support": 10,
702
+ "predicted": 4
703
+ },
704
+ "T1548.002": {
705
+ "precision": 1.0,
706
+ "recall": 0.6667,
707
+ "f1": 0.8,
708
+ "support": 6,
709
+ "predicted": 4
710
+ },
711
+ "T1552.001": {
712
+ "precision": 0.0,
713
+ "recall": 0.0,
714
+ "f1": 0.0,
715
+ "support": 3,
716
+ "predicted": 0
717
+ },
718
+ "T1562.001": {
719
+ "precision": 0.8,
720
+ "recall": 0.6154,
721
+ "f1": 0.6957,
722
+ "support": 13,
723
+ "predicted": 10
724
+ },
725
+ "T1564.001": {
726
+ "precision": 0.3333,
727
+ "recall": 1.0,
728
+ "f1": 0.5,
729
+ "support": 1,
730
+ "predicted": 3
731
+ },
732
+ "T1566.001": {
733
+ "precision": 0.7,
734
+ "recall": 0.4667,
735
+ "f1": 0.56,
736
+ "support": 15,
737
+ "predicted": 10
738
+ },
739
+ "T1569.002": {
740
+ "precision": 0.3333,
741
+ "recall": 1.0,
742
+ "f1": 0.5,
743
+ "support": 2,
744
+ "predicted": 6
745
+ },
746
+ "T1570": {
747
+ "precision": 1.0,
748
+ "recall": 0.3846,
749
+ "f1": 0.5556,
750
+ "support": 13,
751
+ "predicted": 5
752
+ },
753
+ "T1573.001": {
754
+ "precision": 0.3529,
755
+ "recall": 0.6667,
756
+ "f1": 0.4615,
757
+ "support": 9,
758
+ "predicted": 17
759
+ },
760
+ "T1574.002": {
761
+ "precision": 0.9231,
762
+ "recall": 0.6667,
763
+ "f1": 0.7742,
764
+ "support": 18,
765
+ "predicted": 13
766
+ }
767
+ }
768
+ }
769
+ }
results/securebert__document.json ADDED
@@ -0,0 +1,769 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "run": "securebert__document",
3
+ "base_model": "ehsanaghaei/SecureBERT",
4
+ "split_scheme": "document",
5
+ "best_epoch": 6,
6
+ "dev_macro_f1": 0.366,
7
+ "global_threshold": {
8
+ "threshold": 0.8,
9
+ "micro_f1": 0.4255,
10
+ "macro_f1": 0.3525,
11
+ "micro_precision": 0.3276,
12
+ "micro_recall": 0.607,
13
+ "macro_precision": 0.316,
14
+ "macro_recall": 0.4914,
15
+ "n_examples": 3635,
16
+ "n_positive_labels": 911,
17
+ "per_class": {
18
+ "T1003.001": {
19
+ "precision": 0.4242,
20
+ "recall": 0.6667,
21
+ "f1": 0.5185,
22
+ "support": 21,
23
+ "predicted": 33
24
+ },
25
+ "T1005": {
26
+ "precision": 0.093,
27
+ "recall": 0.5,
28
+ "f1": 0.1569,
29
+ "support": 8,
30
+ "predicted": 43
31
+ },
32
+ "T1012": {
33
+ "precision": 0.5,
34
+ "recall": 0.25,
35
+ "f1": 0.3333,
36
+ "support": 4,
37
+ "predicted": 2
38
+ },
39
+ "T1016": {
40
+ "precision": 0.4444,
41
+ "recall": 0.3636,
42
+ "f1": 0.4,
43
+ "support": 11,
44
+ "predicted": 9
45
+ },
46
+ "T1021.001": {
47
+ "precision": 0.3438,
48
+ "recall": 0.9167,
49
+ "f1": 0.5,
50
+ "support": 12,
51
+ "predicted": 32
52
+ },
53
+ "T1027": {
54
+ "precision": 0.3304,
55
+ "recall": 0.7451,
56
+ "f1": 0.4578,
57
+ "support": 102,
58
+ "predicted": 230
59
+ },
60
+ "T1033": {
61
+ "precision": 0.7778,
62
+ "recall": 0.5385,
63
+ "f1": 0.6364,
64
+ "support": 13,
65
+ "predicted": 9
66
+ },
67
+ "T1036.005": {
68
+ "precision": 0.1429,
69
+ "recall": 0.1,
70
+ "f1": 0.1176,
71
+ "support": 10,
72
+ "predicted": 7
73
+ },
74
+ "T1041": {
75
+ "precision": 0.2368,
76
+ "recall": 0.5294,
77
+ "f1": 0.3273,
78
+ "support": 17,
79
+ "predicted": 38
80
+ },
81
+ "T1047": {
82
+ "precision": 0.6875,
83
+ "recall": 0.3929,
84
+ "f1": 0.5,
85
+ "support": 28,
86
+ "predicted": 16
87
+ },
88
+ "T1053.005": {
89
+ "precision": 0.4364,
90
+ "recall": 0.9231,
91
+ "f1": 0.5926,
92
+ "support": 26,
93
+ "predicted": 55
94
+ },
95
+ "T1055": {
96
+ "precision": 0.3571,
97
+ "recall": 0.625,
98
+ "f1": 0.4545,
99
+ "support": 40,
100
+ "predicted": 70
101
+ },
102
+ "T1056.001": {
103
+ "precision": 0.6364,
104
+ "recall": 0.9333,
105
+ "f1": 0.7568,
106
+ "support": 15,
107
+ "predicted": 22
108
+ },
109
+ "T1057": {
110
+ "precision": 0.3704,
111
+ "recall": 0.6667,
112
+ "f1": 0.4762,
113
+ "support": 15,
114
+ "predicted": 27
115
+ },
116
+ "T1059.003": {
117
+ "precision": 0.2403,
118
+ "recall": 0.6078,
119
+ "f1": 0.3444,
120
+ "support": 51,
121
+ "predicted": 129
122
+ },
123
+ "T1068": {
124
+ "precision": 0.0,
125
+ "recall": 0.0,
126
+ "f1": 0.0,
127
+ "support": 3,
128
+ "predicted": 0
129
+ },
130
+ "T1070.004": {
131
+ "precision": 0.3,
132
+ "recall": 0.75,
133
+ "f1": 0.4286,
134
+ "support": 12,
135
+ "predicted": 30
136
+ },
137
+ "T1071.001": {
138
+ "precision": 0.2308,
139
+ "recall": 0.7143,
140
+ "f1": 0.3488,
141
+ "support": 21,
142
+ "predicted": 65
143
+ },
144
+ "T1072": {
145
+ "precision": 0.0,
146
+ "recall": 0.0,
147
+ "f1": 0.0,
148
+ "support": 3,
149
+ "predicted": 0
150
+ },
151
+ "T1074.001": {
152
+ "precision": 0.1667,
153
+ "recall": 0.3333,
154
+ "f1": 0.2222,
155
+ "support": 3,
156
+ "predicted": 6
157
+ },
158
+ "T1078": {
159
+ "precision": 0.1081,
160
+ "recall": 0.5217,
161
+ "f1": 0.1791,
162
+ "support": 23,
163
+ "predicted": 111
164
+ },
165
+ "T1082": {
166
+ "precision": 0.4483,
167
+ "recall": 0.5417,
168
+ "f1": 0.4906,
169
+ "support": 24,
170
+ "predicted": 29
171
+ },
172
+ "T1083": {
173
+ "precision": 0.2,
174
+ "recall": 0.4667,
175
+ "f1": 0.28,
176
+ "support": 15,
177
+ "predicted": 35
178
+ },
179
+ "T1090": {
180
+ "precision": 0.2857,
181
+ "recall": 0.6087,
182
+ "f1": 0.3889,
183
+ "support": 23,
184
+ "predicted": 49
185
+ },
186
+ "T1095": {
187
+ "precision": 0.0909,
188
+ "recall": 0.6667,
189
+ "f1": 0.16,
190
+ "support": 3,
191
+ "predicted": 22
192
+ },
193
+ "T1105": {
194
+ "precision": 0.3939,
195
+ "recall": 0.6842,
196
+ "f1": 0.5,
197
+ "support": 57,
198
+ "predicted": 99
199
+ },
200
+ "T1106": {
201
+ "precision": 0.3704,
202
+ "recall": 0.5882,
203
+ "f1": 0.4545,
204
+ "support": 34,
205
+ "predicted": 54
206
+ },
207
+ "T1110": {
208
+ "precision": 0.7,
209
+ "recall": 0.7368,
210
+ "f1": 0.7179,
211
+ "support": 19,
212
+ "predicted": 20
213
+ },
214
+ "T1112": {
215
+ "precision": 0.2449,
216
+ "recall": 0.8571,
217
+ "f1": 0.381,
218
+ "support": 14,
219
+ "predicted": 49
220
+ },
221
+ "T1113": {
222
+ "precision": 0.4211,
223
+ "recall": 0.8889,
224
+ "f1": 0.5714,
225
+ "support": 9,
226
+ "predicted": 19
227
+ },
228
+ "T1140": {
229
+ "precision": 0.446,
230
+ "recall": 0.9118,
231
+ "f1": 0.599,
232
+ "support": 68,
233
+ "predicted": 139
234
+ },
235
+ "T1190": {
236
+ "precision": 0.5,
237
+ "recall": 0.0667,
238
+ "f1": 0.1176,
239
+ "support": 15,
240
+ "predicted": 2
241
+ },
242
+ "T1204.002": {
243
+ "precision": 0.1522,
244
+ "recall": 0.4667,
245
+ "f1": 0.2295,
246
+ "support": 15,
247
+ "predicted": 46
248
+ },
249
+ "T1210": {
250
+ "precision": 0.0,
251
+ "recall": 0.0,
252
+ "f1": 0.0,
253
+ "support": 2,
254
+ "predicted": 1
255
+ },
256
+ "T1218.011": {
257
+ "precision": 0.6364,
258
+ "recall": 0.5,
259
+ "f1": 0.56,
260
+ "support": 14,
261
+ "predicted": 11
262
+ },
263
+ "T1219": {
264
+ "precision": 0.0,
265
+ "recall": 0.0,
266
+ "f1": 0.0,
267
+ "support": 19,
268
+ "predicted": 0
269
+ },
270
+ "T1484.001": {
271
+ "precision": 0.0,
272
+ "recall": 0.0,
273
+ "f1": 0.0,
274
+ "support": 5,
275
+ "predicted": 0
276
+ },
277
+ "T1518.001": {
278
+ "precision": 0.3333,
279
+ "recall": 0.2857,
280
+ "f1": 0.3077,
281
+ "support": 7,
282
+ "predicted": 6
283
+ },
284
+ "T1543.003": {
285
+ "precision": 0.7,
286
+ "recall": 0.5385,
287
+ "f1": 0.6087,
288
+ "support": 13,
289
+ "predicted": 10
290
+ },
291
+ "T1547.001": {
292
+ "precision": 0.2581,
293
+ "recall": 0.5,
294
+ "f1": 0.3404,
295
+ "support": 16,
296
+ "predicted": 31
297
+ },
298
+ "T1548.002": {
299
+ "precision": 0.4,
300
+ "recall": 1.0,
301
+ "f1": 0.5714,
302
+ "support": 2,
303
+ "predicted": 5
304
+ },
305
+ "T1552.001": {
306
+ "precision": 0.0,
307
+ "recall": 0.0,
308
+ "f1": 0.0,
309
+ "support": 4,
310
+ "predicted": 4
311
+ },
312
+ "T1562.001": {
313
+ "precision": 0.4444,
314
+ "recall": 0.4615,
315
+ "f1": 0.4528,
316
+ "support": 26,
317
+ "predicted": 27
318
+ },
319
+ "T1564.001": {
320
+ "precision": 0.0,
321
+ "recall": 0.0,
322
+ "f1": 0.0,
323
+ "support": 3,
324
+ "predicted": 0
325
+ },
326
+ "T1566.001": {
327
+ "precision": 0.1667,
328
+ "recall": 0.4615,
329
+ "f1": 0.2449,
330
+ "support": 13,
331
+ "predicted": 36
332
+ },
333
+ "T1569.002": {
334
+ "precision": 0.0,
335
+ "recall": 0.0,
336
+ "f1": 0.0,
337
+ "support": 5,
338
+ "predicted": 2
339
+ },
340
+ "T1570": {
341
+ "precision": 0.1875,
342
+ "recall": 0.5,
343
+ "f1": 0.2727,
344
+ "support": 6,
345
+ "predicted": 16
346
+ },
347
+ "T1573.001": {
348
+ "precision": 0.4444,
349
+ "recall": 0.5,
350
+ "f1": 0.4706,
351
+ "support": 16,
352
+ "predicted": 18
353
+ },
354
+ "T1574.002": {
355
+ "precision": 0.8333,
356
+ "recall": 0.7692,
357
+ "f1": 0.8,
358
+ "support": 26,
359
+ "predicted": 24
360
+ }
361
+ }
362
+ },
363
+ "per_class_threshold": {
364
+ "thresholds": {
365
+ "T1003.001": 0.9,
366
+ "T1005": 0.8,
367
+ "T1012": 0.8,
368
+ "T1016": 0.75,
369
+ "T1021.001": 0.9,
370
+ "T1027": 0.7,
371
+ "T1033": 0.65,
372
+ "T1036.005": 0.85,
373
+ "T1041": 0.85,
374
+ "T1047": 0.85,
375
+ "T1053.005": 0.9,
376
+ "T1055": 0.95,
377
+ "T1056.001": 0.9,
378
+ "T1057": 0.8,
379
+ "T1059.003": 0.8,
380
+ "T1068": 0.05,
381
+ "T1070.004": 0.8,
382
+ "T1071.001": 0.95,
383
+ "T1072": 0.25,
384
+ "T1074.001": 0.25,
385
+ "T1078": 0.7,
386
+ "T1082": 0.9,
387
+ "T1083": 0.9,
388
+ "T1090": 0.9,
389
+ "T1095": 0.9,
390
+ "T1105": 0.9,
391
+ "T1106": 0.95,
392
+ "T1110": 0.75,
393
+ "T1112": 0.9,
394
+ "T1113": 0.85,
395
+ "T1140": 0.95,
396
+ "T1190": 0.75,
397
+ "T1204.002": 0.85,
398
+ "T1210": 0.15,
399
+ "T1218.011": 0.8,
400
+ "T1219": 0.55,
401
+ "T1484.001": 0.45,
402
+ "T1518.001": 0.75,
403
+ "T1543.003": 0.6,
404
+ "T1547.001": 0.85,
405
+ "T1548.002": 0.7,
406
+ "T1552.001": 0.05,
407
+ "T1562.001": 0.85,
408
+ "T1564.001": 0.35,
409
+ "T1566.001": 0.95,
410
+ "T1569.002": 0.55,
411
+ "T1570": 0.9,
412
+ "T1573.001": 0.85,
413
+ "T1574.002": 0.85
414
+ },
415
+ "micro_f1": 0.2517,
416
+ "macro_f1": 0.3691,
417
+ "micro_precision": 0.163,
418
+ "micro_recall": 0.5521,
419
+ "macro_precision": 0.3664,
420
+ "macro_recall": 0.5132,
421
+ "n_examples": 3635,
422
+ "n_positive_labels": 911,
423
+ "per_class": {
424
+ "T1003.001": {
425
+ "precision": 0.6364,
426
+ "recall": 0.6667,
427
+ "f1": 0.6512,
428
+ "support": 21,
429
+ "predicted": 22
430
+ },
431
+ "T1005": {
432
+ "precision": 0.093,
433
+ "recall": 0.5,
434
+ "f1": 0.1569,
435
+ "support": 8,
436
+ "predicted": 43
437
+ },
438
+ "T1012": {
439
+ "precision": 0.5,
440
+ "recall": 0.25,
441
+ "f1": 0.3333,
442
+ "support": 4,
443
+ "predicted": 2
444
+ },
445
+ "T1016": {
446
+ "precision": 0.4615,
447
+ "recall": 0.5455,
448
+ "f1": 0.5,
449
+ "support": 11,
450
+ "predicted": 13
451
+ },
452
+ "T1021.001": {
453
+ "precision": 0.4348,
454
+ "recall": 0.8333,
455
+ "f1": 0.5714,
456
+ "support": 12,
457
+ "predicted": 23
458
+ },
459
+ "T1027": {
460
+ "precision": 0.2927,
461
+ "recall": 0.8235,
462
+ "f1": 0.4319,
463
+ "support": 102,
464
+ "predicted": 287
465
+ },
466
+ "T1033": {
467
+ "precision": 0.4348,
468
+ "recall": 0.7692,
469
+ "f1": 0.5556,
470
+ "support": 13,
471
+ "predicted": 23
472
+ },
473
+ "T1036.005": {
474
+ "precision": 0.0,
475
+ "recall": 0.0,
476
+ "f1": 0.0,
477
+ "support": 10,
478
+ "predicted": 3
479
+ },
480
+ "T1041": {
481
+ "precision": 0.28,
482
+ "recall": 0.4118,
483
+ "f1": 0.3333,
484
+ "support": 17,
485
+ "predicted": 25
486
+ },
487
+ "T1047": {
488
+ "precision": 0.7778,
489
+ "recall": 0.25,
490
+ "f1": 0.3784,
491
+ "support": 28,
492
+ "predicted": 9
493
+ },
494
+ "T1053.005": {
495
+ "precision": 0.7857,
496
+ "recall": 0.8462,
497
+ "f1": 0.8148,
498
+ "support": 26,
499
+ "predicted": 28
500
+ },
501
+ "T1055": {
502
+ "precision": 0.75,
503
+ "recall": 0.45,
504
+ "f1": 0.5625,
505
+ "support": 40,
506
+ "predicted": 24
507
+ },
508
+ "T1056.001": {
509
+ "precision": 0.7692,
510
+ "recall": 0.6667,
511
+ "f1": 0.7143,
512
+ "support": 15,
513
+ "predicted": 13
514
+ },
515
+ "T1057": {
516
+ "precision": 0.3704,
517
+ "recall": 0.6667,
518
+ "f1": 0.4762,
519
+ "support": 15,
520
+ "predicted": 27
521
+ },
522
+ "T1059.003": {
523
+ "precision": 0.2403,
524
+ "recall": 0.6078,
525
+ "f1": 0.3444,
526
+ "support": 51,
527
+ "predicted": 129
528
+ },
529
+ "T1068": {
530
+ "precision": 0.0039,
531
+ "recall": 0.6667,
532
+ "f1": 0.0077,
533
+ "support": 3,
534
+ "predicted": 517
535
+ },
536
+ "T1070.004": {
537
+ "precision": 0.3,
538
+ "recall": 0.75,
539
+ "f1": 0.4286,
540
+ "support": 12,
541
+ "predicted": 30
542
+ },
543
+ "T1071.001": {
544
+ "precision": 0.4583,
545
+ "recall": 0.5238,
546
+ "f1": 0.4889,
547
+ "support": 21,
548
+ "predicted": 24
549
+ },
550
+ "T1072": {
551
+ "precision": 0.0,
552
+ "recall": 0.0,
553
+ "f1": 0.0,
554
+ "support": 3,
555
+ "predicted": 75
556
+ },
557
+ "T1074.001": {
558
+ "precision": 0.0049,
559
+ "recall": 0.3333,
560
+ "f1": 0.0096,
561
+ "support": 3,
562
+ "predicted": 205
563
+ },
564
+ "T1078": {
565
+ "precision": 0.0816,
566
+ "recall": 0.5217,
567
+ "f1": 0.1412,
568
+ "support": 23,
569
+ "predicted": 147
570
+ },
571
+ "T1082": {
572
+ "precision": 0.6,
573
+ "recall": 0.5,
574
+ "f1": 0.5455,
575
+ "support": 24,
576
+ "predicted": 20
577
+ },
578
+ "T1083": {
579
+ "precision": 0.3125,
580
+ "recall": 0.3333,
581
+ "f1": 0.3226,
582
+ "support": 15,
583
+ "predicted": 16
584
+ },
585
+ "T1090": {
586
+ "precision": 0.4483,
587
+ "recall": 0.5652,
588
+ "f1": 0.5,
589
+ "support": 23,
590
+ "predicted": 29
591
+ },
592
+ "T1095": {
593
+ "precision": 0.3333,
594
+ "recall": 0.3333,
595
+ "f1": 0.3333,
596
+ "support": 3,
597
+ "predicted": 3
598
+ },
599
+ "T1105": {
600
+ "precision": 0.5455,
601
+ "recall": 0.5263,
602
+ "f1": 0.5357,
603
+ "support": 57,
604
+ "predicted": 55
605
+ },
606
+ "T1106": {
607
+ "precision": 0.6316,
608
+ "recall": 0.3529,
609
+ "f1": 0.4528,
610
+ "support": 34,
611
+ "predicted": 19
612
+ },
613
+ "T1110": {
614
+ "precision": 0.6,
615
+ "recall": 0.7895,
616
+ "f1": 0.6818,
617
+ "support": 19,
618
+ "predicted": 25
619
+ },
620
+ "T1112": {
621
+ "precision": 0.2581,
622
+ "recall": 0.5714,
623
+ "f1": 0.3556,
624
+ "support": 14,
625
+ "predicted": 31
626
+ },
627
+ "T1113": {
628
+ "precision": 0.4118,
629
+ "recall": 0.7778,
630
+ "f1": 0.5385,
631
+ "support": 9,
632
+ "predicted": 17
633
+ },
634
+ "T1140": {
635
+ "precision": 0.6438,
636
+ "recall": 0.6912,
637
+ "f1": 0.6667,
638
+ "support": 68,
639
+ "predicted": 73
640
+ },
641
+ "T1190": {
642
+ "precision": 0.3333,
643
+ "recall": 0.0667,
644
+ "f1": 0.1111,
645
+ "support": 15,
646
+ "predicted": 3
647
+ },
648
+ "T1204.002": {
649
+ "precision": 0.1429,
650
+ "recall": 0.2667,
651
+ "f1": 0.186,
652
+ "support": 15,
653
+ "predicted": 28
654
+ },
655
+ "T1210": {
656
+ "precision": 0.009,
657
+ "recall": 1.0,
658
+ "f1": 0.0179,
659
+ "support": 2,
660
+ "predicted": 222
661
+ },
662
+ "T1218.011": {
663
+ "precision": 0.6364,
664
+ "recall": 0.5,
665
+ "f1": 0.56,
666
+ "support": 14,
667
+ "predicted": 11
668
+ },
669
+ "T1219": {
670
+ "precision": 0.0,
671
+ "recall": 0.0,
672
+ "f1": 0.0,
673
+ "support": 19,
674
+ "predicted": 9
675
+ },
676
+ "T1484.001": {
677
+ "precision": 0.1379,
678
+ "recall": 0.8,
679
+ "f1": 0.2353,
680
+ "support": 5,
681
+ "predicted": 29
682
+ },
683
+ "T1518.001": {
684
+ "precision": 0.2,
685
+ "recall": 0.2857,
686
+ "f1": 0.2353,
687
+ "support": 7,
688
+ "predicted": 10
689
+ },
690
+ "T1543.003": {
691
+ "precision": 0.2917,
692
+ "recall": 0.5385,
693
+ "f1": 0.3784,
694
+ "support": 13,
695
+ "predicted": 24
696
+ },
697
+ "T1547.001": {
698
+ "precision": 0.4,
699
+ "recall": 0.375,
700
+ "f1": 0.3871,
701
+ "support": 16,
702
+ "predicted": 15
703
+ },
704
+ "T1548.002": {
705
+ "precision": 0.1667,
706
+ "recall": 1.0,
707
+ "f1": 0.2857,
708
+ "support": 2,
709
+ "predicted": 12
710
+ },
711
+ "T1552.001": {
712
+ "precision": 0.0047,
713
+ "recall": 0.75,
714
+ "f1": 0.0093,
715
+ "support": 4,
716
+ "predicted": 641
717
+ },
718
+ "T1562.001": {
719
+ "precision": 0.6111,
720
+ "recall": 0.4231,
721
+ "f1": 0.5,
722
+ "support": 26,
723
+ "predicted": 18
724
+ },
725
+ "T1564.001": {
726
+ "precision": 0.0217,
727
+ "recall": 0.3333,
728
+ "f1": 0.0408,
729
+ "support": 3,
730
+ "predicted": 46
731
+ },
732
+ "T1566.001": {
733
+ "precision": 0.2857,
734
+ "recall": 0.1538,
735
+ "f1": 0.2,
736
+ "support": 13,
737
+ "predicted": 7
738
+ },
739
+ "T1569.002": {
740
+ "precision": 0.1333,
741
+ "recall": 0.4,
742
+ "f1": 0.2,
743
+ "support": 5,
744
+ "predicted": 15
745
+ },
746
+ "T1570": {
747
+ "precision": 0.6,
748
+ "recall": 0.5,
749
+ "f1": 0.5455,
750
+ "support": 6,
751
+ "predicted": 5
752
+ },
753
+ "T1573.001": {
754
+ "precision": 0.6154,
755
+ "recall": 0.5,
756
+ "f1": 0.5517,
757
+ "support": 16,
758
+ "predicted": 13
759
+ },
760
+ "T1574.002": {
761
+ "precision": 0.9048,
762
+ "recall": 0.7308,
763
+ "f1": 0.8085,
764
+ "support": 26,
765
+ "predicted": 21
766
+ }
767
+ }
768
+ }
769
+ }
results/securebert__random.json ADDED
@@ -0,0 +1,769 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "run": "securebert__random",
3
+ "base_model": "ehsanaghaei/SecureBERT",
4
+ "split_scheme": "random",
5
+ "best_epoch": 6,
6
+ "dev_macro_f1": 0.417,
7
+ "global_threshold": {
8
+ "threshold": 0.85,
9
+ "micro_f1": 0.4702,
10
+ "macro_f1": 0.3877,
11
+ "micro_precision": 0.3913,
12
+ "micro_recall": 0.5889,
13
+ "macro_precision": 0.3928,
14
+ "macro_recall": 0.4509,
15
+ "n_examples": 2767,
16
+ "n_positive_labels": 737,
17
+ "per_class": {
18
+ "T1003.001": {
19
+ "precision": 0.6,
20
+ "recall": 0.7059,
21
+ "f1": 0.6486,
22
+ "support": 17,
23
+ "predicted": 20
24
+ },
25
+ "T1005": {
26
+ "precision": 0.4,
27
+ "recall": 0.1667,
28
+ "f1": 0.2353,
29
+ "support": 12,
30
+ "predicted": 5
31
+ },
32
+ "T1012": {
33
+ "precision": 0.0,
34
+ "recall": 0.0,
35
+ "f1": 0.0,
36
+ "support": 4,
37
+ "predicted": 1
38
+ },
39
+ "T1016": {
40
+ "precision": 0.5,
41
+ "recall": 0.1111,
42
+ "f1": 0.1818,
43
+ "support": 9,
44
+ "predicted": 2
45
+ },
46
+ "T1021.001": {
47
+ "precision": 0.5263,
48
+ "recall": 0.625,
49
+ "f1": 0.5714,
50
+ "support": 16,
51
+ "predicted": 19
52
+ },
53
+ "T1027": {
54
+ "precision": 0.3163,
55
+ "recall": 0.6458,
56
+ "f1": 0.4247,
57
+ "support": 96,
58
+ "predicted": 196
59
+ },
60
+ "T1033": {
61
+ "precision": 0.1111,
62
+ "recall": 0.1667,
63
+ "f1": 0.1333,
64
+ "support": 6,
65
+ "predicted": 9
66
+ },
67
+ "T1036.005": {
68
+ "precision": 0.3333,
69
+ "recall": 0.125,
70
+ "f1": 0.1818,
71
+ "support": 8,
72
+ "predicted": 3
73
+ },
74
+ "T1041": {
75
+ "precision": 0.2632,
76
+ "recall": 0.3846,
77
+ "f1": 0.3125,
78
+ "support": 13,
79
+ "predicted": 19
80
+ },
81
+ "T1047": {
82
+ "precision": 0.4,
83
+ "recall": 0.4,
84
+ "f1": 0.4,
85
+ "support": 10,
86
+ "predicted": 10
87
+ },
88
+ "T1053.005": {
89
+ "precision": 0.5625,
90
+ "recall": 0.6429,
91
+ "f1": 0.6,
92
+ "support": 14,
93
+ "predicted": 16
94
+ },
95
+ "T1055": {
96
+ "precision": 0.4688,
97
+ "recall": 0.6977,
98
+ "f1": 0.5607,
99
+ "support": 43,
100
+ "predicted": 64
101
+ },
102
+ "T1056.001": {
103
+ "precision": 0.4,
104
+ "recall": 0.6667,
105
+ "f1": 0.5,
106
+ "support": 6,
107
+ "predicted": 10
108
+ },
109
+ "T1057": {
110
+ "precision": 0.75,
111
+ "recall": 0.5,
112
+ "f1": 0.6,
113
+ "support": 12,
114
+ "predicted": 8
115
+ },
116
+ "T1059.003": {
117
+ "precision": 0.3836,
118
+ "recall": 0.6222,
119
+ "f1": 0.4746,
120
+ "support": 45,
121
+ "predicted": 73
122
+ },
123
+ "T1068": {
124
+ "precision": 0.0,
125
+ "recall": 0.0,
126
+ "f1": 0.0,
127
+ "support": 1,
128
+ "predicted": 0
129
+ },
130
+ "T1070.004": {
131
+ "precision": 0.5714,
132
+ "recall": 0.8,
133
+ "f1": 0.6667,
134
+ "support": 15,
135
+ "predicted": 21
136
+ },
137
+ "T1071.001": {
138
+ "precision": 0.3636,
139
+ "recall": 0.5714,
140
+ "f1": 0.4444,
141
+ "support": 28,
142
+ "predicted": 44
143
+ },
144
+ "T1072": {
145
+ "precision": 0.0,
146
+ "recall": 0.0,
147
+ "f1": 0.0,
148
+ "support": 2,
149
+ "predicted": 0
150
+ },
151
+ "T1074.001": {
152
+ "precision": 0.0,
153
+ "recall": 0.0,
154
+ "f1": 0.0,
155
+ "support": 3,
156
+ "predicted": 0
157
+ },
158
+ "T1078": {
159
+ "precision": 0.2273,
160
+ "recall": 0.4762,
161
+ "f1": 0.3077,
162
+ "support": 21,
163
+ "predicted": 44
164
+ },
165
+ "T1082": {
166
+ "precision": 0.3913,
167
+ "recall": 0.5294,
168
+ "f1": 0.45,
169
+ "support": 17,
170
+ "predicted": 23
171
+ },
172
+ "T1083": {
173
+ "precision": 0.2174,
174
+ "recall": 0.3846,
175
+ "f1": 0.2778,
176
+ "support": 13,
177
+ "predicted": 23
178
+ },
179
+ "T1090": {
180
+ "precision": 0.2439,
181
+ "recall": 0.5556,
182
+ "f1": 0.339,
183
+ "support": 18,
184
+ "predicted": 41
185
+ },
186
+ "T1095": {
187
+ "precision": 0.5,
188
+ "recall": 0.5833,
189
+ "f1": 0.5385,
190
+ "support": 12,
191
+ "predicted": 14
192
+ },
193
+ "T1105": {
194
+ "precision": 0.3898,
195
+ "recall": 0.575,
196
+ "f1": 0.4646,
197
+ "support": 40,
198
+ "predicted": 59
199
+ },
200
+ "T1106": {
201
+ "precision": 0.4048,
202
+ "recall": 0.8095,
203
+ "f1": 0.5397,
204
+ "support": 21,
205
+ "predicted": 42
206
+ },
207
+ "T1110": {
208
+ "precision": 0.6,
209
+ "recall": 1.0,
210
+ "f1": 0.75,
211
+ "support": 3,
212
+ "predicted": 5
213
+ },
214
+ "T1112": {
215
+ "precision": 0.2927,
216
+ "recall": 0.8,
217
+ "f1": 0.4286,
218
+ "support": 15,
219
+ "predicted": 41
220
+ },
221
+ "T1113": {
222
+ "precision": 0.7143,
223
+ "recall": 0.625,
224
+ "f1": 0.6667,
225
+ "support": 8,
226
+ "predicted": 7
227
+ },
228
+ "T1140": {
229
+ "precision": 0.4357,
230
+ "recall": 0.9104,
231
+ "f1": 0.5894,
232
+ "support": 67,
233
+ "predicted": 140
234
+ },
235
+ "T1190": {
236
+ "precision": 0.0,
237
+ "recall": 0.0,
238
+ "f1": 0.0,
239
+ "support": 13,
240
+ "predicted": 1
241
+ },
242
+ "T1204.002": {
243
+ "precision": 0.1935,
244
+ "recall": 0.6,
245
+ "f1": 0.2927,
246
+ "support": 10,
247
+ "predicted": 31
248
+ },
249
+ "T1210": {
250
+ "precision": 0.0,
251
+ "recall": 0.0,
252
+ "f1": 0.0,
253
+ "support": 2,
254
+ "predicted": 0
255
+ },
256
+ "T1218.011": {
257
+ "precision": 0.6364,
258
+ "recall": 0.6364,
259
+ "f1": 0.6364,
260
+ "support": 11,
261
+ "predicted": 11
262
+ },
263
+ "T1219": {
264
+ "precision": 0.4,
265
+ "recall": 0.5,
266
+ "f1": 0.4444,
267
+ "support": 4,
268
+ "predicted": 5
269
+ },
270
+ "T1484.001": {
271
+ "precision": 0.0,
272
+ "recall": 0.0,
273
+ "f1": 0.0,
274
+ "support": 4,
275
+ "predicted": 0
276
+ },
277
+ "T1518.001": {
278
+ "precision": 1.0,
279
+ "recall": 0.25,
280
+ "f1": 0.4,
281
+ "support": 4,
282
+ "predicted": 1
283
+ },
284
+ "T1543.003": {
285
+ "precision": 0.6,
286
+ "recall": 0.75,
287
+ "f1": 0.6667,
288
+ "support": 4,
289
+ "predicted": 5
290
+ },
291
+ "T1547.001": {
292
+ "precision": 0.5,
293
+ "recall": 0.4,
294
+ "f1": 0.4444,
295
+ "support": 10,
296
+ "predicted": 8
297
+ },
298
+ "T1548.002": {
299
+ "precision": 1.0,
300
+ "recall": 0.3333,
301
+ "f1": 0.5,
302
+ "support": 6,
303
+ "predicted": 2
304
+ },
305
+ "T1552.001": {
306
+ "precision": 0.0,
307
+ "recall": 0.0,
308
+ "f1": 0.0,
309
+ "support": 3,
310
+ "predicted": 0
311
+ },
312
+ "T1562.001": {
313
+ "precision": 0.5238,
314
+ "recall": 0.8462,
315
+ "f1": 0.6471,
316
+ "support": 13,
317
+ "predicted": 21
318
+ },
319
+ "T1564.001": {
320
+ "precision": 0.0,
321
+ "recall": 0.0,
322
+ "f1": 0.0,
323
+ "support": 1,
324
+ "predicted": 0
325
+ },
326
+ "T1566.001": {
327
+ "precision": 0.3871,
328
+ "recall": 0.8,
329
+ "f1": 0.5217,
330
+ "support": 15,
331
+ "predicted": 31
332
+ },
333
+ "T1569.002": {
334
+ "precision": 1.0,
335
+ "recall": 0.5,
336
+ "f1": 0.6667,
337
+ "support": 2,
338
+ "predicted": 1
339
+ },
340
+ "T1570": {
341
+ "precision": 0.4286,
342
+ "recall": 0.2308,
343
+ "f1": 0.3,
344
+ "support": 13,
345
+ "predicted": 7
346
+ },
347
+ "T1573.001": {
348
+ "precision": 0.4444,
349
+ "recall": 0.4444,
350
+ "f1": 0.4444,
351
+ "support": 9,
352
+ "predicted": 9
353
+ },
354
+ "T1574.002": {
355
+ "precision": 0.7647,
356
+ "recall": 0.7222,
357
+ "f1": 0.7429,
358
+ "support": 18,
359
+ "predicted": 17
360
+ }
361
+ }
362
+ },
363
+ "per_class_threshold": {
364
+ "thresholds": {
365
+ "T1003.001": 0.9,
366
+ "T1005": 0.85,
367
+ "T1012": 0.8,
368
+ "T1016": 0.8,
369
+ "T1021.001": 0.9,
370
+ "T1027": 0.8,
371
+ "T1033": 0.85,
372
+ "T1036.005": 0.85,
373
+ "T1041": 0.8,
374
+ "T1047": 0.9,
375
+ "T1053.005": 0.9,
376
+ "T1055": 0.9,
377
+ "T1056.001": 0.85,
378
+ "T1057": 0.65,
379
+ "T1059.003": 0.95,
380
+ "T1068": 0.6,
381
+ "T1070.004": 0.9,
382
+ "T1071.001": 0.9,
383
+ "T1072": 0.05,
384
+ "T1074.001": 0.35,
385
+ "T1078": 0.9,
386
+ "T1082": 0.85,
387
+ "T1083": 0.9,
388
+ "T1090": 0.9,
389
+ "T1095": 0.8,
390
+ "T1105": 0.8,
391
+ "T1106": 0.9,
392
+ "T1110": 0.85,
393
+ "T1112": 0.9,
394
+ "T1113": 0.9,
395
+ "T1140": 0.95,
396
+ "T1190": 0.75,
397
+ "T1204.002": 0.85,
398
+ "T1210": 0.65,
399
+ "T1218.011": 0.85,
400
+ "T1219": 0.85,
401
+ "T1484.001": 0.75,
402
+ "T1518.001": 0.85,
403
+ "T1543.003": 0.75,
404
+ "T1547.001": 0.85,
405
+ "T1548.002": 0.75,
406
+ "T1552.001": 0.7,
407
+ "T1562.001": 0.85,
408
+ "T1564.001": 0.45,
409
+ "T1566.001": 0.95,
410
+ "T1569.002": 0.65,
411
+ "T1570": 0.85,
412
+ "T1573.001": 0.8,
413
+ "T1574.002": 0.85
414
+ },
415
+ "micro_f1": 0.3729,
416
+ "macro_f1": 0.4147,
417
+ "micro_precision": 0.2787,
418
+ "micro_recall": 0.5631,
419
+ "macro_precision": 0.4306,
420
+ "macro_recall": 0.5064,
421
+ "n_examples": 2767,
422
+ "n_positive_labels": 737,
423
+ "per_class": {
424
+ "T1003.001": {
425
+ "precision": 0.6667,
426
+ "recall": 0.3529,
427
+ "f1": 0.4615,
428
+ "support": 17,
429
+ "predicted": 9
430
+ },
431
+ "T1005": {
432
+ "precision": 0.4,
433
+ "recall": 0.1667,
434
+ "f1": 0.2353,
435
+ "support": 12,
436
+ "predicted": 5
437
+ },
438
+ "T1012": {
439
+ "precision": 0.3333,
440
+ "recall": 0.25,
441
+ "f1": 0.2857,
442
+ "support": 4,
443
+ "predicted": 3
444
+ },
445
+ "T1016": {
446
+ "precision": 0.4444,
447
+ "recall": 0.4444,
448
+ "f1": 0.4444,
449
+ "support": 9,
450
+ "predicted": 9
451
+ },
452
+ "T1021.001": {
453
+ "precision": 0.5,
454
+ "recall": 0.4375,
455
+ "f1": 0.4667,
456
+ "support": 16,
457
+ "predicted": 14
458
+ },
459
+ "T1027": {
460
+ "precision": 0.2969,
461
+ "recall": 0.7083,
462
+ "f1": 0.4185,
463
+ "support": 96,
464
+ "predicted": 229
465
+ },
466
+ "T1033": {
467
+ "precision": 0.1111,
468
+ "recall": 0.1667,
469
+ "f1": 0.1333,
470
+ "support": 6,
471
+ "predicted": 9
472
+ },
473
+ "T1036.005": {
474
+ "precision": 0.3333,
475
+ "recall": 0.125,
476
+ "f1": 0.1818,
477
+ "support": 8,
478
+ "predicted": 3
479
+ },
480
+ "T1041": {
481
+ "precision": 0.2692,
482
+ "recall": 0.5385,
483
+ "f1": 0.359,
484
+ "support": 13,
485
+ "predicted": 26
486
+ },
487
+ "T1047": {
488
+ "precision": 1.0,
489
+ "recall": 0.3,
490
+ "f1": 0.4615,
491
+ "support": 10,
492
+ "predicted": 3
493
+ },
494
+ "T1053.005": {
495
+ "precision": 0.6429,
496
+ "recall": 0.6429,
497
+ "f1": 0.6429,
498
+ "support": 14,
499
+ "predicted": 14
500
+ },
501
+ "T1055": {
502
+ "precision": 0.5714,
503
+ "recall": 0.6512,
504
+ "f1": 0.6087,
505
+ "support": 43,
506
+ "predicted": 49
507
+ },
508
+ "T1056.001": {
509
+ "precision": 0.4,
510
+ "recall": 0.6667,
511
+ "f1": 0.5,
512
+ "support": 6,
513
+ "predicted": 10
514
+ },
515
+ "T1057": {
516
+ "precision": 0.3462,
517
+ "recall": 0.75,
518
+ "f1": 0.4737,
519
+ "support": 12,
520
+ "predicted": 26
521
+ },
522
+ "T1059.003": {
523
+ "precision": 0.6053,
524
+ "recall": 0.5111,
525
+ "f1": 0.5542,
526
+ "support": 45,
527
+ "predicted": 38
528
+ },
529
+ "T1068": {
530
+ "precision": 0.0,
531
+ "recall": 0.0,
532
+ "f1": 0.0,
533
+ "support": 1,
534
+ "predicted": 5
535
+ },
536
+ "T1070.004": {
537
+ "precision": 0.6154,
538
+ "recall": 0.5333,
539
+ "f1": 0.5714,
540
+ "support": 15,
541
+ "predicted": 13
542
+ },
543
+ "T1071.001": {
544
+ "precision": 0.4545,
545
+ "recall": 0.5357,
546
+ "f1": 0.4918,
547
+ "support": 28,
548
+ "predicted": 33
549
+ },
550
+ "T1072": {
551
+ "precision": 0.0023,
552
+ "recall": 0.5,
553
+ "f1": 0.0045,
554
+ "support": 2,
555
+ "predicted": 442
556
+ },
557
+ "T1074.001": {
558
+ "precision": 0.0377,
559
+ "recall": 0.6667,
560
+ "f1": 0.0714,
561
+ "support": 3,
562
+ "predicted": 53
563
+ },
564
+ "T1078": {
565
+ "precision": 0.3103,
566
+ "recall": 0.4286,
567
+ "f1": 0.36,
568
+ "support": 21,
569
+ "predicted": 29
570
+ },
571
+ "T1082": {
572
+ "precision": 0.3913,
573
+ "recall": 0.5294,
574
+ "f1": 0.45,
575
+ "support": 17,
576
+ "predicted": 23
577
+ },
578
+ "T1083": {
579
+ "precision": 0.3125,
580
+ "recall": 0.3846,
581
+ "f1": 0.3448,
582
+ "support": 13,
583
+ "predicted": 16
584
+ },
585
+ "T1090": {
586
+ "precision": 0.3571,
587
+ "recall": 0.5556,
588
+ "f1": 0.4348,
589
+ "support": 18,
590
+ "predicted": 28
591
+ },
592
+ "T1095": {
593
+ "precision": 0.3684,
594
+ "recall": 0.5833,
595
+ "f1": 0.4516,
596
+ "support": 12,
597
+ "predicted": 19
598
+ },
599
+ "T1105": {
600
+ "precision": 0.3288,
601
+ "recall": 0.6,
602
+ "f1": 0.4248,
603
+ "support": 40,
604
+ "predicted": 73
605
+ },
606
+ "T1106": {
607
+ "precision": 0.5769,
608
+ "recall": 0.7143,
609
+ "f1": 0.6383,
610
+ "support": 21,
611
+ "predicted": 26
612
+ },
613
+ "T1110": {
614
+ "precision": 0.6,
615
+ "recall": 1.0,
616
+ "f1": 0.75,
617
+ "support": 3,
618
+ "predicted": 5
619
+ },
620
+ "T1112": {
621
+ "precision": 0.4231,
622
+ "recall": 0.7333,
623
+ "f1": 0.5366,
624
+ "support": 15,
625
+ "predicted": 26
626
+ },
627
+ "T1113": {
628
+ "precision": 0.75,
629
+ "recall": 0.375,
630
+ "f1": 0.5,
631
+ "support": 8,
632
+ "predicted": 4
633
+ },
634
+ "T1140": {
635
+ "precision": 0.561,
636
+ "recall": 0.6866,
637
+ "f1": 0.6174,
638
+ "support": 67,
639
+ "predicted": 82
640
+ },
641
+ "T1190": {
642
+ "precision": 0.6,
643
+ "recall": 0.2308,
644
+ "f1": 0.3333,
645
+ "support": 13,
646
+ "predicted": 5
647
+ },
648
+ "T1204.002": {
649
+ "precision": 0.1935,
650
+ "recall": 0.6,
651
+ "f1": 0.2927,
652
+ "support": 10,
653
+ "predicted": 31
654
+ },
655
+ "T1210": {
656
+ "precision": 0.2,
657
+ "recall": 0.5,
658
+ "f1": 0.2857,
659
+ "support": 2,
660
+ "predicted": 5
661
+ },
662
+ "T1218.011": {
663
+ "precision": 0.6364,
664
+ "recall": 0.6364,
665
+ "f1": 0.6364,
666
+ "support": 11,
667
+ "predicted": 11
668
+ },
669
+ "T1219": {
670
+ "precision": 0.4,
671
+ "recall": 0.5,
672
+ "f1": 0.4444,
673
+ "support": 4,
674
+ "predicted": 5
675
+ },
676
+ "T1484.001": {
677
+ "precision": 0.0,
678
+ "recall": 0.0,
679
+ "f1": 0.0,
680
+ "support": 4,
681
+ "predicted": 0
682
+ },
683
+ "T1518.001": {
684
+ "precision": 1.0,
685
+ "recall": 0.25,
686
+ "f1": 0.4,
687
+ "support": 4,
688
+ "predicted": 1
689
+ },
690
+ "T1543.003": {
691
+ "precision": 0.375,
692
+ "recall": 0.75,
693
+ "f1": 0.5,
694
+ "support": 4,
695
+ "predicted": 8
696
+ },
697
+ "T1547.001": {
698
+ "precision": 0.5,
699
+ "recall": 0.4,
700
+ "f1": 0.4444,
701
+ "support": 10,
702
+ "predicted": 8
703
+ },
704
+ "T1548.002": {
705
+ "precision": 1.0,
706
+ "recall": 0.5,
707
+ "f1": 0.6667,
708
+ "support": 6,
709
+ "predicted": 3
710
+ },
711
+ "T1552.001": {
712
+ "precision": 0.0,
713
+ "recall": 0.0,
714
+ "f1": 0.0,
715
+ "support": 3,
716
+ "predicted": 1
717
+ },
718
+ "T1562.001": {
719
+ "precision": 0.5238,
720
+ "recall": 0.8462,
721
+ "f1": 0.6471,
722
+ "support": 13,
723
+ "predicted": 21
724
+ },
725
+ "T1564.001": {
726
+ "precision": 0.1111,
727
+ "recall": 1.0,
728
+ "f1": 0.2,
729
+ "support": 1,
730
+ "predicted": 9
731
+ },
732
+ "T1566.001": {
733
+ "precision": 0.7143,
734
+ "recall": 0.6667,
735
+ "f1": 0.6897,
736
+ "support": 15,
737
+ "predicted": 14
738
+ },
739
+ "T1569.002": {
740
+ "precision": 0.3333,
741
+ "recall": 1.0,
742
+ "f1": 0.5,
743
+ "support": 2,
744
+ "predicted": 6
745
+ },
746
+ "T1570": {
747
+ "precision": 0.4286,
748
+ "recall": 0.2308,
749
+ "f1": 0.3,
750
+ "support": 13,
751
+ "predicted": 7
752
+ },
753
+ "T1573.001": {
754
+ "precision": 0.3077,
755
+ "recall": 0.4444,
756
+ "f1": 0.3636,
757
+ "support": 9,
758
+ "predicted": 13
759
+ },
760
+ "T1574.002": {
761
+ "precision": 0.7647,
762
+ "recall": 0.7222,
763
+ "f1": 0.7429,
764
+ "support": 18,
765
+ "predicted": 17
766
+ }
767
+ }
768
+ }
769
+ }