Hugging Face's logo

saiteki-kai
/
QA-DeBERTa-v3-large-diff-binary

Text Classification
Transformers
Safetensors
deberta-v2
multi_label_classification
question-answering
Generated from Trainer
Eval Results
Model card Files
xet
Community
saiteki-kai commited on
Commit
52d26e4
·
verified ·
1 Parent(s): 4f044f8

End of training

Browse files
Files changed (6) hide show
  1. README.md +97 -82
  2. all_results.json +210 -0
  3. eval_results.json +104 -0
  4. test_results.json +103 -0
  5. train_results.json +8 -0
  6. trainer_state.json +0 -0
README.md CHANGED
@@ -3,12 +3,27 @@ library_name: transformers
3
  license: mit
4
  base_model: microsoft/deberta-v3-large
5
  tags:
 
 
 
6
  - generated_from_trainer
 
 
7
  metrics:
8
  - accuracy
9
  model-index:
10
  - name: QA-DeBERTa-v3-large-diff-binary
11
- results: []
 
 
 
 
 
 
 
 
 
 
12
  ---
13
 
14
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -16,105 +31,105 @@ should probably proofread and complete it, then remove this comment. -->
16
 
17
  # QA-DeBERTa-v3-large-diff-binary
18
 
19
- This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 0.0818
22
- - Accuracy: 0.6855
23
- - Macro F1: 0.6425
24
- - Macro Precision: 0.6875
25
- - Macro Recall: 0.6310
26
- - Micro F1: 0.7528
27
- - Micro Precision: 0.7796
28
- - Micro Recall: 0.7277
29
- - Flagged/accuracy: 0.8531
30
- - Flagged/precision: 0.8802
31
- - Flagged/recall: 0.8519
32
- - Flagged/f1: 0.8659
33
- - Flagged/aucpr: 0.9073
34
- - Flagged/fpr: 0.1454
35
- - Animal Abuse/accuracy: 0.9947
36
- - Animal Abuse/precision: 0.7700
37
- - Animal Abuse/recall: 0.7689
38
- - Animal Abuse/f1: 0.7695
39
- - Animal Abuse/fpr: 0.0027
40
  - Animal Abuse/threshold: 0.5
41
- - Child Abuse/accuracy: 0.9963
42
- - Child Abuse/precision: 0.6731
43
- - Child Abuse/recall: 0.6306
44
- - Child Abuse/f1: 0.6512
45
- - Child Abuse/fpr: 0.0017
46
  - Child Abuse/threshold: 0.5
47
- - Controversial Topics,politics/accuracy: 0.9692
48
- - Controversial Topics,politics/precision: 0.4970
49
- - Controversial Topics,politics/recall: 0.4902
50
- - Controversial Topics,politics/f1: 0.4936
51
- - Controversial Topics,politics/fpr: 0.0157
52
  - Controversial Topics,politics/threshold: 0.5
53
- - Discrimination,stereotype,injustice/accuracy: 0.9559
54
  - Discrimination,stereotype,injustice/precision: 0.7313
55
- - Discrimination,stereotype,injustice/recall: 0.7044
56
- - Discrimination,stereotype,injustice/f1: 0.7176
57
- - Discrimination,stereotype,injustice/fpr: 0.0224
58
  - Discrimination,stereotype,injustice/threshold: 0.5
59
  - Drug Abuse,weapons,banned Substance/accuracy: 0.9742
60
- - Drug Abuse,weapons,banned Substance/precision: 0.7716
61
- - Drug Abuse,weapons,banned Substance/recall: 0.7711
62
- - Drug Abuse,weapons,banned Substance/f1: 0.7713
63
- - Drug Abuse,weapons,banned Substance/fpr: 0.0136
64
  - Drug Abuse,weapons,banned Substance/threshold: 0.5
65
- - Financial Crime,property Crime,theft/accuracy: 0.9603
66
- - Financial Crime,property Crime,theft/precision: 0.7730
67
- - Financial Crime,property Crime,theft/recall: 0.8383
68
- - Financial Crime,property Crime,theft/f1: 0.8044
69
- - Financial Crime,property Crime,theft/fpr: 0.0265
70
  - Financial Crime,property Crime,theft/threshold: 0.5
71
- - Hate Speech,offensive Language/accuracy: 0.9510
72
- - Hate Speech,offensive Language/precision: 0.7896
73
- - Hate Speech,offensive Language/recall: 0.6176
74
- - Hate Speech,offensive Language/f1: 0.6931
75
- - Hate Speech,offensive Language/fpr: 0.0162
76
  - Hate Speech,offensive Language/threshold: 0.5
77
- - Misinformation Regarding Ethics,laws And Safety/accuracy: 0.9874
78
- - Misinformation Regarding Ethics,laws And Safety/precision: 0.3968
79
- - Misinformation Regarding Ethics,laws And Safety/recall: 0.0684
80
- - Misinformation Regarding Ethics,laws And Safety/f1: 0.1167
81
- - Misinformation Regarding Ethics,laws And Safety/fpr: 0.0013
82
  - Misinformation Regarding Ethics,laws And Safety/threshold: 0.5
83
- - Non Violent Unethical Behavior/accuracy: 0.8883
84
- - Non Violent Unethical Behavior/precision: 0.7764
85
- - Non Violent Unethical Behavior/recall: 0.6154
86
- - Non Violent Unethical Behavior/f1: 0.6866
87
- - Non Violent Unethical Behavior/fpr: 0.0440
88
  - Non Violent Unethical Behavior/threshold: 0.5
89
  - Privacy Violation/accuracy: 0.9809
90
- - Privacy Violation/precision: 0.7869
91
- - Privacy Violation/recall: 0.8415
92
- - Privacy Violation/f1: 0.8133
93
- - Privacy Violation/fpr: 0.0118
94
  - Privacy Violation/threshold: 0.5
95
- - Self Harm/accuracy: 0.9954
96
- - Self Harm/precision: 0.6315
97
- - Self Harm/recall: 0.7732
98
- - Self Harm/f1: 0.6952
99
- - Self Harm/fpr: 0.0031
100
  - Self Harm/threshold: 0.5
101
- - Sexually Explicit,adult Content/accuracy: 0.9839
102
- - Sexually Explicit,adult Content/precision: 0.6607
103
- - Sexually Explicit,adult Content/recall: 0.6835
104
- - Sexually Explicit,adult Content/f1: 0.6719
105
- - Sexually Explicit,adult Content/fpr: 0.0087
106
  - Sexually Explicit,adult Content/threshold: 0.5
107
  - Terrorism,organized Crime/accuracy: 0.9921
108
- - Terrorism,organized Crime/precision: 0.5159
109
- - Terrorism,organized Crime/recall: 0.1684
110
- - Terrorism,organized Crime/f1: 0.2539
111
- - Terrorism,organized Crime/fpr: 0.0013
112
  - Terrorism,organized Crime/threshold: 0.5
113
- - Violence,aiding And Abetting,incitement/accuracy: 0.9231
114
- - Violence,aiding And Abetting,incitement/precision: 0.8507
115
- - Violence,aiding And Abetting,incitement/recall: 0.8623
116
- - Violence,aiding And Abetting,incitement/f1: 0.8564
117
- - Violence,aiding And Abetting,incitement/fpr: 0.0549
118
  - Violence,aiding And Abetting,incitement/threshold: 0.5
119
 
120
  ## Model description
 
3
  license: mit
4
  base_model: microsoft/deberta-v3-large
5
  tags:
6
+ - multi_label_classification
7
+ - question-answering
8
+ - text-classification
9
  - generated_from_trainer
10
+ datasets:
11
+ - beavertails
12
  metrics:
13
  - accuracy
14
  model-index:
15
  - name: QA-DeBERTa-v3-large-diff-binary
16
+ results:
17
+ - task:
18
+ name: Text Classification
19
+ type: text-classification
20
+ dataset:
21
+ name: saiteki-kai/Beavertails-it
22
+ type: beavertails
23
+ metrics:
24
+ - name: Accuracy
25
+ type: accuracy
26
+ value: 0.6889576471371062
27
  ---
28
 
29
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
31
 
32
  # QA-DeBERTa-v3-large-diff-binary
33
 
34
+ This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) on the saiteki-kai/Beavertails-it dataset.
35
  It achieves the following results on the evaluation set:
36
+ - Loss: 0.0823
37
+ - Accuracy: 0.6890
38
+ - Macro F1: 0.6419
39
+ - Macro Precision: 0.7045
40
+ - Macro Recall: 0.6272
41
+ - Micro F1: 0.7567
42
+ - Micro Precision: 0.7754
43
+ - Micro Recall: 0.7390
44
+ - Flagged/accuracy: 0.8562
45
+ - Flagged/precision: 0.8819
46
+ - Flagged/recall: 0.8563
47
+ - Flagged/f1: 0.8689
48
+ - Flagged/aucpr: 0.9091
49
+ - Flagged/fpr: 0.1439
50
+ - Animal Abuse/accuracy: 0.9945
51
+ - Animal Abuse/precision: 0.7337
52
+ - Animal Abuse/recall: 0.8169
53
+ - Animal Abuse/f1: 0.7730
54
+ - Animal Abuse/fpr: 0.0034
55
  - Animal Abuse/threshold: 0.5
56
+ - Child Abuse/accuracy: 0.9964
57
+ - Child Abuse/precision: 0.7007
58
+ - Child Abuse/recall: 0.6186
59
+ - Child Abuse/f1: 0.6571
60
+ - Child Abuse/fpr: 0.0015
61
  - Child Abuse/threshold: 0.5
62
+ - Controversial Topics,politics/accuracy: 0.9715
63
+ - Controversial Topics,politics/precision: 0.5467
64
+ - Controversial Topics,politics/recall: 0.4099
65
+ - Controversial Topics,politics/f1: 0.4685
66
+ - Controversial Topics,politics/fpr: 0.0107
67
  - Controversial Topics,politics/threshold: 0.5
68
+ - Discrimination,stereotype,injustice/accuracy: 0.9564
69
  - Discrimination,stereotype,injustice/precision: 0.7313
70
+ - Discrimination,stereotype,injustice/recall: 0.7146
71
+ - Discrimination,stereotype,injustice/f1: 0.7229
72
+ - Discrimination,stereotype,injustice/fpr: 0.0227
73
  - Discrimination,stereotype,injustice/threshold: 0.5
74
  - Drug Abuse,weapons,banned Substance/accuracy: 0.9742
75
+ - Drug Abuse,weapons,banned Substance/precision: 0.7637
76
+ - Drug Abuse,weapons,banned Substance/recall: 0.7847
77
+ - Drug Abuse,weapons,banned Substance/f1: 0.7741
78
+ - Drug Abuse,weapons,banned Substance/fpr: 0.0145
79
  - Drug Abuse,weapons,banned Substance/threshold: 0.5
80
+ - Financial Crime,property Crime,theft/accuracy: 0.9601
81
+ - Financial Crime,property Crime,theft/precision: 0.7676
82
+ - Financial Crime,property Crime,theft/recall: 0.8464
83
+ - Financial Crime,property Crime,theft/f1: 0.8051
84
+ - Financial Crime,property Crime,theft/fpr: 0.0276
85
  - Financial Crime,property Crime,theft/threshold: 0.5
86
+ - Hate Speech,offensive Language/accuracy: 0.9506
87
+ - Hate Speech,offensive Language/precision: 0.7660
88
+ - Hate Speech,offensive Language/recall: 0.6462
89
+ - Hate Speech,offensive Language/f1: 0.7010
90
+ - Hate Speech,offensive Language/fpr: 0.0194
91
  - Hate Speech,offensive Language/threshold: 0.5
92
+ - Misinformation Regarding Ethics,laws And Safety/accuracy: 0.9879
93
+ - Misinformation Regarding Ethics,laws And Safety/precision: 0.5179
94
+ - Misinformation Regarding Ethics,laws And Safety/recall: 0.0397
95
+ - Misinformation Regarding Ethics,laws And Safety/f1: 0.0737
96
+ - Misinformation Regarding Ethics,laws And Safety/fpr: 0.0005
97
  - Misinformation Regarding Ethics,laws And Safety/threshold: 0.5
98
+ - Non Violent Unethical Behavior/accuracy: 0.8880
99
+ - Non Violent Unethical Behavior/precision: 0.7571
100
+ - Non Violent Unethical Behavior/recall: 0.6422
101
+ - Non Violent Unethical Behavior/f1: 0.6950
102
+ - Non Violent Unethical Behavior/fpr: 0.0511
103
  - Non Violent Unethical Behavior/threshold: 0.5
104
  - Privacy Violation/accuracy: 0.9809
105
+ - Privacy Violation/precision: 0.7844
106
+ - Privacy Violation/recall: 0.8439
107
+ - Privacy Violation/f1: 0.8131
108
+ - Privacy Violation/fpr: 0.0120
109
  - Privacy Violation/threshold: 0.5
110
+ - Self Harm/accuracy: 0.9965
111
+ - Self Harm/precision: 0.7672
112
+ - Self Harm/recall: 0.7073
113
+ - Self Harm/f1: 0.7360
114
+ - Self Harm/fpr: 0.0015
115
  - Self Harm/threshold: 0.5
116
+ - Sexually Explicit,adult Content/accuracy: 0.9843
117
+ - Sexually Explicit,adult Content/precision: 0.6691
118
+ - Sexually Explicit,adult Content/recall: 0.6876
119
+ - Sexually Explicit,adult Content/f1: 0.6783
120
+ - Sexually Explicit,adult Content/fpr: 0.0084
121
  - Sexually Explicit,adult Content/threshold: 0.5
122
  - Terrorism,organized Crime/accuracy: 0.9921
123
+ - Terrorism,organized Crime/precision: 0.5180
124
+ - Terrorism,organized Crime/recall: 0.1497
125
+ - Terrorism,organized Crime/f1: 0.2323
126
+ - Terrorism,organized Crime/fpr: 0.0011
127
  - Terrorism,organized Crime/threshold: 0.5
128
+ - Violence,aiding And Abetting,incitement/accuracy: 0.9221
129
+ - Violence,aiding And Abetting,incitement/precision: 0.8400
130
+ - Violence,aiding And Abetting,incitement/recall: 0.8736
131
+ - Violence,aiding And Abetting,incitement/f1: 0.8565
132
+ - Violence,aiding And Abetting,incitement/fpr: 0.0603
133
  - Violence,aiding And Abetting,incitement/threshold: 0.5
134
 
135
  ## Model description
all_results.json ADDED
@@ -0,0 +1,210 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 4.000946297610598,
3
+ "eval_accuracy": 0.6889576471371062,
4
+ "eval_animal_abuse/accuracy": 0.994510430182653,
5
+ "eval_animal_abuse/f1": 0.7730398899587345,
6
+ "eval_animal_abuse/fpr": 0.0034328408440749783,
7
+ "eval_animal_abuse/precision": 0.7336814621409922,
8
+ "eval_animal_abuse/recall": 0.8168604651162791,
9
+ "eval_animal_abuse/threshold": 0.5,
10
+ "eval_child_abuse/accuracy": 0.9964234620886981,
11
+ "eval_child_abuse/f1": 0.6570972886762361,
12
+ "eval_child_abuse/fpr": 0.0014720396112477183,
13
+ "eval_child_abuse/precision": 0.7006802721088435,
14
+ "eval_child_abuse/recall": 0.6186186186186187,
15
+ "eval_child_abuse/threshold": 0.5,
16
+ "eval_controversial_topics,politics/accuracy": 0.9715041421299531,
17
+ "eval_controversial_topics,politics/f1": 0.46850760161340366,
18
+ "eval_controversial_topics,politics/fpr": 0.010742723778143858,
19
+ "eval_controversial_topics,politics/precision": 0.5467052860246199,
20
+ "eval_controversial_topics,politics/recall": 0.40988056460369166,
21
+ "eval_controversial_topics,politics/threshold": 0.5,
22
+ "eval_discrimination,stereotype,injustice/accuracy": 0.9564327777223276,
23
+ "eval_discrimination,stereotype,injustice/f1": 0.7228864670405248,
24
+ "eval_discrimination,stereotype,injustice/fpr": 0.02268044963313691,
25
+ "eval_discrimination,stereotype,injustice/precision": 0.7313209162920146,
26
+ "eval_discrimination,stereotype,injustice/recall": 0.7146443514644352,
27
+ "eval_discrimination,stereotype,injustice/threshold": 0.5,
28
+ "eval_drug_abuse,weapons,banned_substance/accuracy": 0.974199021858469,
29
+ "eval_drug_abuse,weapons,banned_substance/f1": 0.7740713765477057,
30
+ "eval_drug_abuse,weapons,banned_substance/fpr": 0.014490198843604543,
31
+ "eval_drug_abuse,weapons,banned_substance/precision": 0.7637252083932164,
32
+ "eval_drug_abuse,weapons,banned_substance/recall": 0.7847017129356173,
33
+ "eval_drug_abuse,weapons,banned_substance/threshold": 0.5,
34
+ "eval_financial_crime,property_crime,theft/accuracy": 0.9601091259939448,
35
+ "eval_financial_crime,property_crime,theft/f1": 0.8050723459600065,
36
+ "eval_financial_crime,property_crime,theft/fpr": 0.02762471665775938,
37
+ "eval_financial_crime,property_crime,theft/precision": 0.7676329251278872,
38
+ "eval_financial_crime,property_crime,theft/recall": 0.8463510511023756,
39
+ "eval_financial_crime,property_crime,theft/threshold": 0.5,
40
+ "eval_flagged/accuracy": 0.8562065409056127,
41
+ "eval_flagged/aucpr": 0.9090708301585808,
42
+ "eval_flagged/f1": 0.8688912482936447,
43
+ "eval_flagged/fpr": 0.14386978209503754,
44
+ "eval_flagged/precision": 0.8818929154222729,
45
+ "eval_flagged/recall": 0.8562673761621477,
46
+ "eval_hate_speech,offensive_language/accuracy": 0.9506271417639818,
47
+ "eval_hate_speech,offensive_language/f1": 0.7009873060648801,
48
+ "eval_hate_speech,offensive_language/fpr": 0.019422620135209172,
49
+ "eval_hate_speech,offensive_language/precision": 0.765962131219727,
50
+ "eval_hate_speech,offensive_language/recall": 0.6461738484398217,
51
+ "eval_hate_speech,offensive_language/threshold": 0.5,
52
+ "eval_loss": 0.08228794485330582,
53
+ "eval_macro_f1": 0.6418844118541835,
54
+ "eval_macro_precision": 0.704528612500405,
55
+ "eval_macro_recall": 0.6272299800362929,
56
+ "eval_micro_f1": 0.7567282129720675,
57
+ "eval_micro_precision": 0.7753502863859401,
58
+ "eval_micro_recall": 0.7389796753027367,
59
+ "eval_misinformation_regarding_ethics,laws_and_safety/accuracy": 0.9878730412216788,
60
+ "eval_misinformation_regarding_ethics,laws_and_safety/f1": 0.07369758576874205,
61
+ "eval_misinformation_regarding_ethics,laws_and_safety/fpr": 0.00045467558055335624,
62
+ "eval_misinformation_regarding_ethics,laws_and_safety/precision": 0.5178571428571429,
63
+ "eval_misinformation_regarding_ethics,laws_and_safety/recall": 0.03967168262653899,
64
+ "eval_misinformation_regarding_ethics,laws_and_safety/threshold": 0.5,
65
+ "eval_non_violent_unethical_behavior/accuracy": 0.8879628705459627,
66
+ "eval_non_violent_unethical_behavior/f1": 0.6949590108247656,
67
+ "eval_non_violent_unethical_behavior/fpr": 0.05109201129380491,
68
+ "eval_non_violent_unethical_behavior/precision": 0.7571301687555512,
69
+ "eval_non_violent_unethical_behavior/recall": 0.642223338355935,
70
+ "eval_non_violent_unethical_behavior/threshold": 0.5,
71
+ "eval_privacy_violation/accuracy": 0.9808530458794956,
72
+ "eval_privacy_violation/f1": 0.8130583076173461,
73
+ "eval_privacy_violation/fpr": 0.012038916497515203,
74
+ "eval_privacy_violation/precision": 0.784393607019743,
75
+ "eval_privacy_violation/recall": 0.8438975050573162,
76
+ "eval_privacy_violation/threshold": 0.5,
77
+ "eval_runtime": 598.0644,
78
+ "eval_samples_per_second": 100.514,
79
+ "eval_self_harm/accuracy": 0.9965399075090661,
80
+ "eval_self_harm/f1": 0.7360406091370558,
81
+ "eval_self_harm/fpr": 0.0014739380946000244,
82
+ "eval_self_harm/precision": 0.7671957671957672,
83
+ "eval_self_harm/recall": 0.7073170731707317,
84
+ "eval_self_harm/threshold": 0.5,
85
+ "eval_sexually_explicit,adult_content/accuracy": 0.984296503310377,
86
+ "eval_sexually_explicit,adult_content/f1": 0.678254942058623,
87
+ "eval_sexually_explicit,adult_content/fpr": 0.00838631598684097,
88
+ "eval_sexually_explicit,adult_content/precision": 0.6691324815063887,
89
+ "eval_sexually_explicit,adult_content/recall": 0.6876295784381479,
90
+ "eval_sexually_explicit,adult_content/threshold": 0.5,
91
+ "eval_steps_per_second": 1.572,
92
+ "eval_terrorism,organized_crime/accuracy": 0.9920817114149783,
93
+ "eval_terrorism,organized_crime/f1": 0.23225806451612904,
94
+ "eval_terrorism,organized_crime/fpr": 0.001123538980094912,
95
+ "eval_terrorism,organized_crime/precision": 0.5179856115107914,
96
+ "eval_terrorism,organized_crime/recall": 0.1496881496881497,
97
+ "eval_terrorism,organized_crime/threshold": 0.5,
98
+ "eval_violence,aiding_and_abetting,incitement/accuracy": 0.9220980137738297,
99
+ "eval_violence,aiding_and_abetting,incitement/f1": 0.8564509701744168,
100
+ "eval_violence,aiding_and_abetting,incitement/fpr": 0.06031004940845822,
101
+ "eval_violence,aiding_and_abetting,incitement/precision": 0.8399975948529854,
102
+ "eval_violence,aiding_and_abetting,incitement/recall": 0.8735617808904452,
103
+ "eval_violence,aiding_and_abetting,incitement/threshold": 0.5,
104
+ "test_accuracy": 0.6766379207090669,
105
+ "test_animal_abuse/accuracy": 0.9931578632171517,
106
+ "test_animal_abuse/f1": 0.7676664972038637,
107
+ "test_animal_abuse/fpr": 0.005037019055710637,
108
+ "test_animal_abuse/precision": 0.6945722171113156,
109
+ "test_animal_abuse/recall": 0.8579545454545454,
110
+ "test_animal_abuse/threshold": 0.5,
111
+ "test_child_abuse/accuracy": 0.9963169241825368,
112
+ "test_child_abuse/f1": 0.6070287539936102,
113
+ "test_child_abuse/fpr": 0.0012642986152919908,
114
+ "test_child_abuse/precision": 0.6934306569343066,
115
+ "test_child_abuse/recall": 0.5397727272727273,
116
+ "test_child_abuse/threshold": 0.5,
117
+ "test_controversial_topics,politics/accuracy": 0.9715085638998683,
118
+ "test_controversial_topics,politics/f1": 0.45519610649871173,
119
+ "test_controversial_topics,politics/fpr": 0.011352768779885837,
120
+ "test_controversial_topics,politics/precision": 0.519268451992162,
121
+ "test_controversial_topics,politics/recall": 0.40519877675840976,
122
+ "test_controversial_topics,politics/threshold": 0.5,
123
+ "test_discrimination,stereotype,injustice/accuracy": 0.9543957360162894,
124
+ "test_discrimination,stereotype,injustice/f1": 0.7218772826880935,
125
+ "test_discrimination,stereotype,injustice/fpr": 0.023755877742946668,
126
+ "test_discrimination,stereotype,injustice/precision": 0.7309541420118343,
127
+ "test_discrimination,stereotype,injustice/recall": 0.713023088023088,
128
+ "test_discrimination,stereotype,injustice/threshold": 0.5,
129
+ "test_drug_abuse,weapons,banned_substance/accuracy": 0.9734848484848485,
130
+ "test_drug_abuse,weapons,banned_substance/f1": 0.7634566582075598,
131
+ "test_drug_abuse,weapons,banned_substance/fpr": 0.014630821418381233,
132
+ "test_drug_abuse,weapons,banned_substance/precision": 0.755884686590849,
133
+ "test_drug_abuse,weapons,banned_substance/recall": 0.7711818672423097,
134
+ "test_drug_abuse,weapons,banned_substance/threshold": 0.5,
135
+ "test_financial_crime,property_crime,theft/accuracy": 0.9579590370104204,
136
+ "test_financial_crime,property_crime,theft/f1": 0.7996575342465754,
137
+ "test_financial_crime,property_crime,theft/fpr": 0.0271945610877824,
138
+ "test_financial_crime,property_crime,theft/precision": 0.7744610281923715,
139
+ "test_financial_crime,property_crime,theft/recall": 0.8265486725663717,
140
+ "test_financial_crime,property_crime,theft/threshold": 0.5,
141
+ "test_flagged/accuracy": 0.8487393699844292,
142
+ "test_flagged/aucpr": 0.9047138471115324,
143
+ "test_flagged/f1": 0.8630417395312266,
144
+ "test_flagged/fpr": 0.15492622560685337,
145
+ "test_flagged/precision": 0.8747698480309983,
146
+ "test_flagged/recall": 0.8516239499170635,
147
+ "test_hate_speech,offensive_language/accuracy": 0.9497844053180021,
148
+ "test_hate_speech,offensive_language/f1": 0.686307519640853,
149
+ "test_hate_speech,offensive_language/fpr": 0.017700424021299645,
150
+ "test_hate_speech,offensive_language/precision": 0.7730720606826802,
151
+ "test_hate_speech,offensive_language/recall": 0.6170534813319879,
152
+ "test_hate_speech,offensive_language/threshold": 0.5,
153
+ "test_loss": 0.08679678291082382,
154
+ "test_macro_f1": 0.6252685023565255,
155
+ "test_macro_precision": 0.6847558918566842,
156
+ "test_macro_recall": 0.6093259557964672,
157
+ "test_micro_f1": 0.7468949012290593,
158
+ "test_micro_precision": 0.7705170480223624,
159
+ "test_micro_recall": 0.724678064292686,
160
+ "test_misinformation_regarding_ethics,laws_and_safety/accuracy": 0.987633249490957,
161
+ "test_misinformation_regarding_ethics,laws_and_safety/f1": 0.05707762557077625,
162
+ "test_misinformation_regarding_ethics,laws_and_safety/fpr": 0.0005304959379168174,
163
+ "test_misinformation_regarding_ethics,laws_and_safety/precision": 0.4166666666666667,
164
+ "test_misinformation_regarding_ethics,laws_and_safety/recall": 0.030637254901960783,
165
+ "test_misinformation_regarding_ethics,laws_and_safety/threshold": 0.5,
166
+ "test_non_violent_unethical_behavior/accuracy": 0.8818571086357647,
167
+ "test_non_violent_unethical_behavior/f1": 0.6774575924790517,
168
+ "test_non_violent_unethical_behavior/fpr": 0.050999362507968556,
169
+ "test_non_violent_unethical_behavior/precision": 0.7528845280276187,
170
+ "test_non_violent_unethical_behavior/recall": 0.6157675731906672,
171
+ "test_non_violent_unethical_behavior/threshold": 0.5,
172
+ "test_privacy_violation/accuracy": 0.9793687866810397,
173
+ "test_privacy_violation/f1": 0.8086111111111111,
174
+ "test_privacy_violation/fpr": 0.01268442169778534,
175
+ "test_privacy_violation/precision": 0.7837910608508347,
176
+ "test_privacy_violation/recall": 0.8350545037292025,
177
+ "test_privacy_violation/threshold": 0.5,
178
+ "test_runtime": 664.4818,
179
+ "test_samples_per_second": 100.517,
180
+ "test_self_harm/accuracy": 0.9963917834471194,
181
+ "test_self_harm/f1": 0.714792899408284,
182
+ "test_self_harm/fpr": 0.0011910505367265691,
183
+ "test_self_harm/precision": 0.7926509186351706,
184
+ "test_self_harm/recall": 0.6508620689655172,
185
+ "test_self_harm/threshold": 0.5,
186
+ "test_sexually_explicit,adult_content/accuracy": 0.9838902862618277,
187
+ "test_sexually_explicit,adult_content/f1": 0.6515544041450777,
188
+ "test_sexually_explicit,adult_content/fpr": 0.009186954524575089,
189
+ "test_sexually_explicit,adult_content/precision": 0.6264009962640099,
190
+ "test_sexually_explicit,adult_content/recall": 0.6788124156545209,
191
+ "test_sexually_explicit,adult_content/threshold": 0.5,
192
+ "test_steps_per_second": 1.571,
193
+ "test_terrorism,organized_crime/accuracy": 0.9913911845730028,
194
+ "test_terrorism,organized_crime/f1": 0.1958041958041958,
195
+ "test_terrorism,organized_crime/fpr": 0.001343680173923544,
196
+ "test_terrorism,organized_crime/precision": 0.44025157232704404,
197
+ "test_terrorism,organized_crime/recall": 0.12589928057553956,
198
+ "test_terrorism,organized_crime/threshold": 0.5,
199
+ "test_violence,aiding_and_abetting,incitement/accuracy": 0.9157533836387591,
200
+ "test_violence,aiding_and_abetting,incitement/f1": 0.8472708519935944,
201
+ "test_violence,aiding_and_abetting,incitement/fpr": 0.06457640343312376,
202
+ "test_violence,aiding_and_abetting,incitement/precision": 0.8322934997067136,
203
+ "test_violence,aiding_and_abetting,incitement/recall": 0.8627971254836927,
204
+ "test_violence,aiding_and_abetting,incitement/threshold": 0.5,
205
+ "total_flos": 0.0,
206
+ "train_loss": 0.09237335174007019,
207
+ "train_runtime": 91030.0055,
208
+ "train_samples_per_second": 59.433,
209
+ "train_steps_per_second": 0.929
210
+ }
eval_results.json ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 4.000946297610598,
3
+ "eval_accuracy": 0.6889576471371062,
4
+ "eval_animal_abuse/accuracy": 0.994510430182653,
5
+ "eval_animal_abuse/f1": 0.7730398899587345,
6
+ "eval_animal_abuse/fpr": 0.0034328408440749783,
7
+ "eval_animal_abuse/precision": 0.7336814621409922,
8
+ "eval_animal_abuse/recall": 0.8168604651162791,
9
+ "eval_animal_abuse/threshold": 0.5,
10
+ "eval_child_abuse/accuracy": 0.9964234620886981,
11
+ "eval_child_abuse/f1": 0.6570972886762361,
12
+ "eval_child_abuse/fpr": 0.0014720396112477183,
13
+ "eval_child_abuse/precision": 0.7006802721088435,
14
+ "eval_child_abuse/recall": 0.6186186186186187,
15
+ "eval_child_abuse/threshold": 0.5,
16
+ "eval_controversial_topics,politics/accuracy": 0.9715041421299531,
17
+ "eval_controversial_topics,politics/f1": 0.46850760161340366,
18
+ "eval_controversial_topics,politics/fpr": 0.010742723778143858,
19
+ "eval_controversial_topics,politics/precision": 0.5467052860246199,
20
+ "eval_controversial_topics,politics/recall": 0.40988056460369166,
21
+ "eval_controversial_topics,politics/threshold": 0.5,
22
+ "eval_discrimination,stereotype,injustice/accuracy": 0.9564327777223276,
23
+ "eval_discrimination,stereotype,injustice/f1": 0.7228864670405248,
24
+ "eval_discrimination,stereotype,injustice/fpr": 0.02268044963313691,
25
+ "eval_discrimination,stereotype,injustice/precision": 0.7313209162920146,
26
+ "eval_discrimination,stereotype,injustice/recall": 0.7146443514644352,
27
+ "eval_discrimination,stereotype,injustice/threshold": 0.5,
28
+ "eval_drug_abuse,weapons,banned_substance/accuracy": 0.974199021858469,
29
+ "eval_drug_abuse,weapons,banned_substance/f1": 0.7740713765477057,
30
+ "eval_drug_abuse,weapons,banned_substance/fpr": 0.014490198843604543,
31
+ "eval_drug_abuse,weapons,banned_substance/precision": 0.7637252083932164,
32
+ "eval_drug_abuse,weapons,banned_substance/recall": 0.7847017129356173,
33
+ "eval_drug_abuse,weapons,banned_substance/threshold": 0.5,
34
+ "eval_financial_crime,property_crime,theft/accuracy": 0.9601091259939448,
35
+ "eval_financial_crime,property_crime,theft/f1": 0.8050723459600065,
36
+ "eval_financial_crime,property_crime,theft/fpr": 0.02762471665775938,
37
+ "eval_financial_crime,property_crime,theft/precision": 0.7676329251278872,
38
+ "eval_financial_crime,property_crime,theft/recall": 0.8463510511023756,
39
+ "eval_financial_crime,property_crime,theft/threshold": 0.5,
40
+ "eval_flagged/accuracy": 0.8562065409056127,
41
+ "eval_flagged/aucpr": 0.9090708301585808,
42
+ "eval_flagged/f1": 0.8688912482936447,
43
+ "eval_flagged/fpr": 0.14386978209503754,
44
+ "eval_flagged/precision": 0.8818929154222729,
45
+ "eval_flagged/recall": 0.8562673761621477,
46
+ "eval_hate_speech,offensive_language/accuracy": 0.9506271417639818,
47
+ "eval_hate_speech,offensive_language/f1": 0.7009873060648801,
48
+ "eval_hate_speech,offensive_language/fpr": 0.019422620135209172,
49
+ "eval_hate_speech,offensive_language/precision": 0.765962131219727,
50
+ "eval_hate_speech,offensive_language/recall": 0.6461738484398217,
51
+ "eval_hate_speech,offensive_language/threshold": 0.5,
52
+ "eval_loss": 0.08228794485330582,
53
+ "eval_macro_f1": 0.6418844118541835,
54
+ "eval_macro_precision": 0.704528612500405,
55
+ "eval_macro_recall": 0.6272299800362929,
56
+ "eval_micro_f1": 0.7567282129720675,
57
+ "eval_micro_precision": 0.7753502863859401,
58
+ "eval_micro_recall": 0.7389796753027367,
59
+ "eval_misinformation_regarding_ethics,laws_and_safety/accuracy": 0.9878730412216788,
60
+ "eval_misinformation_regarding_ethics,laws_and_safety/f1": 0.07369758576874205,
61
+ "eval_misinformation_regarding_ethics,laws_and_safety/fpr": 0.00045467558055335624,
62
+ "eval_misinformation_regarding_ethics,laws_and_safety/precision": 0.5178571428571429,
63
+ "eval_misinformation_regarding_ethics,laws_and_safety/recall": 0.03967168262653899,
64
+ "eval_misinformation_regarding_ethics,laws_and_safety/threshold": 0.5,
65
+ "eval_non_violent_unethical_behavior/accuracy": 0.8879628705459627,
66
+ "eval_non_violent_unethical_behavior/f1": 0.6949590108247656,
67
+ "eval_non_violent_unethical_behavior/fpr": 0.05109201129380491,
68
+ "eval_non_violent_unethical_behavior/precision": 0.7571301687555512,
69
+ "eval_non_violent_unethical_behavior/recall": 0.642223338355935,
70
+ "eval_non_violent_unethical_behavior/threshold": 0.5,
71
+ "eval_privacy_violation/accuracy": 0.9808530458794956,
72
+ "eval_privacy_violation/f1": 0.8130583076173461,
73
+ "eval_privacy_violation/fpr": 0.012038916497515203,
74
+ "eval_privacy_violation/precision": 0.784393607019743,
75
+ "eval_privacy_violation/recall": 0.8438975050573162,
76
+ "eval_privacy_violation/threshold": 0.5,
77
+ "eval_runtime": 598.0644,
78
+ "eval_samples_per_second": 100.514,
79
+ "eval_self_harm/accuracy": 0.9965399075090661,
80
+ "eval_self_harm/f1": 0.7360406091370558,
81
+ "eval_self_harm/fpr": 0.0014739380946000244,
82
+ "eval_self_harm/precision": 0.7671957671957672,
83
+ "eval_self_harm/recall": 0.7073170731707317,
84
+ "eval_self_harm/threshold": 0.5,
85
+ "eval_sexually_explicit,adult_content/accuracy": 0.984296503310377,
86
+ "eval_sexually_explicit,adult_content/f1": 0.678254942058623,
87
+ "eval_sexually_explicit,adult_content/fpr": 0.00838631598684097,
88
+ "eval_sexually_explicit,adult_content/precision": 0.6691324815063887,
89
+ "eval_sexually_explicit,adult_content/recall": 0.6876295784381479,
90
+ "eval_sexually_explicit,adult_content/threshold": 0.5,
91
+ "eval_steps_per_second": 1.572,
92
+ "eval_terrorism,organized_crime/accuracy": 0.9920817114149783,
93
+ "eval_terrorism,organized_crime/f1": 0.23225806451612904,
94
+ "eval_terrorism,organized_crime/fpr": 0.001123538980094912,
95
+ "eval_terrorism,organized_crime/precision": 0.5179856115107914,
96
+ "eval_terrorism,organized_crime/recall": 0.1496881496881497,
97
+ "eval_terrorism,organized_crime/threshold": 0.5,
98
+ "eval_violence,aiding_and_abetting,incitement/accuracy": 0.9220980137738297,
99
+ "eval_violence,aiding_and_abetting,incitement/f1": 0.8564509701744168,
100
+ "eval_violence,aiding_and_abetting,incitement/fpr": 0.06031004940845822,
101
+ "eval_violence,aiding_and_abetting,incitement/precision": 0.8399975948529854,
102
+ "eval_violence,aiding_and_abetting,incitement/recall": 0.8735617808904452,
103
+ "eval_violence,aiding_and_abetting,incitement/threshold": 0.5
104
+ }
test_results.json ADDED
@@ -0,0 +1,103 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "test_accuracy": 0.6766379207090669,
3
+ "test_animal_abuse/accuracy": 0.9931578632171517,
4
+ "test_animal_abuse/f1": 0.7676664972038637,
5
+ "test_animal_abuse/fpr": 0.005037019055710637,
6
+ "test_animal_abuse/precision": 0.6945722171113156,
7
+ "test_animal_abuse/recall": 0.8579545454545454,
8
+ "test_animal_abuse/threshold": 0.5,
9
+ "test_child_abuse/accuracy": 0.9963169241825368,
10
+ "test_child_abuse/f1": 0.6070287539936102,
11
+ "test_child_abuse/fpr": 0.0012642986152919908,
12
+ "test_child_abuse/precision": 0.6934306569343066,
13
+ "test_child_abuse/recall": 0.5397727272727273,
14
+ "test_child_abuse/threshold": 0.5,
15
+ "test_controversial_topics,politics/accuracy": 0.9715085638998683,
16
+ "test_controversial_topics,politics/f1": 0.45519610649871173,
17
+ "test_controversial_topics,politics/fpr": 0.011352768779885837,
18
+ "test_controversial_topics,politics/precision": 0.519268451992162,
19
+ "test_controversial_topics,politics/recall": 0.40519877675840976,
20
+ "test_controversial_topics,politics/threshold": 0.5,
21
+ "test_discrimination,stereotype,injustice/accuracy": 0.9543957360162894,
22
+ "test_discrimination,stereotype,injustice/f1": 0.7218772826880935,
23
+ "test_discrimination,stereotype,injustice/fpr": 0.023755877742946668,
24
+ "test_discrimination,stereotype,injustice/precision": 0.7309541420118343,
25
+ "test_discrimination,stereotype,injustice/recall": 0.713023088023088,
26
+ "test_discrimination,stereotype,injustice/threshold": 0.5,
27
+ "test_drug_abuse,weapons,banned_substance/accuracy": 0.9734848484848485,
28
+ "test_drug_abuse,weapons,banned_substance/f1": 0.7634566582075598,
29
+ "test_drug_abuse,weapons,banned_substance/fpr": 0.014630821418381233,
30
+ "test_drug_abuse,weapons,banned_substance/precision": 0.755884686590849,
31
+ "test_drug_abuse,weapons,banned_substance/recall": 0.7711818672423097,
32
+ "test_drug_abuse,weapons,banned_substance/threshold": 0.5,
33
+ "test_financial_crime,property_crime,theft/accuracy": 0.9579590370104204,
34
+ "test_financial_crime,property_crime,theft/f1": 0.7996575342465754,
35
+ "test_financial_crime,property_crime,theft/fpr": 0.0271945610877824,
36
+ "test_financial_crime,property_crime,theft/precision": 0.7744610281923715,
37
+ "test_financial_crime,property_crime,theft/recall": 0.8265486725663717,
38
+ "test_financial_crime,property_crime,theft/threshold": 0.5,
39
+ "test_flagged/accuracy": 0.8487393699844292,
40
+ "test_flagged/aucpr": 0.9047138471115324,
41
+ "test_flagged/f1": 0.8630417395312266,
42
+ "test_flagged/fpr": 0.15492622560685337,
43
+ "test_flagged/precision": 0.8747698480309983,
44
+ "test_flagged/recall": 0.8516239499170635,
45
+ "test_hate_speech,offensive_language/accuracy": 0.9497844053180021,
46
+ "test_hate_speech,offensive_language/f1": 0.686307519640853,
47
+ "test_hate_speech,offensive_language/fpr": 0.017700424021299645,
48
+ "test_hate_speech,offensive_language/precision": 0.7730720606826802,
49
+ "test_hate_speech,offensive_language/recall": 0.6170534813319879,
50
+ "test_hate_speech,offensive_language/threshold": 0.5,
51
+ "test_loss": 0.08679678291082382,
52
+ "test_macro_f1": 0.6252685023565255,
53
+ "test_macro_precision": 0.6847558918566842,
54
+ "test_macro_recall": 0.6093259557964672,
55
+ "test_micro_f1": 0.7468949012290593,
56
+ "test_micro_precision": 0.7705170480223624,
57
+ "test_micro_recall": 0.724678064292686,
58
+ "test_misinformation_regarding_ethics,laws_and_safety/accuracy": 0.987633249490957,
59
+ "test_misinformation_regarding_ethics,laws_and_safety/f1": 0.05707762557077625,
60
+ "test_misinformation_regarding_ethics,laws_and_safety/fpr": 0.0005304959379168174,
61
+ "test_misinformation_regarding_ethics,laws_and_safety/precision": 0.4166666666666667,
62
+ "test_misinformation_regarding_ethics,laws_and_safety/recall": 0.030637254901960783,
63
+ "test_misinformation_regarding_ethics,laws_and_safety/threshold": 0.5,
64
+ "test_non_violent_unethical_behavior/accuracy": 0.8818571086357647,
65
+ "test_non_violent_unethical_behavior/f1": 0.6774575924790517,
66
+ "test_non_violent_unethical_behavior/fpr": 0.050999362507968556,
67
+ "test_non_violent_unethical_behavior/precision": 0.7528845280276187,
68
+ "test_non_violent_unethical_behavior/recall": 0.6157675731906672,
69
+ "test_non_violent_unethical_behavior/threshold": 0.5,
70
+ "test_privacy_violation/accuracy": 0.9793687866810397,
71
+ "test_privacy_violation/f1": 0.8086111111111111,
72
+ "test_privacy_violation/fpr": 0.01268442169778534,
73
+ "test_privacy_violation/precision": 0.7837910608508347,
74
+ "test_privacy_violation/recall": 0.8350545037292025,
75
+ "test_privacy_violation/threshold": 0.5,
76
+ "test_runtime": 664.4818,
77
+ "test_samples_per_second": 100.517,
78
+ "test_self_harm/accuracy": 0.9963917834471194,
79
+ "test_self_harm/f1": 0.714792899408284,
80
+ "test_self_harm/fpr": 0.0011910505367265691,
81
+ "test_self_harm/precision": 0.7926509186351706,
82
+ "test_self_harm/recall": 0.6508620689655172,
83
+ "test_self_harm/threshold": 0.5,
84
+ "test_sexually_explicit,adult_content/accuracy": 0.9838902862618277,
85
+ "test_sexually_explicit,adult_content/f1": 0.6515544041450777,
86
+ "test_sexually_explicit,adult_content/fpr": 0.009186954524575089,
87
+ "test_sexually_explicit,adult_content/precision": 0.6264009962640099,
88
+ "test_sexually_explicit,adult_content/recall": 0.6788124156545209,
89
+ "test_sexually_explicit,adult_content/threshold": 0.5,
90
+ "test_steps_per_second": 1.571,
91
+ "test_terrorism,organized_crime/accuracy": 0.9913911845730028,
92
+ "test_terrorism,organized_crime/f1": 0.1958041958041958,
93
+ "test_terrorism,organized_crime/fpr": 0.001343680173923544,
94
+ "test_terrorism,organized_crime/precision": 0.44025157232704404,
95
+ "test_terrorism,organized_crime/recall": 0.12589928057553956,
96
+ "test_terrorism,organized_crime/threshold": 0.5,
97
+ "test_violence,aiding_and_abetting,incitement/accuracy": 0.9157533836387591,
98
+ "test_violence,aiding_and_abetting,incitement/f1": 0.8472708519935944,
99
+ "test_violence,aiding_and_abetting,incitement/fpr": 0.06457640343312376,
100
+ "test_violence,aiding_and_abetting,incitement/precision": 0.8322934997067136,
101
+ "test_violence,aiding_and_abetting,incitement/recall": 0.8627971254836927,
102
+ "test_violence,aiding_and_abetting,incitement/threshold": 0.5
103
+ }
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 4.000946297610598,
3
+ "total_flos": 0.0,
4
+ "train_loss": 0.09237335174007019,
5
+ "train_runtime": 91030.0055,
6
+ "train_samples_per_second": 59.433,
7
+ "train_steps_per_second": 0.929
8
+ }
trainer_state.json ADDED
The diff for this file is too large to render. See raw diff