kbourro commited on
Commit
5f029c0
·
1 Parent(s): 061ed03

Add calibration.json for temperature scaling metrics, remove predictions_val.csv, and update thresholds in results.json and threshold.json. Update README.md to reflect changes in metrics and file structure.

Browse files
.ipynb_checkpoints/README-checkpoint.md DELETED
@@ -1,167 +0,0 @@
1
- # AI Detector LoRA (DeBERTa-v3-large)
2
-
3
- LoRA adapter for binary AI-text vs Human-text detection, trained on ~2.3M English samples
4
- (`label: 1 = AI, 0 = Human`) using `microsoft/deberta-v3-large` as the base model.
5
-
6
- - **Base model:** `microsoft/deberta-v3-large`
7
- - **Task:** Binary classification (AI vs Human)
8
- - **Head:** Single-logit + `BCEWithLogitsLoss`
9
- - **Adapter type:** LoRA (`peft`)
10
- - **Hardware:** H100 SXM, bf16, multi-GPU
11
- - **Final decision threshold:** **0.9033** (max-F1 on validation)
12
-
13
- ---
14
-
15
- ## Files in this repo
16
-
17
- - `adapter/` – LoRA weights saved with `peft_model.save_pretrained(...)`
18
- - `threshold.json` – chosen deployment threshold and validation F1
19
- - `results.json` – hyperparameters, validation threshold search, test metrics
20
- - `training_log_history.csv` – raw Trainer log history
21
- - `predictions_val.csv` – validation probabilities and labels
22
- - `predictions_test.csv` – test probabilities and labels
23
- - `figures/` – training and evaluation plots
24
- - `README.md` – this file
25
-
26
- ---
27
-
28
- ## Metrics (test set)
29
-
30
- Using threshold **0.9033**:
31
-
32
- | Metric | Value |
33
- |--------------|---------|
34
- | AUROC | 0.9970 |
35
- | Average Precision (AP) | 0.9966 |
36
- | F1 | 0.9740 |
37
- | Accuracy | 0.9767 |
38
- | Precision | 0.9857 |
39
- | Recall | 0.9625 |
40
- | Specificity | 0.9884 |
41
-
42
- Confusion matrix (test):
43
-
44
- - **True Negatives (Human correctly)**: 123,399
45
- - **False Positives (Human → AI)**: 1,449
46
- - **False Negatives (AI → Human)**: 3,882
47
- - **True Positives (AI correctly)**: 99,657
48
-
49
- ---
50
-
51
- ## Plots
52
-
53
- ### Training & validation
54
-
55
- - Learning curves:
56
-
57
- ![Learning curves](./figures/fig_learning_curves.png)
58
-
59
- - Eval metrics over time:
60
-
61
- ![Eval metrics](./figures/fig_eval_metrics.png)
62
-
63
- ### Validation set
64
-
65
- - ROC:
66
-
67
- ![ROC (val)](./figures/fig_roc_val.png)
68
-
69
- - Precision–Recall:
70
-
71
- ![PR (val)](./figures/fig_pr_val.png)
72
-
73
- - Calibration curve:
74
-
75
- ![Calibration (val)](./figures/fig_calibration_val.png)
76
-
77
- - F1 vs threshold:
78
-
79
- ![F1 vs threshold (val)](./figures/fig_threshold_f1_val.png)
80
-
81
- ### Test set
82
-
83
- - ROC:
84
-
85
- ![ROC (test)](./figures/fig_roc_test.png)
86
-
87
- - Precision–Recall:
88
-
89
- ![PR (test)](./figures/fig_pr_test.png)
90
-
91
- - Calibration curve:
92
-
93
- ![Calibration (test)](./figures/fig_calibration_test.png)
94
-
95
- - Confusion matrix:
96
-
97
- ![Confusion matrix (test)](./figures/fig_confusion_test.png)
98
-
99
- ---
100
-
101
- ## Usage
102
-
103
- ### Load base + LoRA adapter
104
-
105
- ```python
106
- from transformers import AutoTokenizer, AutoModelForSequenceClassification
107
- from peft import PeftModel
108
- import torch
109
- import json
110
-
111
- base_model_id = "microsoft/deberta-v3-large"
112
- adapter_id = "<your-username>/<your-private-repo>"
113
-
114
- tokenizer = AutoTokenizer.from_pretrained(base_model_id)
115
-
116
- base_model = AutoModelForSequenceClassification.from_pretrained(
117
- base_model_id,
118
- num_labels=1, # single logit for BCEWithLogitsLoss
119
- )
120
- model = PeftModel.from_pretrained(base_model, adapter_id)
121
- model.eval()
122
- ````
123
-
124
- ### Inference with threshold
125
-
126
- ```python
127
- # load threshold
128
- with open("threshold.json") as f:
129
- thr = json.load(f)["threshold"] # 0.9033
130
-
131
- def predict_proba(texts):
132
- enc = tokenizer(
133
- texts,
134
- padding=True,
135
- truncation=True,
136
- max_length=512,
137
- return_tensors="pt",
138
- )
139
- with torch.no_grad():
140
- logits = model(**enc).logits.squeeze(-1)
141
- probs = torch.sigmoid(logits)
142
- return probs.cpu().numpy()
143
-
144
- def predict_label(texts, threshold=thr):
145
- probs = predict_proba(texts)
146
- return (probs >= threshold).astype(int)
147
-
148
- # example
149
- texts = ["Some example text to classify"]
150
- probs = predict_proba(texts)
151
- labels = predict_label(texts)
152
- print(probs, labels) # label 1 = AI, 0 = Human
153
- ```
154
-
155
- ---
156
-
157
- ## Notes
158
-
159
- * Classifier head is **trainable** together with LoRA layers (unfrozen after applying PEFT).
160
- * Training used:
161
-
162
- * `bf16=True`
163
- * `optim="adamw_torch_fused"`
164
- * cosine-with-restarts scheduler
165
- * LR scaled down from HPO to account for full-dataset (~14k steps).
166
- * Threshold `0.9033` was chosen as the **max-F1** point on the validation set.
167
- You can adjust it if you prefer fewer false positives or fewer false negatives.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
.ipynb_checkpoints/predictions_test-checkpoint.csv DELETED
The diff for this file is too large to render. See raw diff
 
.ipynb_checkpoints/predictions_val-checkpoint.csv DELETED
The diff for this file is too large to render. See raw diff
 
.ipynb_checkpoints/results-checkpoint.json DELETED
@@ -1,94 +0,0 @@
1
- {
2
- "hyperparameters": {
3
- "include_attention_output_dense": false,
4
- "learning_rate_sampled": 0.00044569416489470884,
5
- "weight_decay": 0.022491619139739856,
6
- "warmup_ratio": 0.0463266472104081,
7
- "lr_scheduler_num_cycles": 1,
8
- "per_device_train_batch_size": 8,
9
- "gradient_accumulation_steps": 4,
10
- "num_train_epochs": 2,
11
- "lora_r": 32,
12
- "lora_alpha": 128,
13
- "lora_dropout": 0.0,
14
- "lora_target_modules": [
15
- "query_proj",
16
- "key_proj",
17
- "value_proj"
18
- ],
19
- "learning_rate": 4.456941648947089e-05,
20
- "lr_scheduler_type": "cosine_with_restarts",
21
- "max_grad_norm": 0.5,
22
- "optim": "adamw_torch_fused"
23
- },
24
- "threshold_optimization": {
25
- "max_f1": {
26
- "threshold": 0.9032942056655884,
27
- "metrics": {
28
- "threshold": 0.9032942056655884,
29
- "auroc": 0.9969044529302581,
30
- "average_precision": 0.9965060417039346,
31
- "f1": 0.9734939759036144,
32
- "accuracy": 0.9762551119595773,
33
- "precision": 0.9854536098796707,
34
- "recall": 0.9618211495185389,
35
- "specificity": 0.9882255881198587,
36
- "precision_human": 0.9689546846776094,
37
- "recall_human": 0.9882255881198587,
38
- "precision_ai": 0.9854536098796707,
39
- "recall_ai": 0.9618211495185389,
40
- "confusion_matrix": {
41
- "true_negative": 123377,
42
- "false_positive": 1470,
43
- "false_negative": 3953,
44
- "true_positive": 99586
45
- }
46
- }
47
- },
48
- "precision_at_95recall": {
49
- "threshold": 5.1442217227304354e-05,
50
- "metrics": {
51
- "threshold": 5.1442217227304354e-05,
52
- "auroc": 0.9969044529302581,
53
- "average_precision": 0.9965060417039346,
54
- "f1": 0.6238698501167432,
55
- "accuracy": 0.45335090592242955,
56
- "precision": 0.45335090592242955,
57
- "recall": 1.0,
58
- "specificity": 0.0,
59
- "precision_human": 0.0,
60
- "recall_human": 0.0,
61
- "precision_ai": 0.45335090592242955,
62
- "recall_ai": 1.0,
63
- "confusion_matrix": {
64
- "true_negative": 0,
65
- "false_positive": 124847,
66
- "false_negative": 0,
67
- "true_positive": 103539
68
- }
69
- }
70
- }
71
- },
72
- "test_metrics": {
73
- "threshold": 0.9032942056655884,
74
- "auroc": 0.9970131530896283,
75
- "average_precision": 0.9966291954050931,
76
- "f1": 0.9739500109946493,
77
- "accuracy": 0.976658040956797,
78
- "precision": 0.9856685063200997,
79
- "recall": 0.9625068814649552,
80
- "specificity": 0.9883938869665513,
81
- "precision_human": 0.9695005538925684,
82
- "recall_human": 0.9883938869665513,
83
- "precision_ai": 0.9856685063200997,
84
- "recall_ai": 0.9625068814649552,
85
- "confusion_matrix": {
86
- "true_negative": 123399,
87
- "false_positive": 1449,
88
- "false_negative": 3882,
89
- "true_positive": 99657
90
- }
91
- },
92
- "timestamp": "20251113_111139",
93
- "seed": 42
94
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
.ipynb_checkpoints/threshold-checkpoint.json DELETED
@@ -1,9 +0,0 @@
1
- {
2
- "threshold": 0.9032942056655884,
3
- "method": "max_f1",
4
- "validation_f1": 0.9734939759036144,
5
- "alternative_thresholds": {
6
- "max_f1": 0.9032942056655884,
7
- "precision_at_95recall": 5.1442217227304354e-05
8
- }
9
- }
 
 
 
 
 
 
 
 
 
 
README.md CHANGED
@@ -28,17 +28,19 @@ LoRA adapter for binary AI-text vs Human-text detection, trained on ~2.3M Englis
28
  - **Head:** Single-logit + `BCEWithLogitsLoss`
29
  - **Adapter type:** LoRA (`peft`)
30
  - **Hardware:** H100 SXM, bf16, multi-GPU
31
- - **Final decision threshold:** **0.9033** (max-F1 on validation)
32
 
33
  ---
34
 
35
  ## Files in this repo
36
 
37
  - `adapter/` – LoRA weights saved with `peft_model.save_pretrained(...)`
 
38
  - `threshold.json` – chosen deployment threshold and validation F1
 
39
  - `results.json` – hyperparameters, validation threshold search, test metrics
40
  - `training_log_history.csv` – raw Trainer log history
41
- - `predictions_val.csv` – validation probabilities and labels
42
  - `predictions_test.csv` – test probabilities and labels
43
  - `figures/` – training and evaluation plots
44
  - `README.md` – this file
@@ -47,24 +49,32 @@ LoRA adapter for binary AI-text vs Human-text detection, trained on ~2.3M Englis
47
 
48
  ## Metrics (test set)
49
 
50
- Using threshold **0.9033**:
51
 
52
- | Metric | Value |
53
- |--------------|---------|
54
- | AUROC | 0.9970 |
55
- | Average Precision (AP) | 0.9966 |
56
- | F1 | 0.9740 |
57
- | Accuracy | 0.9767 |
58
- | Precision | 0.9857 |
59
- | Recall | 0.9625 |
60
- | Specificity | 0.9884 |
61
 
62
  Confusion matrix (test):
63
 
64
- - **True Negatives (Human correctly)**: 123,399
65
- - **False Positives (Human → AI)**: 1,449
66
- - **False Negatives (AI → Human)**: 3,882
67
- - **True Positives (AI correctly)**: 99,657
 
 
 
 
 
 
 
 
68
 
69
  ---
70
 
@@ -84,19 +94,19 @@ Confusion matrix (test):
84
 
85
  - ROC:
86
 
87
- ![ROC (val)](./figures/fig_roc_val.png)
88
 
89
  - Precision–Recall:
90
 
91
- ![PR (val)](./figures/fig_pr_val.png)
92
 
93
  - Calibration curve:
94
 
95
- ![Calibration (val)](./figures/fig_calibration_val.png)
96
 
97
  - F1 vs threshold:
98
 
99
- ![F1 vs threshold (val)](./figures/fig_threshold_f1_val.png)
100
 
101
  ### Test set
102
 
@@ -129,7 +139,7 @@ import torch
129
  import json
130
 
131
  base_model_id = "microsoft/deberta-v3-large"
132
- adapter_id = "stealthcode/ai-detection"
133
 
134
  tokenizer = AutoTokenizer.from_pretrained(base_model_id)
135
 
@@ -139,14 +149,14 @@ base_model = AutoModelForSequenceClassification.from_pretrained(
139
  )
140
  model = PeftModel.from_pretrained(base_model, adapter_id)
141
  model.eval()
142
- ````
143
 
144
  ### Inference with threshold
145
 
146
  ```python
147
  # load threshold
148
  with open("threshold.json") as f:
149
- thr = json.load(f)["threshold"] # 0.9033
150
 
151
  def predict_proba(texts):
152
  enc = tokenizer(
@@ -172,16 +182,54 @@ labels = predict_label(texts)
172
  print(probs, labels) # label 1 = AI, 0 = Human
173
  ```
174
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
175
  ---
176
 
177
  ## Notes
178
 
179
- * Classifier head is **trainable** together with LoRA layers (unfrozen after applying PEFT).
180
- * Training used:
 
 
 
 
 
181
 
182
- * `bf16=True`
183
- * `optim="adamw_torch_fused"`
184
- * cosine-with-restarts scheduler
185
- * LR scaled down from HPO to account for full-dataset (~14k steps).
186
- * Threshold `0.9033` was chosen as the **max-F1** point on the validation set.
187
  You can adjust it if you prefer fewer false positives or fewer false negatives.
 
28
  - **Head:** Single-logit + `BCEWithLogitsLoss`
29
  - **Adapter type:** LoRA (`peft`)
30
  - **Hardware:** H100 SXM, bf16, multi-GPU
31
+ - **Final decision threshold:** **0.9284** (max-F1 on calibration set)
32
 
33
  ---
34
 
35
  ## Files in this repo
36
 
37
  - `adapter/` – LoRA weights saved with `peft_model.save_pretrained(...)`
38
+ - `merged_model/` – fully merged model (base + LoRA) for standalone use
39
  - `threshold.json` – chosen deployment threshold and validation F1
40
+ - `calibration.json` – temperature scaling parameters and calibration metrics
41
  - `results.json` – hyperparameters, validation threshold search, test metrics
42
  - `training_log_history.csv` – raw Trainer log history
43
+ - `predictions_calib.csv` – calibration-set probabilities and labels
44
  - `predictions_test.csv` – test probabilities and labels
45
  - `figures/` – training and evaluation plots
46
  - `README.md` – this file
 
49
 
50
  ## Metrics (test set)
51
 
52
+ Using threshold **0.9284**:
53
 
54
+ | Metric | Value |
55
+ | ---------------------- | ------ |
56
+ | AUROC | 0.9979 |
57
+ | Average Precision (AP) | 0.9977 |
58
+ | F1 | 0.9773 |
59
+ | Accuracy | 0.9797 |
60
+ | Precision | 0.9909 |
61
+ | Recall | 0.9640 |
62
+ | Specificity | 0.9927 |
63
 
64
  Confusion matrix (test):
65
 
66
+ - **True Negatives (Human correctly)**: 123,936
67
+ - **False Positives (Human → AI)**: 912
68
+ - **False Negatives (AI → Human)**: 3,723
69
+ - **True Positives (AI correctly)**: 99,816
70
+
71
+ ### Calibration
72
+
73
+ - **Method:** temperature scaling
74
+ - **Temperature (T):** 1.2807
75
+ - **Calibration set:** calibration
76
+ - Test ECE: 0.0119 → 0.0159 (after calibration)
77
+ - Test Brier: 0.01812 → 0.01829 (after calibration)
78
 
79
  ---
80
 
 
94
 
95
  - ROC:
96
 
97
+ ![ROC (calib)](./figures/fig_roc_calib.png)
98
 
99
  - Precision–Recall:
100
 
101
+ ![PR (calib)](./figures/fig_pr_calib.png)
102
 
103
  - Calibration curve:
104
 
105
+ ![Calibration (calib)](./figures/fig_calibration_calib.png)
106
 
107
  - F1 vs threshold:
108
 
109
+ ![F1 vs threshold (calib)](./figures/fig_threshold_f1_calib.png)
110
 
111
  ### Test set
112
 
 
139
  import json
140
 
141
  base_model_id = "microsoft/deberta-v3-large"
142
+ adapter_id = "stealthcode/ai-detection" # or local: "./adapter"
143
 
144
  tokenizer = AutoTokenizer.from_pretrained(base_model_id)
145
 
 
149
  )
150
  model = PeftModel.from_pretrained(base_model, adapter_id)
151
  model.eval()
152
+ ```
153
 
154
  ### Inference with threshold
155
 
156
  ```python
157
  # load threshold
158
  with open("threshold.json") as f:
159
+ thr = json.load(f)["threshold"] # 0.9284
160
 
161
  def predict_proba(texts):
162
  enc = tokenizer(
 
182
  print(probs, labels) # label 1 = AI, 0 = Human
183
  ```
184
 
185
+ ### Load merged model (no PEFT required)
186
+
187
+ ```python
188
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
189
+ import torch, json
190
+
191
+ model_dir = "./merged_model"
192
+ tokenizer = AutoTokenizer.from_pretrained(model_dir)
193
+ model = AutoModelForSequenceClassification.from_pretrained(model_dir)
194
+ model.eval()
195
+
196
+ with open("threshold.json") as f:
197
+ thr = json.load(f)["threshold"] # 0.9284
198
+
199
+ def predict_proba(texts):
200
+ enc = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
201
+ with torch.no_grad():
202
+ logits = model(**enc).logits.squeeze(-1)
203
+ probs = torch.sigmoid(logits)
204
+ return probs.cpu().numpy()
205
+ ```
206
+
207
+ ### Optional: apply temperature scaling to logits
208
+
209
+ ```python
210
+ import json
211
+ with open("calibration.json") as f:
212
+ T = json.load(f)["temperature"] # e.g., 1.2807
213
+
214
+ def predict_proba_calibrated(texts):
215
+ enc = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
216
+ with torch.no_grad():
217
+ logits = model(**enc).logits.squeeze(-1)
218
+ probs = torch.sigmoid(logits / T)
219
+ return probs.cpu().numpy()
220
+ ```
221
+
222
  ---
223
 
224
  ## Notes
225
 
226
+ - Classifier head is **trainable** together with LoRA layers (unfrozen after applying PEFT).
227
+ - Training used:
228
+
229
+ - `bf16=True`
230
+ - `optim="adamw_torch_fused"`
231
+ - cosine-with-restarts scheduler
232
+ - LR scaled down from HPO to account for full-dataset (~14k steps).
233
 
234
+ - Threshold `0.9284` was chosen as the **max-F1** point on the calibration set.
 
 
 
 
235
  You can adjust it if you prefer fewer false positives or fewer false negatives.
adapter/.ipynb_checkpoints/README-checkpoint.md DELETED
@@ -1,206 +0,0 @@
1
- ---
2
- base_model: microsoft/deberta-v3-large
3
- library_name: peft
4
- tags:
5
- - base_model:adapter:microsoft/deberta-v3-large
6
- - lora
7
- - transformers
8
- ---
9
-
10
- # Model Card for Model ID
11
-
12
- <!-- Provide a quick summary of what the model is/does. -->
13
-
14
-
15
-
16
- ## Model Details
17
-
18
- ### Model Description
19
-
20
- <!-- Provide a longer summary of what this model is. -->
21
-
22
-
23
-
24
- - **Developed by:** [More Information Needed]
25
- - **Funded by [optional]:** [More Information Needed]
26
- - **Shared by [optional]:** [More Information Needed]
27
- - **Model type:** [More Information Needed]
28
- - **Language(s) (NLP):** [More Information Needed]
29
- - **License:** [More Information Needed]
30
- - **Finetuned from model [optional]:** [More Information Needed]
31
-
32
- ### Model Sources [optional]
33
-
34
- <!-- Provide the basic links for the model. -->
35
-
36
- - **Repository:** [More Information Needed]
37
- - **Paper [optional]:** [More Information Needed]
38
- - **Demo [optional]:** [More Information Needed]
39
-
40
- ## Uses
41
-
42
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
43
-
44
- ### Direct Use
45
-
46
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
47
-
48
- [More Information Needed]
49
-
50
- ### Downstream Use [optional]
51
-
52
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
53
-
54
- [More Information Needed]
55
-
56
- ### Out-of-Scope Use
57
-
58
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
59
-
60
- [More Information Needed]
61
-
62
- ## Bias, Risks, and Limitations
63
-
64
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
65
-
66
- [More Information Needed]
67
-
68
- ### Recommendations
69
-
70
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
71
-
72
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
73
-
74
- ## How to Get Started with the Model
75
-
76
- Use the code below to get started with the model.
77
-
78
- [More Information Needed]
79
-
80
- ## Training Details
81
-
82
- ### Training Data
83
-
84
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
85
-
86
- [More Information Needed]
87
-
88
- ### Training Procedure
89
-
90
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
91
-
92
- #### Preprocessing [optional]
93
-
94
- [More Information Needed]
95
-
96
-
97
- #### Training Hyperparameters
98
-
99
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
100
-
101
- #### Speeds, Sizes, Times [optional]
102
-
103
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
104
-
105
- [More Information Needed]
106
-
107
- ## Evaluation
108
-
109
- <!-- This section describes the evaluation protocols and provides the results. -->
110
-
111
- ### Testing Data, Factors & Metrics
112
-
113
- #### Testing Data
114
-
115
- <!-- This should link to a Dataset Card if possible. -->
116
-
117
- [More Information Needed]
118
-
119
- #### Factors
120
-
121
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
122
-
123
- [More Information Needed]
124
-
125
- #### Metrics
126
-
127
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
128
-
129
- [More Information Needed]
130
-
131
- ### Results
132
-
133
- [More Information Needed]
134
-
135
- #### Summary
136
-
137
-
138
-
139
- ## Model Examination [optional]
140
-
141
- <!-- Relevant interpretability work for the model goes here -->
142
-
143
- [More Information Needed]
144
-
145
- ## Environmental Impact
146
-
147
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
148
-
149
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
150
-
151
- - **Hardware Type:** [More Information Needed]
152
- - **Hours used:** [More Information Needed]
153
- - **Cloud Provider:** [More Information Needed]
154
- - **Compute Region:** [More Information Needed]
155
- - **Carbon Emitted:** [More Information Needed]
156
-
157
- ## Technical Specifications [optional]
158
-
159
- ### Model Architecture and Objective
160
-
161
- [More Information Needed]
162
-
163
- ### Compute Infrastructure
164
-
165
- [More Information Needed]
166
-
167
- #### Hardware
168
-
169
- [More Information Needed]
170
-
171
- #### Software
172
-
173
- [More Information Needed]
174
-
175
- ## Citation [optional]
176
-
177
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
178
-
179
- **BibTeX:**
180
-
181
- [More Information Needed]
182
-
183
- **APA:**
184
-
185
- [More Information Needed]
186
-
187
- ## Glossary [optional]
188
-
189
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
190
-
191
- [More Information Needed]
192
-
193
- ## More Information [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Authors [optional]
198
-
199
- [More Information Needed]
200
-
201
- ## Model Card Contact
202
-
203
- [More Information Needed]
204
- ### Framework versions
205
-
206
- - PEFT 0.17.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
adapter/.ipynb_checkpoints/adapter_config-checkpoint.json DELETED
@@ -1,41 +0,0 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "microsoft/deberta-v3-large",
5
- "bias": "none",
6
- "corda_config": null,
7
- "eva_config": null,
8
- "exclude_modules": null,
9
- "fan_in_fan_out": false,
10
- "inference_mode": true,
11
- "init_lora_weights": true,
12
- "layer_replication": null,
13
- "layers_pattern": null,
14
- "layers_to_transform": null,
15
- "loftq_config": {},
16
- "lora_alpha": 128,
17
- "lora_bias": false,
18
- "lora_dropout": 0.0,
19
- "megatron_config": null,
20
- "megatron_core": "megatron.core",
21
- "modules_to_save": [
22
- "classifier",
23
- "score"
24
- ],
25
- "peft_type": "LORA",
26
- "qalora_group_size": 16,
27
- "r": 32,
28
- "rank_pattern": {},
29
- "revision": null,
30
- "target_modules": [
31
- "query_proj",
32
- "key_proj",
33
- "value_proj"
34
- ],
35
- "target_parameters": null,
36
- "task_type": "SEQ_CLS",
37
- "trainable_token_indices": null,
38
- "use_dora": false,
39
- "use_qalora": false,
40
- "use_rslora": false
41
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
adapter/README.md CHANGED
@@ -203,4 +203,4 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
203
  [More Information Needed]
204
  ### Framework versions
205
 
206
- - PEFT 0.17.1
 
203
  [More Information Needed]
204
  ### Framework versions
205
 
206
+ - PEFT 0.18.0
adapter/adapter_config.json CHANGED
@@ -1,9 +1,12 @@
1
  {
 
2
  "alpha_pattern": {},
 
3
  "auto_mapping": null,
4
  "base_model_name_or_path": "microsoft/deberta-v3-large",
5
  "bias": "none",
6
  "corda_config": null,
 
7
  "eva_config": null,
8
  "exclude_modules": null,
9
  "fan_in_fan_out": false,
@@ -19,18 +22,21 @@
19
  "megatron_config": null,
20
  "megatron_core": "megatron.core",
21
  "modules_to_save": [
 
 
22
  "classifier",
23
  "score"
24
  ],
25
  "peft_type": "LORA",
 
26
  "qalora_group_size": 16,
27
  "r": 32,
28
  "rank_pattern": {},
29
  "revision": null,
30
  "target_modules": [
31
  "query_proj",
32
- "key_proj",
33
- "value_proj"
34
  ],
35
  "target_parameters": null,
36
  "task_type": "SEQ_CLS",
 
1
  {
2
+ "alora_invocation_tokens": null,
3
  "alpha_pattern": {},
4
+ "arrow_config": null,
5
  "auto_mapping": null,
6
  "base_model_name_or_path": "microsoft/deberta-v3-large",
7
  "bias": "none",
8
  "corda_config": null,
9
+ "ensure_weight_tying": false,
10
  "eva_config": null,
11
  "exclude_modules": null,
12
  "fan_in_fan_out": false,
 
22
  "megatron_config": null,
23
  "megatron_core": "megatron.core",
24
  "modules_to_save": [
25
+ "classifier",
26
+ "pooler.dense",
27
  "classifier",
28
  "score"
29
  ],
30
  "peft_type": "LORA",
31
+ "peft_version": "0.18.0",
32
  "qalora_group_size": 16,
33
  "r": 32,
34
  "rank_pattern": {},
35
  "revision": null,
36
  "target_modules": [
37
  "query_proj",
38
+ "value_proj",
39
+ "key_proj"
40
  ],
41
  "target_parameters": null,
42
  "task_type": "SEQ_CLS",
adapter/adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fe302bb59b8aa23d2077aafa0f9403ded12e36f656ded2725bd708a446e3daf9
3
- size 18900396
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2320198a394f889f1be50d439b5217d996e84da64a4a65671bf653607879cfcb
3
+ size 23099012
calibration.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "temperature": 1.2806789875030518,
3
+ "method": "temperature_scaling",
4
+ "calibration_set": "calibration",
5
+ "calibration_metrics": {
6
+ "temperature": 1.2806789875030518,
7
+ "optimization_method": "LBFGS_logspace",
8
+ "uncalibrated_nll": 0.06460460661246972,
9
+ "calibrated_nll": 0.06279846573841724,
10
+ "uncalibrated_ece": 0.012124871567496009,
11
+ "calibrated_ece": 0.016240862688628014,
12
+ "uncalibrated_brier": 0.01822748637167701,
13
+ "calibrated_brier": 0.018437309998858068,
14
+ "nll_improvement": 0.001806140874052481,
15
+ "ece_improvement": -0.004115991121132005,
16
+ "brier_improvement": -0.00020982362718105843
17
+ },
18
+ "test_metrics": {
19
+ "ece_before": 0.011862174308705089,
20
+ "ece_after": 0.015908939599173937,
21
+ "ece_improvement": -0.004046765290468848,
22
+ "brier_before": 0.01812282837704726,
23
+ "brier_after": 0.018294590049400802,
24
+ "brier_improvement": -0.0001717616723535438
25
+ }
26
+ }
merged_model/added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "[MASK]": 128000
3
+ }
merged_model/config.json ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "DebertaV2ForSequenceClassification"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.1,
6
+ "bos_token_id": 1,
7
+ "dtype": "float32",
8
+ "eos_token_id": 2,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 1024,
12
+ "id2label": {
13
+ "0": "LABEL_0"
14
+ },
15
+ "initializer_range": 0.02,
16
+ "intermediate_size": 4096,
17
+ "label2id": {
18
+ "LABEL_0": 0
19
+ },
20
+ "layer_norm_eps": 1e-07,
21
+ "legacy": true,
22
+ "max_position_embeddings": 512,
23
+ "max_relative_positions": -1,
24
+ "model_type": "deberta-v2",
25
+ "norm_rel_ebd": "layer_norm",
26
+ "num_attention_heads": 16,
27
+ "num_hidden_layers": 24,
28
+ "pad_token_id": 0,
29
+ "pooler_dropout": 0,
30
+ "pooler_hidden_act": "gelu",
31
+ "pooler_hidden_size": 1024,
32
+ "pos_att_type": [
33
+ "p2c",
34
+ "c2p"
35
+ ],
36
+ "position_biased_input": false,
37
+ "position_buckets": 256,
38
+ "relative_attention": true,
39
+ "share_att_key": true,
40
+ "transformers_version": "4.57.1",
41
+ "type_vocab_size": 0,
42
+ "use_cache": false,
43
+ "vocab_size": 128100
44
+ }
merged_model/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f780611b3aca302b2c564ebcb57391dcced23c391e62f96ee4509fa71076185f
3
+ size 1740300340
merged_model/special_tokens_map.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "cls_token": {
10
+ "content": "[CLS]",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "eos_token": {
17
+ "content": "[SEP]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "mask_token": {
24
+ "content": "[MASK]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "pad_token": {
31
+ "content": "[PAD]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ },
37
+ "sep_token": {
38
+ "content": "[SEP]",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false
43
+ },
44
+ "unk_token": {
45
+ "content": "[UNK]",
46
+ "lstrip": false,
47
+ "normalized": true,
48
+ "rstrip": false,
49
+ "single_word": false
50
+ }
51
+ }
classifier_head.pt → merged_model/spm.model RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a30f288a0a619ac7013849cfd0bfc4e6f81073b3fdfd5b155b1354c404b81759
3
- size 6053
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c679fbf93643d19aab7ee10c0b99e460bdbc02fedf34b92b05af343b4af586fd
3
+ size 2464616
merged_model/tokenizer_config.json ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "[CLS]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "[SEP]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "[UNK]",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "128000": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "bos_token": "[CLS]",
45
+ "clean_up_tokenization_spaces": false,
46
+ "cls_token": "[CLS]",
47
+ "do_lower_case": false,
48
+ "eos_token": "[SEP]",
49
+ "extra_special_tokens": {},
50
+ "mask_token": "[MASK]",
51
+ "model_max_length": 1000000000000000019884624838656,
52
+ "pad_token": "[PAD]",
53
+ "sep_token": "[SEP]",
54
+ "sp_model_kwargs": {},
55
+ "split_by_punct": false,
56
+ "tokenizer_class": "DebertaV2Tokenizer",
57
+ "unk_token": "[UNK]",
58
+ "vocab_type": "spm"
59
+ }
predictions_calib.csv ADDED
The diff for this file is too large to render. See raw diff
 
predictions_test.csv CHANGED
The diff for this file is too large to render. See raw diff
 
predictions_val.csv DELETED
The diff for this file is too large to render. See raw diff
 
results.json CHANGED
@@ -16,79 +16,105 @@
16
  "key_proj",
17
  "value_proj"
18
  ],
19
- "learning_rate": 4.456941648947089e-05,
20
  "lr_scheduler_type": "cosine_with_restarts",
21
  "max_grad_norm": 0.5,
22
  "optim": "adamw_torch_fused"
23
  },
24
  "threshold_optimization": {
25
  "max_f1": {
26
- "threshold": 0.9032942056655884,
27
  "metrics": {
28
- "threshold": 0.9032942056655884,
29
- "auroc": 0.9969044529302581,
30
- "average_precision": 0.9965060417039346,
31
- "f1": 0.9734939759036144,
32
- "accuracy": 0.9762551119595773,
33
- "precision": 0.9854536098796707,
34
- "recall": 0.9618211495185389,
35
- "specificity": 0.9882255881198587,
36
- "precision_human": 0.9689546846776094,
37
- "recall_human": 0.9882255881198587,
38
- "precision_ai": 0.9854536098796707,
39
- "recall_ai": 0.9618211495185389,
40
  "confusion_matrix": {
41
- "true_negative": 123377,
42
- "false_positive": 1470,
43
- "false_negative": 3953,
44
- "true_positive": 99586
45
  }
46
  }
47
  },
48
  "precision_at_95recall": {
49
- "threshold": 5.1442217227304354e-05,
50
  "metrics": {
51
- "threshold": 5.1442217227304354e-05,
52
- "auroc": 0.9969044529302581,
53
- "average_precision": 0.9965060417039346,
54
- "f1": 0.6238698501167432,
55
- "accuracy": 0.45335090592242955,
56
- "precision": 0.45335090592242955,
57
  "recall": 1.0,
58
  "specificity": 0.0,
59
  "precision_human": 0.0,
60
  "recall_human": 0.0,
61
- "precision_ai": 0.45335090592242955,
62
  "recall_ai": 1.0,
63
  "confusion_matrix": {
64
  "true_negative": 0,
65
- "false_positive": 124847,
66
  "false_negative": 0,
67
- "true_positive": 103539
68
  }
69
  }
70
  }
71
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
72
  "test_metrics": {
73
- "threshold": 0.9032942056655884,
74
- "auroc": 0.9970131530896283,
75
- "average_precision": 0.9966291954050931,
76
- "f1": 0.9739500109946493,
77
- "accuracy": 0.976658040956797,
78
- "precision": 0.9856685063200997,
79
- "recall": 0.9625068814649552,
80
- "specificity": 0.9883938869665513,
81
- "precision_human": 0.9695005538925684,
82
- "recall_human": 0.9883938869665513,
83
- "precision_ai": 0.9856685063200997,
84
- "recall_ai": 0.9625068814649552,
85
  "confusion_matrix": {
86
- "true_negative": 123399,
87
- "false_positive": 1449,
88
- "false_negative": 3882,
89
- "true_positive": 99657
90
  }
91
  },
92
- "timestamp": "20251113_111139",
93
  "seed": 42
94
  }
 
16
  "key_proj",
17
  "value_proj"
18
  ],
19
+ "learning_rate": 0.0001554357238163802,
20
  "lr_scheduler_type": "cosine_with_restarts",
21
  "max_grad_norm": 0.5,
22
  "optim": "adamw_torch_fused"
23
  },
24
  "threshold_optimization": {
25
  "max_f1": {
26
+ "threshold": 0.9284088015556335,
27
  "metrics": {
28
+ "threshold": 0.9284088015556335,
29
+ "auroc": 0.9978600960936826,
30
+ "average_precision": 0.997597673288253,
31
+ "f1": 0.9773827668313225,
32
+ "accuracy": 0.9797765846236365,
33
+ "precision": 0.9912838341196921,
34
+ "recall": 0.9638661853653825,
35
+ "specificity": 0.9929714251386692,
36
+ "precision_human": 0.9707053998766749,
37
+ "recall_human": 0.9929714251386692,
38
+ "precision_ai": 0.9912838341196921,
39
+ "recall_ai": 0.9638661853653825,
40
  "confusion_matrix": {
41
+ "true_negative": 99176,
42
+ "false_positive": 702,
43
+ "false_negative": 2993,
44
+ "true_positive": 79838
45
  }
46
  }
47
  },
48
  "precision_at_95recall": {
49
+ "threshold": 1.9947297005273867e-06,
50
  "metrics": {
51
+ "threshold": 1.9947297005273867e-06,
52
+ "auroc": 0.9978600960936826,
53
+ "average_precision": 0.997597673288253,
54
+ "f1": 0.6238683437523537,
55
+ "accuracy": 0.45334931503100556,
56
+ "precision": 0.45334931503100556,
57
  "recall": 1.0,
58
  "specificity": 0.0,
59
  "precision_human": 0.0,
60
  "recall_human": 0.0,
61
+ "precision_ai": 0.45334931503100556,
62
  "recall_ai": 1.0,
63
  "confusion_matrix": {
64
  "true_negative": 0,
65
+ "false_positive": 99878,
66
  "false_negative": 0,
67
+ "true_positive": 82831
68
  }
69
  }
70
  }
71
  },
72
+ "calibration": {
73
+ "temperature": 1.2806789875030518,
74
+ "method": "temperature_scaling",
75
+ "calibration_set": "calibration",
76
+ "calibration_metrics": {
77
+ "temperature": 1.2806789875030518,
78
+ "optimization_method": "LBFGS_logspace",
79
+ "uncalibrated_nll": 0.06460460661246972,
80
+ "calibrated_nll": 0.06279846573841724,
81
+ "uncalibrated_ece": 0.012124871567496009,
82
+ "calibrated_ece": 0.016240862688628014,
83
+ "uncalibrated_brier": 0.01822748637167701,
84
+ "calibrated_brier": 0.018437309998858068,
85
+ "nll_improvement": 0.001806140874052481,
86
+ "ece_improvement": -0.004115991121132005,
87
+ "brier_improvement": -0.00020982362718105843
88
+ },
89
+ "test_metrics": {
90
+ "ece_before": 0.011862174308705089,
91
+ "ece_after": 0.015908939599173937,
92
+ "ece_improvement": -0.004046765290468848,
93
+ "brier_before": 0.01812282837704726,
94
+ "brier_after": 0.018294590049400802,
95
+ "brier_improvement": -0.0001717616723535438
96
+ }
97
+ },
98
  "test_metrics": {
99
+ "threshold": 0.9284088015556335,
100
+ "auroc": 0.997910815020985,
101
+ "average_precision": 0.9976513211537581,
102
+ "f1": 0.9773091101352641,
103
+ "accuracy": 0.9797054998752118,
104
+ "precision": 0.9909459137479152,
105
+ "recall": 0.9640425346970707,
106
+ "specificity": 0.9926951172625913,
107
+ "precision_human": 0.9708363687636594,
108
+ "recall_human": 0.9926951172625913,
109
+ "precision_ai": 0.9909459137479152,
110
+ "recall_ai": 0.9640425346970707,
111
  "confusion_matrix": {
112
+ "true_negative": 123936,
113
+ "false_positive": 912,
114
+ "false_negative": 3723,
115
+ "true_positive": 99816
116
  }
117
  },
118
+ "timestamp": "20251115_090814",
119
  "seed": 42
120
  }
threshold.json CHANGED
@@ -1,9 +1,9 @@
1
  {
2
- "threshold": 0.9032942056655884,
3
  "method": "max_f1",
4
- "validation_f1": 0.9734939759036144,
5
  "alternative_thresholds": {
6
- "max_f1": 0.9032942056655884,
7
- "precision_at_95recall": 5.1442217227304354e-05
8
  }
9
  }
 
1
  {
2
+ "threshold": 0.9284088015556335,
3
  "method": "max_f1",
4
+ "calibration_f1": 0.9773827668313225,
5
  "alternative_thresholds": {
6
+ "max_f1": 0.9284088015556335,
7
+ "precision_at_95recall": 1.9947297005273867e-06
8
  }
9
  }
training_log_history.csv CHANGED
@@ -1,144 +1,52 @@
1
  loss,grad_norm,learning_rate,epoch,step,eval_loss,eval_auroc,eval_ap,eval_f1,eval_max_f1,eval_best_threshold,eval_accuracy,eval_precision_human,eval_recall_human,eval_precision_ai,eval_recall_ai,eval_runtime,eval_samples_per_second,eval_steps_per_second,train_runtime,train_samples_per_second,train_steps_per_second,total_flos,train_loss
2
- 0.7281,1.3650943040847778,1.3397755107862095e-05,0.028021997267855266,200,,,,,,,,,,,,,,,,,,,
3
- ,,,0.028021997267855266,200,0.5034370422363281,0.8567927920680234,0.7639926105515766,0.8347106400605876,0.837193236817485,0.532181978225708,0.8270340563782368,0.959174450135583,0.7139779089605678,0.7363754484917389,0.9633568027506544,246.8726,925.117,14.457,,,,,
4
- 0.4371,1.1207246780395508,2.6862835618276262e-05,0.05604399453571053,400,,,,,,,,,,,,,,,,,,,
5
- ,,,0.05604399453571053,400,0.3717988133430481,0.9213253952884014,0.8819850262727043,0.8641301759262867,0.8645202782854652,0.6361271142959595,0.862605413641817,0.9628228489938203,0.7787291644973448,0.7831802841221254,0.9637431306077903,246.6166,926.077,14.472,,,,,
6
- 0.2689,1.5876444578170776,4.032791612869042e-05,0.0840659918035658,600,,,,,,,,,,,,,,,,,,,
7
- ,,,0.0840659918035658,600,0.2004675269126892,0.9836750912020398,0.9784411063029117,0.921532737340532,0.9410314678576056,0.8459424376487732,0.9254814218034381,0.9686869740593922,0.8925324597307104,0.8816362898530268,0.965211176464907,246.4382,926.748,14.482,,,,,
8
- 0.1474,0.5951786637306213,4.455828097794425e-05,0.11208798907142106,800,,,,,,,,,,,,,,,,,,,
9
- ,,,0.11208798907142106,800,0.22212685644626617,0.9876414514694006,0.9855191906869644,0.9181336820638506,0.9423700865680394,0.9407896995544434,0.9216064031945916,0.9722504349669249,0.8817592733505811,0.8718120164294584,0.9696539468219705,245.9329,928.652,14.512,,,,,
10
- 0.1288,3.1992154121398926,4.450206516694645e-05,0.14010998633927632,1000,,,,,,,,,,,,,,,,,,,
11
- ,,,0.14010998633927632,1000,0.12653978168964386,0.9925797356069876,0.9916142379137276,0.9520024486594546,0.9573109145286222,0.7943849563598633,0.9560568511204715,0.9673535182486506,0.9517329210954208,0.942911278480413,0.9612706323221202,245.8558,928.943,14.517,,,,,
12
- 0.1172,0.6789711117744446,4.439853384372035e-05,0.1681319836071316,1200,,,,,,,,,,,,,,,,,,,
13
- ,,,0.1681319836071316,1200,0.14555694162845612,0.9921387821614682,0.9912700139992648,0.9446152828753761,0.9561932798497733,0.8887588381767273,0.9486614766229103,0.9704634681927069,0.9345278621032144,0.9244281725559809,0.9657037444827553,245.8898,928.815,14.515,,,,,
14
- 0.1092,0.6279441118240356,4.4247907495230585e-05,0.19615398087498687,1400,,,,,,,,,,,,,,,,,,,
15
- ,,,0.19615398087498687,1400,0.15372662246227264,0.9927978868223,0.9919292076748125,0.9417183531079946,0.9590575982598013,0.9353464841842651,0.9456490327778411,0.9726421280960468,0.9266382051631197,0.9163133657395564,0.9685722288219898,245.8833,928.839,14.515,,,,,
16
- 0.1062,1.4446195363998413,4.405050690503654e-05,0.22417597814284212,1600,,,,,,,,,,,,,,,,,,,
17
- ,,,0.22417597814284212,1600,0.11368440091609955,0.9942393901033173,0.993596916377211,0.9563939433106902,0.9633088138126524,0.8757869601249695,0.9601157689175344,0.970357305416477,0.956250450551475,0.9481557415950035,0.964776557625629,245.857,928.938,14.517,,,,,
18
- 0.0976,0.7851302027702332,4.3806752470131086e-05,0.2521979754106974,1800,,,,,,,,,,,,,,,,,,,
19
- ,,,0.2521979754106974,1800,0.14997108280658722,0.9933556710094985,0.9925489998985919,0.9429900968712875,0.9596468920355709,0.9252299070358276,0.9468399989491475,0.973705889275567,0.9278076365471337,0.9176323292879076,0.969789161571968,245.7705,929.265,14.522,,,,,
20
- 0.097,1.144687294960022,4.351716330563604e-05,0.28021997267855264,2000,,,,,,,,,,,,,,,,,,,
21
- ,,,0.28021997267855264,2000,0.13728895783424377,0.9936304293189042,0.9929477433592986,0.9478551974580118,0.9613444886776696,0.9086179733276367,0.9517133274368832,0.9725250545928577,0.938172322923258,0.9284940110607787,0.9680410280184278,245.8436,928.989,14.517,,,,,
22
- 0.0913,0.8097946643829346,4.3182356139261264e-05,0.30824196994640796,2200,,,,,,,,,,,,,,,,,,,
23
- ,,,0.30824196994640796,2200,0.1629137396812439,0.992883712964101,0.9919489878089339,0.9421523335980788,0.9580045862640264,0.9407896995544434,0.9461044021962818,0.9722851723153884,0.9278556953711342,0.917551536011131,0.9681086353934266,246.024,928.308,14.507,,,,,
24
- 0.0942,1.610180139541626,4.280304399788167e-05,0.3362639672142632,2400,,,,,,,,,,,,,,,,,,,
25
- ,,,0.3362639672142632,2400,0.15682530403137207,0.9938745855338663,0.993259688256961,0.9434721308393647,0.9617119643400787,0.9585376977920532,0.9473347753364917,0.9734525158420412,0.9289930875391479,0.9188491500443973,0.9694511246969741,245.8341,929.025,14.518,,,,,
26
- 0.0898,0.44423627853393555,4.238003468902928e-05,0.3642859644821185,2600,,,,,,,,,,,,,,,,,,,
27
- ,,,0.3642859644821185,2600,0.10643962770700455,0.9950490299401135,0.9944538510424438,0.9604357495208773,0.9664626252455953,0.873214840888977,0.9639338663490757,0.9712200562506061,0.9625461564955505,0.9553194335620234,0.9656071625184713,245.8591,928.93,14.516,,,,,
28
- 0.0911,1.0590174198150635,4.191422908053442e-05,0.39230796174997373,2800,,,,,,,,,,,,,,,,,,,
29
- ,,,0.39230796174997373,2800,0.11612347513437271,0.9946141573591816,0.9939944771051153,0.9569791183516552,0.9643653756980626,0.8840392827987671,0.9606061667527782,0.9717250028503022,0.9557458328994689,0.9476759603000228,0.9664667420005988,245.8438,928.988,14.517,,,,,
30
- 0.0856,1.3164970874786377,4.140661918197958e-05,0.420329959017829,3000,,,,,,,,,,,,,,,,,,,
31
- ,,,0.420329959017829,3000,0.11796951293945312,0.9949918667417889,0.994355458108942,0.9567068978687648,0.9665062880160855,0.9343951344490051,0.9602952895536504,0.9726869656811111,0.95415989170745,0.9459671252018089,0.9676933329470054,245.7791,929.233,14.521,,,,,
32
- 0.0821,1.2711987495422363,4.0858286032052055e-05,0.44835195628568425,3200,,,,,,,,,,,,,,,,,,,
33
- ,,,0.44835195628568425,3200,0.09971292316913605,0.9955405517378006,0.9949817449087798,0.9628310234105341,0.9679576292752426,0.879974365234375,0.9661931992328777,0.9715293321953654,0.9664789702595977,0.9598322263600415,0.9658486174291813,245.8241,929.063,14.519,,,,,
34
- 0.085,0.5998378396034241,4.027039739629442e-05,0.4763739535535395,3400,,,,,,,,,,,,,,,,,,,
35
- ,,,0.4763739535535395,3400,0.12078217417001724,0.9949955823550951,0.9944631387897102,0.9548334238379681,0.9663685344296211,0.9263036847114563,0.9584825689840883,0.9728422588551614,0.950587519123407,0.9420179519714272,0.9680023952327143,245.9407,928.622,14.512,,,,,
36
- 0.084,0.671790361404419,3.964420528015589e-05,0.5043959508213948,3600,,,,,,,,,,,,,,,,,,,
37
- ,,,0.5043959508213948,3600,0.1031280905008316,0.9955552436116516,0.9950748276653851,0.9607936256661918,0.9684774337055235,0.8688268065452576,0.9642140936835008,0.9725021058770168,0.9617291564875408,0.954461409427956,0.9672104231255856,245.879,928.855,14.515,,,,,
38
- 0.081,1.193284034729004,3.898104326264103e-05,0.53241794808925,3800,,,,,,,,,,,,,,,,,,,
39
- ,,,0.53241794808925,3800,0.0943664014339447,0.9957421605444731,0.995280599833219,0.96472756750497,0.9691255242991724,0.8365545868873596,0.9679927841461385,0.9713508874648102,0.9700593526476408,0.963955450556868,0.9655009223577589,245.7307,929.416,14.524,,,,,
40
- 0.0807,0.6923601031303406,3.828232365623414e-05,0.5604399453571053,4000,,,,,,,,,,,,,,,,,,,
41
- ,,,0.5604399453571053,4000,0.13602322340011597,0.9945068890032484,0.993905565216706,0.9503015258677455,0.9644104098812593,0.9539660811424255,0.9540646099147934,0.9732024629237288,0.9419048915873028,0.9325641073321308,0.9687267599648441,245.8443,928.986,14.517,,,,,
42
- 0.0779,0.29442542791366577,3.7549534499147724e-05,0.5884619426249605,4200,,,,,,,,,,,,,,,,,,,
43
- ,,,0.5884619426249605,4200,0.10123751312494278,0.9954435493591816,0.9949460283029852,0.9623870601479885,0.9677929996672676,0.8652240633964539,0.9657684796791397,0.971621087925462,0.965581872211587,0.9588074581795523,0.9659934903756072,245.9464,928.601,14.511,,,,,
44
- 0.0795,1.8506108522415161,3.678423638630061e-05,0.6164839398928159,4400,,,,,,,,,,,,,,,,,,,
45
- ,,,0.6164839398928159,4400,0.11482585221529007,0.9954225057051109,0.994962660756873,0.9564496059726255,0.9688606356963216,0.9433475732803345,0.9600019265629242,0.9735773688690983,0.9526860877714323,0.9443879568434729,0.9688233419291281,245.8734,928.876,14.516,,,,,
46
- 0.0764,1.2500754594802856,3.598805914577467e-05,0.6445059371606712,4600,,,,,,,,,,,,,,,,,,,
47
- ,,,0.6445059371606712,4600,0.12350087612867355,0.9953538855682633,0.9948421489775661,0.954069819959019,0.9679151708414333,0.9511421918869019,0.9576988081581183,0.9736889722331886,0.9482406465513789,0.939495515065261,0.9691034296255517,245.8249,929.06,14.518,,,,,
48
- 0.0776,0.9441678524017334,3.516269836782809e-05,0.6725279344285264,4800,,,,,,,,,,,,,,,,,,,
49
- ,,,0.6725279344285264,4800,0.09324090927839279,0.9960413055155612,0.995564989005534,0.9658611467796905,0.9704891089007048,0.8840392827987671,0.9690348795460317,0.9719646392190493,0.9713729605036565,0.9655069246730685,0.9662156288934605,245.8365,929.016,14.518,,,,,
50
- 0.0736,0.6129169464111328,3.430991179385733e-05,0.7005499316963817,5000,,,,,,,,,,,,,,,,,,,
51
- ,,,0.7005499316963817,5000,0.11999406665563583,0.9952773925550765,0.9948196772549471,0.9540792562949726,0.9679756358671789,0.9416541457176208,0.9576944296060179,0.9739382318814341,0.9479763230193757,0.9392235208249506,0.9694124919112604,245.8584,928.933,14.516,,,,,
52
- 0.0789,0.7752780914306641,3.343151557299794e-05,0.728571928964237,5200,,,,,,,,,,,,,,,,,,,
53
- ,,,0.728571928964237,5200,0.10578533262014389,0.9959060523948745,0.9954666417692407,0.9599800834007594,0.9701210620572613,0.9343951344490051,0.963399682992828,0.9733206559721748,0.9593422348955121,0.9518095165761592,0.9682921411255662,245.8669,928.901,14.516,,,,,
54
- 0.0739,0.8520299196243286,3.2529380394336615e-05,0.7565939262320922,5400,,,,,,,,,,,,,,,,,,,
55
- ,,,0.7565939262320922,5400,0.10391142964363098,0.9957951879085571,0.9952825044933503,0.962313827231671,0.9695487450431454,0.9416541457176208,0.9656546373245295,0.9726057276729814,0.964332342787572,0.957428705270504,0.9672490559112992,245.8211,929.074,14.519,,,,,
56
- 0.0775,0.6676751971244812,3.1605427502971364e-05,0.7846159234999475,5600,,,,,,,,,,,,,,,,,,,
57
- ,,,0.7846159234999475,5600,0.08965875953435898,0.9963213227367876,0.9958643558038499,0.966500156857067,0.9713820883230001,0.8991213440895081,0.969608469871183,0.9726449130121062,0.9717334016836608,0.9659546963031566,0.9670462337863027,245.7698,929.268,14.522,,,,,
58
- 0.0727,1.1384795904159546,3.0661624608404376e-05,0.8126379207678027,5800,,,,,,,,,,,,,,,,,,,
59
- ,,,0.8126379207678027,5800,0.09947814047336578,0.9960502170307564,0.9955943884802091,0.9634165700569075,0.9701781488936472,0.9124361872673035,0.966701111276523,0.9725823095030796,0.966326783983596,0.9597094171091219,0.9671524739470151,245.8302,929.04,14.518,,,,,
60
- 0.0723,0.6806196570396423,2.969998169398137e-05,0.840659918035658,6000,,,,,,,,,,,,,,,,,,,
61
- ,,,0.840659918035658,6000,0.11319559812545776,0.9956365087859012,0.9951325050889013,0.958895197822541,0.9684056849185914,0.9525741338729858,0.9623751018013363,0.9730618357016131,0.9576842054674922,0.9499298657972552,0.9680313698219994,245.806,929.131,14.52,,,,,
62
- 0.0703,0.6376020908355713,2.872254673630171e-05,0.8686819153035132,6200,,,,,,,,,,,,,,,,,,,
63
- ,,,0.8686819153035132,6200,0.09632854163646698,0.9961434762558163,0.9956792827654228,0.9641803258959732,0.9704082233158977,0.9343951344490051,0.9674191938209873,0.9726870118367018,0.9675683036036108,0.961140542823141,0.9672393977148708,245.8119,929.109,14.519,,,,,
64
- 0.0717,0.6587705016136169,2.773140134371577e-05,0.8967039125713685,6400,,,,,,,,,,,,,,,,,,,
65
- ,,,0.8967039125713685,6400,0.11494402587413788,0.9958696310682444,0.9953992776112017,0.9580610203799094,0.9703862409011201,0.9585376977920532,0.9615344197980612,0.9739004034167933,0.9552332054434628,0.9472397387002983,0.9691324042148369,245.8451,928.983,14.517,,,,,
66
- 0.071,1.1992093324661255,2.6728656323197893e-05,0.9247259098392238,6600,,,,,,,,,,,,,,,,,,,
67
- ,,,0.9247259098392238,6600,0.1079399436712265,0.9959413616290814,0.9954836777985783,0.9598131601491273,0.9701123540493861,0.9504110217094421,0.963233298013013,0.9734663674212855,0.9588776662635066,0.9512949435537426,0.9684853050541342,245.9034,928.763,14.514,,,,,
68
- 0.0717,0.6620305180549622,2.5716447185036114e-05,0.952747907107079,6800,,,,,,,,,,,,,,,,,,,
69
- ,,,0.952747907107079,6800,0.09787657111883163,0.9961755673418714,0.9957160605725904,0.963615903975994,0.9705563424993778,0.8991213440895081,0.9668718748084384,0.9729949910063963,0.9662146467275946,0.9596011838059937,0.9676643583577202,245.9366,928.638,14.512,,,,,
70
- 0.0711,1.2997543811798096,2.4696929594912076e-05,0.9807699043749343,7000,,,,,,,,,,,,,,,,,,,
71
- ,,,0.9807699043749343,7000,0.1146920770406723,0.995944675317155,0.9954875225794212,0.9577217544780213,0.9696940029268117,0.954647421836853,0.9611928927342307,0.9742170251042603,0.9542640191594511,0.9461808173729452,0.9695477066612581,245.8587,928.932,14.516,,,,,
72
- 0.0724,0.7974486351013184,2.3672274783056795e-05,1.0086868191530352,7200,,,,,,,,,,,,,,,,,,,
73
- ,,,1.0086868191530352,7200,0.1015845537185669,0.9961790497526729,0.9957323299850726,0.9629252207518783,0.9700351292055995,0.9184802770614624,0.9662282276496807,0.9727491403386985,0.9652614800515832,0.9584976076555024,0.9673939288577251,245.8262,929.055,14.518,,,,,
74
- 0.0649,0.8229542374610901,2.2644664920259076e-05,1.0367088164208904,7400,,,,,,,,,,,,,,,,,,,
75
- ,,,1.0367088164208904,7400,0.08645018190145493,0.9965600272968401,0.9961185117789384,0.9678489558648573,0.971378867175356,0.8255897164344788,0.9708913856365977,0.9722255515337472,0.9745929017116951,0.9692739865355742,0.9664281092148852,245.8743,928.873,14.516,,,,,
76
- 0.068,0.7686835527420044,2.1616288470574255e-05,1.0647308136887457,7600,,,,,,,,,,,,,,,,,,,
77
- ,,,1.0647308136887457,7600,0.0841926634311676,0.9967842136955064,0.9963728317459107,0.9687837112215133,0.9733571550938489,0.8933094143867493,0.971745203296174,0.972796830822804,0.9755941272117071,0.9704690831556503,0.9671041829648731,245.8638,928.913,14.516,,,,,
78
- 0.0684,0.49581971764564514,2.0589335530630446e-05,1.092752810956601,7800,,,,,,,,,,,,,,,,,,,
79
- ,,,1.092752810956601,7800,0.097724050283432,0.9964498048620039,0.996013417423547,0.9642599451633075,0.9715029456084876,0.9334307909011841,0.9674673578940916,0.9733220461966381,0.966991597715604,0.9605082845396786,0.9680410280184278,245.8522,928.956,14.517,,,,,
80
- 0.0665,1.0024373531341553,1.9565993165457813e-05,1.1207748082244562,8000,,,,,,,,,,,,,,,,,,,
81
- ,,,1.1207748082244562,8000,0.08990959823131561,0.9965416445649657,0.996111897432367,0.9669079477905634,0.9719970667313506,0.8918110132217407,0.9699937824560174,0.9725960460123683,0.9725103526716701,0.9668565910188315,0.9669593100184471,246.7329,925.641,14.465,,,,,
82
- 0.0676,0.9732509851455688,1.8548440750774307e-05,1.1487968054923114,8200,,,,,,,,,,,,,,,,,,,
83
- ,,,1.1487968054923114,8200,0.0953540951013565,0.9965252478256964,0.9961004574885287,0.9655225599969152,0.9714322090330724,0.9399133324623108,0.9686802168259,0.9728337725782605,0.9697870193116375,0.9637063408063119,0.9673456378755831,246.7142,925.711,14.466,,,,,
84
- 0.0638,1.3985579013824463,1.753884533164692e-05,1.1768188027601667,8400,,,,,,,,,,,,,,,,,,,
85
- ,,,1.1768188027601667,8400,0.09055861830711365,0.9965989400880163,0.9961690844094493,0.966675668897934,0.9727147099462821,0.9353464841842651,0.9697442049862951,0.9733802149812554,0.9712127644236546,0.9653813922575301,0.9679734206434291,246.574,926.237,14.474,,,,,
86
- 0.0641,0.5897226929664612,1.6539357007413156e-05,1.204840800028022,8600,,,,,,,,,,,,,,,,,,,
87
- ,,,1.204840800028022,8600,0.10399724543094635,0.9962708327892441,0.9957835972645783,0.9633992144268537,0.9704696278600408,0.9532750844955444,0.96666608285972,0.9730076498499081,0.9658141565275897,0.9591430376596274,0.9676933329470054,246.6251,926.045,14.471,,,,,
88
- 0.0693,0.9277693629264832,1.5552104352691164e-05,1.2328627972958772,8800,,,,,,,,,,,,,,,,,,,
89
- ,,,1.2328627972958772,8800,0.08372964709997177,0.9967832506763049,0.9963809846481838,0.9686703149419614,0.9731677699865736,0.896251380443573,0.9716488751499655,0.9725353686967074,0.9756902448597082,0.9705722652083697,0.9667758042863076,246.6238,926.05,14.471,,,,,
90
- 0.0621,0.4265595078468323,1.457918988423039e-05,1.2608847945637325,9000,,,,,,,,,,,,,,,,,,,
91
- ,,,1.2608847945637325,9000,0.08751900494098663,0.9967499914503182,0.9963140806446376,0.9678585059253784,0.972755227251117,0.8887588381767273,0.9708694928760957,0.9730257576000128,0.9737038134676844,0.9682654757762054,0.9674518780362955,246.5692,926.255,14.475,,,,,
92
- 0.066,0.5132720470428467,1.3622685583256601e-05,1.2889067918315877,9200,,,,,,,,,,,,,,,,,,,
93
- ,,,1.2889067918315877,9200,0.09265980124473572,0.9966772555928973,0.9962576579748246,0.9657953308217793,0.9729735020893797,0.9381240010261536,0.9689122800872207,0.9734461323190002,0.9695787644076349,0.9634930552218004,0.9681086353934266,246.6667,925.889,14.469,,,,,
94
- 0.0638,0.6131060719490051,1.26846284828475e-05,1.316928789099443,9400,,,,,,,,,,,,,,,,,,,
95
- ,,,1.316928789099443,9400,0.0880228728055954,0.996786233571905,0.9963835217229717,0.9674853643116038,0.9736705691338173,0.9161096215248108,0.9705016944996628,0.973477863476741,0.9725343820836704,0.9669207022959676,0.9680506862148562,246.5183,926.446,14.478,,,,,
96
- 0.0631,0.5040121674537659,1.1767016329735986e-05,1.3449507863672983,9600,,,,,,,,,,,,,,,,,,,
97
- ,,,1.3449507863672983,9600,0.088343046605587,0.9967945716541701,0.9963785207323399,0.9681004779370901,0.9732178678012516,0.9207897186279297,0.9710971775853161,0.9730067043218075,0.9741523624916898,0.9687886877127824,0.9674132452505819,246.6015,926.134,14.473,,,,,
98
- 0.0626,0.5754002332687378,1.0871803329780279e-05,1.3729727836351535,9800,,,,,,,,,,,,,,,,,,,
99
- ,,,1.3729727836351535,9800,0.08908010274171829,0.9968349029617605,0.9964338452298704,0.9673004536241675,0.9733966350967752,0.9161096215248108,0.9703309309677476,0.9734012268954734,0.9722940879636676,0.9666380532595171,0.9679637624470007,246.5625,926.28,14.475,,,,,
100
- 0.0635,0.37719520926475525,1.0000895986161401e-05,1.4009947809030088,10000,,,,,,,,,,,,,,,,,,,
101
- ,,,1.4009947809030088,10000,0.10576409846544266,0.9964103740665244,0.9959816504324808,0.9620018891174369,0.9717239996277691,0.9585376977920532,0.9652999746043979,0.9738979588528072,0.9623138721795478,0.9552007160268129,0.9689006075005554,246.5201,926.44,14.478,,,,,
102
- 0.0647,0.4855269193649292,9.156149039171426e-06,1.429016778170864,10200,,,,,,,,,,,,,,,,,,,
103
- ,,,1.429016778170864,10200,0.08731859922409058,0.9969044529302581,0.9965060417039346,0.967862178187226,0.9734939759031145,0.9019206762313843,0.9708563572197946,0.973457141254797,0.9732232252276787,0.9677220017572826,0.9680023952327143,246.5275,926.412,14.477,,,,,
104
- 0.0658,0.9444574117660522,8.339361516239216e-06,1.4570387754387193,10400,,,,,,,,,,,,,,,,,,,
105
- ,,,1.4570387754387193,10400,0.0880136489868164,0.9968816941218545,0.9965015373119736,0.9673096035276295,0.9733884580565982,0.9161096215248108,0.9703309309677476,0.9736215071361525,0.9720618036476647,0.9663771580601317,0.9682438501434242,246.6147,926.084,14.472,,,,,
106
- 0.0611,0.45777803659439087,7.5522729006059505e-06,1.4850607727065746,10600,,,,,,,,,,,,,,,,,,,
107
- ,,,1.4850607727065746,10600,0.09273924678564072,0.9967055123563264,0.9962876686774951,0.9661673703082528,0.9724540697185579,0.9399133324623108,0.969262564255252,0.9734555945770013,0.9702275585316428,0.9642434561773109,0.9680989771969982,246.5306,926.4,14.477,,,,,
108
- 0.0627,0.8916956186294556,6.796559426809692e-06,1.5130827699744298,10800,,,,,,,,,,,,,,,,,,,
109
- ,,,1.5130827699744298,10800,0.08739963173866272,0.9968748801914394,0.9964844262023231,0.9675924808764267,0.9735564214519877,0.9241418242454529,0.9705980226458715,0.9735888389993586,0.9725984605156712,0.9669997877799857,0.9681859009648538,246.7364,925.627,14.465,,,,,
110
- 0.0623,0.7013267278671265,6.0738305108685545e-06,1.541104767242285,11000,,,,,,,,,,,,,,,,,,,
111
- ,,,1.541104767242285,11000,0.0974632278084755,0.9966048359068608,0.9961832096074739,0.964546783625731,0.9723199624672457,0.9362850189208984,0.9677213139159143,0.9737315407011913,0.9670396565396044,0.9605831585198809,0.9685432542327046,246.5852,926.195,14.474,,,,,
112
- 0.063,0.8154935240745544,5.385625322764794e-06,1.5691267645101403,11200,,,,,,,,,,,,,,,,,,,
113
- ,,,1.5691267645101403,11200,0.09752331674098969,0.9966611901489429,0.9962433071485012,0.964554077507007,0.9729103642409853,0.9433475732803345,0.9677169353638139,0.9739989347794509,0.966751303595601,0.9602653444116859,0.9688812911076985,246.5136,926.464,14.478,,,,,
114
- 0.0631,0.685742974281311,4.73340950852946e-06,1.5971487617779956,11400,,,,,,,,,,,,,,,,,,,
115
- ,,,1.5971487617779956,11400,0.09248074889183044,0.9967045142155668,0.996290326082087,0.9660411899313501,0.9726085083041068,0.9399133324623108,0.9691355862443407,0.9736550060313631,0.9697790095076374,0.9637337075627668,0.968359748500565,246.5731,926.241,14.474,,,,,
116
- 0.0627,1.1395540237426758,4.118572068908318e-06,1.6251707590458508,11600,,,,,,,,,,,,,,,,,,,
117
- ,,,1.6251707590458508,11600,0.08698944747447968,0.9968948298167076,0.996494789063069,0.9677260527890946,0.973689873045802,0.9161096215248108,0.9707250006567828,0.9735494265034746,0.9728788036556746,0.9673244873341376,0.9681279517862834,246.6038,926.125,14.473,,,,,
118
- 0.0621,1.5019831657409668,3.5424224012565447e-06,1.653192756313706,11800,,,,,,,,,,,,,,,,,,,
119
- ,,,1.653192756313706,11800,0.10022038221359253,0.9966304956476687,0.9962084684159175,0.9636517328825022,0.9727368874608185,0.9626730680465698,0.9668631177042376,0.973980940371978,0.965165362403582,0.9584499708605222,0.9689102656969838,246.5696,926.254,14.475,,,,,
120
- 0.0621,0.6207771897315979,3.006187510961881e-06,1.6812147535815614,12000,,,,,,,,,,,,,,,,,,,
121
- ,,,1.6812147535815614,12000,0.08885731548070908,0.9968981157711279,0.9965016737672114,0.9672419696290188,0.9739953876205015,0.9433475732803345,0.970256495582041,0.9738992902790713,0.9716292742316596,0.9658865453144563,0.968601203411275,246.5364,926.378,14.477,,,,,
122
- 0.0625,1.3990130424499512,2.511009398335074e-06,1.7092367508494168,12200,,,,,,,,,,,,,,,,,,,
123
- ,,,1.7092367508494168,12200,0.09371371567249298,0.9967498807864655,0.9963397310167501,0.965798508289155,0.9732076432118419,0.9481545686721802,0.9688991444309196,0.9738417951772319,0.9691382251876297,0.9630024389775499,0.9686108616077034,246.5834,926.202,14.474,,,,,
124
- 0.0623,0.5540758967399597,2.057942626532536e-06,1.737258748117272,12400,,,,,,,,,,,,,,,,,,,
125
- ,,,1.737258748117272,12400,0.09393420815467834,0.9967897724943766,0.9963785511182874,0.9659367279417076,0.9731286363853603,0.939024806022644,0.9690305009939313,0.9738176083423182,0.9694105585236329,0.963315530623223,0.9685722288219898,246.6306,926.025,14.471,,,,,
126
- 0.0643,0.9720720052719116,1.6479520756908518e-06,1.7652807453851274,12600,,,,,,,,,,,,,,,,,,,
127
- ,,,1.7652807453851274,12600,0.09308448433876038,0.9967791736358621,0.9963694636590227,0.9660367906669813,0.9731835264636404,0.9441768527030945,0.9691268291401399,0.9737688830257887,0.9696428428396358,0.9635815044009686,0.968504621446991,246.6887,925.807,14.468,,,,,
128
- 0.0624,1.0696525573730469,1.2819108880559477e-06,1.7933027426529826,12800,,,,,,,,,,,,,,,,,,,
129
- ,,,1.7933027426529826,12800,0.09151949733495712,0.9968253090517087,0.9964204520696328,0.9665868246157702,0.9733586501189067,0.9399133324623108,0.9696434982879861,0.9738089880416613,0.9705719801036469,0.964657444638975,0.9685239378398478,246.5498,926.328,14.476,,,,,
130
- 0.0625,1.4033018350601196,9.605986084832452e-07,1.8213247399208379,13000,,,,,,,,,,,,,,,,,,,
131
- ,,,1.8213247399208379,13000,0.09507396817207336,0.9967674833401138,0.9963561447668665,0.9654810607883717,0.9731590960292014,0.9591543078422546,0.9685970243359926,0.9739035882566147,0.9685054506716221,0.9622760982817012,0.9687074435719873,246.7376,925.623,14.465,,,,,
132
- 0.063,0.45164167881011963,6.846995242687752e-07,1.8493467371886931,13200,,,,,,,,,,,,,,,,,,,
133
- ,,,1.8493467371886931,13200,0.09227791428565979,0.9968329623022687,0.9964266292094927,0.9663162762913218,0.9733341164718503,0.9465966820716858,0.969385163714063,0.9738956307751695,0.9699952742156399,0.9639942713790021,0.968649494393417,246.5226,926.43,14.477,,,,,
134
- 0.0618,0.3481653332710266,4.548012078469415e-07,1.8773687344565484,13400,,,,,,,,,,,,,,,,,,,
135
- ,,,1.8773687344565484,13400,0.09297340363264084,0.9968072979972056,0.9963973889545064,0.9661836679640834,0.9732662175973241,0.9425067901611328,0.969262564255252,0.9738363909822815,0.9698270683316379,0.9637972956089685,0.9685818870184182,246.5722,926.244,14.474,,,,,
136
- 0.0631,0.3175712525844574,2.7139326545848965e-07,1.9053907317244037,13600,,,,,,,,,,,,,,,,,,,
137
- ,,,1.9053907317244037,13600,0.09512791037559509,0.9967728122242903,0.9963611581808621,0.9655092748572913,0.9731316412589667,0.9489172697067261,0.968623295648595,0.9739124831868813,0.9685454996916225,0.9623226227369108,0.9687171017684157,246.586,926.192,14.474,,,,,
138
- 0.0606,0.5379019975662231,1.348662944535554e-07,1.933412728992259,13800,,,,,,,,,,,,,,,,,,,
139
- ,,,1.933412728992259,13800,0.09340985864400864,0.9968055931317871,0.9963962951212402,0.9660171262895287,0.9732650683786642,0.9504110217094421,0.969104936379638,0.9738516867673443,0.9695146859756342,0.9634372448244393,0.9686108616077034,246.589,926.181,14.473,,,,,
140
- 0.061,1.021728754043579,4.551105145043261e-08,1.9614347262601142,14000,,,,,,,,,,,,,,,,,,,
141
- ,,,1.9614347262601142,14000,0.09326184540987015,0.9968091848139563,0.9964000739066909,0.9660543477213837,0.9732996051448428,0.9465966820716858,0.9691399647964412,0.9738533696972622,0.9695787644076349,0.9635112934372208,0.9686108616077034,246.5091,926.481,14.478,,,,,
142
- 0.0623,1.5442456007003784,3.5178331215701367e-09,1.9894567235279694,14200,,,,,,,,,,,,,,,,,,,
143
- ,,,1.9894567235279694,14200,0.09308414906263351,0.9968106062700608,0.9964016493909656,0.9661896251908728,0.9732898767747101,0.9473810195922852,0.9692669428073525,0.9738670913565948,0.9698030389196376,0.9637709014030367,0.9686205198041318,246.5533,926.315,14.476,,,,,
144
- ,,,2.0,14276,,,,,,,,,,,,,,,27534.7527,132.711,0.518,3.3997885998922465e+18,0.09163027936652716
 
1
  loss,grad_norm,learning_rate,epoch,step,eval_loss,eval_auroc,eval_ap,eval_f1,eval_max_f1,eval_best_threshold,eval_accuracy,eval_precision_human,eval_recall_human,eval_precision_ai,eval_recall_ai,eval_runtime,eval_samples_per_second,eval_steps_per_second,train_runtime,train_samples_per_second,train_steps_per_second,total_flos,train_loss
2
+ 0.276,3.159646987915039,0.00013013829896707,0.07783918424534911,500,,,,,,,,,,,,,,,,,,,
3
+ ,,,0.07783918424534911,500,0.12149354815483093,0.9924457584277557,0.9916298325594288,0.9536510818288485,0.9557720332927327,0.6680145263671875,0.9579702783883426,0.9616493887295509,0.9614568231515375,0.9535359777528871,0.9537662136972542,251.8553,906.814,14.171,,,,,
4
+ 0.1082,0.4662734270095825,0.00015502115157402368,0.15567836849069822,1000,,,,,,,,,,,,,,,,,,,
5
+ ,,,0.15567836849069822,1000,0.11359784007072449,0.9941361028997843,0.9936079408729138,0.9563341131667457,0.964144751321268,0.8459424376487732,0.9600369549797273,0.9707662766667209,0.9556737446634681,0.9475350777398559,0.9652981002327625,252.3501,905.036,14.143,,,,,
6
+ 0.0901,0.56740403175354,0.00015336171321936976,0.23351755273604732,1500,,,,,,,,,,,,,,,,,,,
7
+ ,,,0.23351755273604732,1500,0.09792134165763855,0.995637180466362,0.9951736481792155,0.9632415978730987,0.9687276503605232,0.8856314420700073,0.9665828903698125,0.971504195528524,0.9672399016396068,0.9607059479089608,0.9657906682506109,252.1455,905.771,14.155,,,,,
8
+ 0.0828,0.6910482048988342,0.0001504606098364759,0.31135673698139643,2000,,,,,,,,,,,,,,,,,,,
9
+ ,,,0.31135673698139643,2000,0.07930342108011246,0.9965300160027056,0.996129497193991,0.9701504169589276,0.9709891509313444,0.6976089477539062,0.9731682327288013,0.968767028089932,0.9825946959077911,0.9786452037697653,0.9618018331256821,252.2542,905.38,14.148,,,,,
10
+ 0.0786,0.7298774719238281,0.00014636546193317465,0.38919592122674557,2500,,,,,,,,,,,,,,,,,,,
11
+ ,,,0.38919592122674557,2500,0.10587891191244125,0.9961595063461319,0.99574080839367,0.9633412681237827,0.9702281512250107,0.9585376977920532,0.9666222973387161,0.9727536699467656,0.965998382019592,0.9593414171599334,0.9673746124648683,252.014,906.243,14.162,,,,,
12
+ 0.075,0.5279271006584167,0.00014114348980363213,0.46703510547209465,3000,,,,,,,,,,,,,,,,,,,
13
+ ,,,0.46703510547209465,3000,0.08700015395879745,0.9963549115948438,0.9959508112193701,0.9664618832348054,0.9711828125530761,0.8791467547416687,0.9695953342148819,0.9720765232989261,0.9723101075716677,0.9666019379957298,0.9663218690541728,252.0033,906.282,14.163,,,,,
14
+ 0.0718,0.8794483542442322,0.00013488041013280436,0.5448742897174438,3500,,,,,,,,,,,,,,,,,,,
15
+ ,,,0.5448742897174438,3500,0.10059615969657898,0.9964150386194646,0.9960424913733611,0.9614705825931823,0.9711960816065123,0.954647421836853,0.9648227124254551,0.9732758550835028,0.9620655682555448,0.9548853558398507,0.9681472681791402,252.084,905.992,14.158,,,,,
16
+ 0.0695,0.40731295943260193,0.00012767902898967842,0.6227134739627929,4000,,,,,,,,,,,,,,,,,,,
17
+ ,,,0.6227134739627929,4000,0.10083704441785812,0.9961781307127794,0.995731264522598,0.9619906765054659,0.9692610702277147,0.9149009585380554,0.9653350030212009,0.9728877169710597,0.9634352447395612,0.9564208797922713,0.9676257255720067,252.2327,905.457,14.15,,,,,
18
+ 0.0669,0.264863520860672,0.00011965755430477945,0.7005526582081419,4500,,,,,,,,,,,,,,,,,,,
19
+ ,,,0.7005526582081419,4500,0.094916433095932,0.9967265965858536,0.9963598631028794,0.9650556403576777,0.972462604745369,0.9161096215248108,0.9682248474074593,0.9731915784051766,0.9685535094956227,0.962298576833695,0.9678285476970031,251.9541,906.459,14.165,,,,,
20
+ 0.0672,0.7680786848068237,0.00011094765553198254,0.7783918424534911,5000,,,,,,,,,,,,,,,,,,,
21
+ ,,,0.7783918424534911,5000,0.09323982149362564,0.9967850366539563,0.9963962011157831,0.9673139455667273,0.9710460087467818,0.8643104434013367,0.9703790950408518,0.972494317999936,0.9733433722876801,0.9678236488446292,0.9668047788755928,251.9538,906.46,14.165,,,,,
22
+ 0.0647,0.4233705997467041,0.0001016923023445425,0.8562310266988402,5500,,,,,,,,,,,,,,,,,,,
23
+ ,,,0.8562310266988402,5500,0.09126096963882446,0.9967433808226673,0.9963771581479701,0.9657864214107987,0.9726255234214075,0.9111796617507935,0.9688728731183173,0.9742069565497575,0.9687056957716245,0.9625206246882314,0.9690744550362665,252.0396,906.151,14.16,,,,,
24
+ 0.0627,0.25091952085494995,9.204341784232336e-05,0.9340702109441893,6000,,,,,,,,,,,,,,,,,,,
25
+ ,,,0.9340702109441893,6000,0.06995870172977448,0.9975342235086746,0.9972417957598282,0.973204830514238,0.9758824625579795,0.8918110132217407,0.9758479066142408,0.9732840473716358,0.9827949410077935,0.9790068315757582,0.9674711944291523,252.1038,905.921,14.157,,,,,
26
+ 0.0606,0.3678501546382904,8.215938479193825e-05,1.011831556005293,6500,,,,,,,,,,,,,,,,,,,
27
+ ,,,1.011831556005293,6500,0.07501858472824097,0.9975025002706578,0.997203865754845,0.971139887346844,0.9754012996088349,0.8902942538261414,0.9738863152732654,0.9744957013881682,0.9778208527237339,0.9731459660760525,0.9691420624112653,252.3023,905.208,14.146,,,,,
28
+ 0.0538,0.6192397475242615,7.220244583391773e-05,1.0896707402506423,7000,,,,,,,,,,,,,,,,,,,
29
+ ,,,1.0896707402506423,7000,0.09348879754543304,0.997100493345388,0.9967079581489795,0.9689502265506753,0.9725104874267961,0.9184802770614624,0.9718853169633865,0.97321942331053,0.9754099017197049,0.9702686474655716,0.967635383768435,252.0093,906.26,14.162,,,,,
30
+ 0.0536,0.6146565675735474,6.233604033151736e-05,1.1675099244959912,7500,,,,,,,,,,,,,,,,,,,
31
+ ,,,1.1675099244959912,7500,0.08755695074796677,0.9973925286764659,0.9970545509180517,0.9708156623418074,0.974796319089823,0.9324532747268677,0.9735754380741376,0.9747314921365554,0.9769878331077239,0.9721743341404359,0.9694607828934025,251.9972,906.304,14.163,,,,,
32
+ 0.0563,0.24420885741710663,5.272212157577683e-05,1.2453491087413404,8000,,,,,,,,,,,,,,,,,,,
33
+ ,,,1.2453491087413404,8000,0.07455883920192719,0.9975485047929438,0.9972585615075422,0.9726402255038238,0.9758848582753115,0.9136765599250793,0.9752655591848888,0.9750665582603982,0.9798072841157577,0.97550810243656,0.969789161571968,252.079,906.01,14.158,,,,,
34
+ 0.0521,0.4137882590293884,4.351849838388919e-05,1.3231882929866896,8500,,,,,,,,,,,,,,,,,,,
35
+ ,,,1.3231882929866896,8500,0.07783409208059311,0.9975517384131479,0.9972440455972116,0.9723962677736611,0.9749894769810089,0.9111796617507935,0.975077281444572,0.9738866219645087,0.9807043821637684,0.9765353333658013,0.9682921411255662,252.489,904.538,14.135,,,,,
36
+ 0.0526,0.3627403974533081,3.4876244727530656e-05,1.4010274772320386,9000,,,,,,,,,,,,,,,,,,,
37
+ ,,,1.4010274772320386,9000,0.0745643824338913,0.9975749102093014,0.9972690713005937,0.9723900247831475,0.9756176280729699,0.9046505093574524,0.9750247388193672,0.975305785387727,0.9791024213637493,0.9746829301427421,0.9701078820541053,252.119,905.866,14.156,,,,,
38
+ 0.052,0.7762022614479065,2.693721991111627e-05,1.4788666614773878,9500,,,,,,,,,,,,,,,,,,,
39
+ ,,,1.4788666614773878,9500,0.07452459633350372,0.9977328074904859,0.9974410894856215,0.9730508384452147,0.975759591492858,0.9124361872673035,0.9755895720403177,0.976881986981624,0.978501686063742,0.9740254712964038,0.9720781541254986,252.2439,905.417,14.149,,,,,
40
+ 0.0507,0.3820905387401581,1.9831740005311437e-05,1.5567058457227367,10000,,,,,,,,,,,,,,,,,,,
41
+ ,,,1.5567058457227367,10000,0.08428945392370224,0.9975299683849125,0.9972155352602694,0.9700319035460719,0.9748023112122028,0.9334307909011841,0.9728135700086695,0.9755487501803781,0.9746970291636964,0.9695218431614696,0.9705425008933831,252.2555,905.376,14.148,,,,,
42
+ 0.0515,0.36162489652633667,1.3676438758331925e-05,1.634545029968086,10500,,,,,,,,,,,,,,,,,,,
43
+ ,,,1.634545029968086,10500,0.07442453503608704,0.9977306493713021,0.997440190110606,0.9729125537103704,0.9761234031726127,0.9136765599250793,0.9754888653420087,0.9760021075993326,0.9792385880317509,0.9748654545454546,0.9709674615362327,252.2157,905.519,14.151,,,,,
44
+ 0.0512,0.5945746302604675,8.572353097359252e-06,1.7123842142134351,11000,,,,,,,,,,,,,,,,,,,
45
+ ,,,1.7123842142134351,11000,0.07524814456701279,0.9977685405428793,0.9974825248399177,0.9727850366057699,0.9761874492694766,0.9207897186279297,0.975357508778997,0.9763919857424856,0.978581784103743,0.9741039521978714,0.9714696877505095,252.2195,905.505,14.15,,,,,
46
+ 0.0509,0.9120739698410034,4.603264645836933e-06,1.7902233984587843,11500,,,,,,,,,,,,,,,,,,,
47
+ ,,,1.7902233984587843,11500,0.06851697713136673,0.9979216958335029,0.997654853113855,0.9747033543129303,0.9769461620177022,0.9124361872673035,0.9771395794838563,0.9764685264549843,0.9818417743317821,0.9779586201532299,0.9714696877505095,252.8119,903.383,14.117,,,,,
48
+ 0.048,0.6627203822135925,1.834324480010042e-06,1.8680625827041333,12000,,,,,,,,,,,,,,,,,,,
49
+ ,,,1.8680625827041333,12000,0.07031949609518051,0.9978716433195517,0.9975978286587303,0.9741437319971922,0.9765590576618682,0.9241418242454529,0.9766141532318093,0.9766512444160816,0.980664333143768,0.9765690214120707,0.9717304590540763,252.5244,904.411,14.133,,,,,
50
+ 0.0488,0.7994762659072876,3.1098369880601253e-07,1.9459017669494822,12500,,,,,,,,,,,,,,,,,,,
51
+ ,,,1.9459017669494822,12500,0.07445533573627472,0.9977876009119543,0.997504157410398,0.972836356080046,0.9761660160257996,0.929440438747406,0.9753881586436997,0.9769268924908395,0.9780771664517369,0.9735279325286289,0.9721457615004974,252.7495,903.606,14.121,,,,,
52
+ ,,,2.0,12848,,,,,,,,,,,,,,,15236.3158,215.85,0.843,3.0700924448014336e+18,0.07117979241486355