Add calibration.json for temperature scaling metrics, remove predictions_val.csv, and update thresholds in results.json and threshold.json. Update README.md to reflect changes in metrics and file structure.
Browse files- .ipynb_checkpoints/README-checkpoint.md +0 -167
- .ipynb_checkpoints/predictions_test-checkpoint.csv +0 -0
- .ipynb_checkpoints/predictions_val-checkpoint.csv +0 -0
- .ipynb_checkpoints/results-checkpoint.json +0 -94
- .ipynb_checkpoints/threshold-checkpoint.json +0 -9
- README.md +78 -30
- adapter/.ipynb_checkpoints/README-checkpoint.md +0 -206
- adapter/.ipynb_checkpoints/adapter_config-checkpoint.json +0 -41
- adapter/README.md +1 -1
- adapter/adapter_config.json +8 -2
- adapter/adapter_model.safetensors +2 -2
- calibration.json +26 -0
- merged_model/added_tokens.json +3 -0
- merged_model/config.json +44 -0
- merged_model/model.safetensors +3 -0
- merged_model/special_tokens_map.json +51 -0
- classifier_head.pt → merged_model/spm.model +2 -2
- merged_model/tokenizer_config.json +59 -0
- predictions_calib.csv +0 -0
- predictions_test.csv +0 -0
- predictions_val.csv +0 -0
- results.json +71 -45
- threshold.json +4 -4
- training_log_history.csv +51 -143
.ipynb_checkpoints/README-checkpoint.md
DELETED
|
@@ -1,167 +0,0 @@
|
|
| 1 |
-
# AI Detector LoRA (DeBERTa-v3-large)
|
| 2 |
-
|
| 3 |
-
LoRA adapter for binary AI-text vs Human-text detection, trained on ~2.3M English samples
|
| 4 |
-
(`label: 1 = AI, 0 = Human`) using `microsoft/deberta-v3-large` as the base model.
|
| 5 |
-
|
| 6 |
-
- **Base model:** `microsoft/deberta-v3-large`
|
| 7 |
-
- **Task:** Binary classification (AI vs Human)
|
| 8 |
-
- **Head:** Single-logit + `BCEWithLogitsLoss`
|
| 9 |
-
- **Adapter type:** LoRA (`peft`)
|
| 10 |
-
- **Hardware:** H100 SXM, bf16, multi-GPU
|
| 11 |
-
- **Final decision threshold:** **0.9033** (max-F1 on validation)
|
| 12 |
-
|
| 13 |
-
---
|
| 14 |
-
|
| 15 |
-
## Files in this repo
|
| 16 |
-
|
| 17 |
-
- `adapter/` – LoRA weights saved with `peft_model.save_pretrained(...)`
|
| 18 |
-
- `threshold.json` – chosen deployment threshold and validation F1
|
| 19 |
-
- `results.json` – hyperparameters, validation threshold search, test metrics
|
| 20 |
-
- `training_log_history.csv` – raw Trainer log history
|
| 21 |
-
- `predictions_val.csv` – validation probabilities and labels
|
| 22 |
-
- `predictions_test.csv` – test probabilities and labels
|
| 23 |
-
- `figures/` – training and evaluation plots
|
| 24 |
-
- `README.md` – this file
|
| 25 |
-
|
| 26 |
-
---
|
| 27 |
-
|
| 28 |
-
## Metrics (test set)
|
| 29 |
-
|
| 30 |
-
Using threshold **0.9033**:
|
| 31 |
-
|
| 32 |
-
| Metric | Value |
|
| 33 |
-
|--------------|---------|
|
| 34 |
-
| AUROC | 0.9970 |
|
| 35 |
-
| Average Precision (AP) | 0.9966 |
|
| 36 |
-
| F1 | 0.9740 |
|
| 37 |
-
| Accuracy | 0.9767 |
|
| 38 |
-
| Precision | 0.9857 |
|
| 39 |
-
| Recall | 0.9625 |
|
| 40 |
-
| Specificity | 0.9884 |
|
| 41 |
-
|
| 42 |
-
Confusion matrix (test):
|
| 43 |
-
|
| 44 |
-
- **True Negatives (Human correctly)**: 123,399
|
| 45 |
-
- **False Positives (Human → AI)**: 1,449
|
| 46 |
-
- **False Negatives (AI → Human)**: 3,882
|
| 47 |
-
- **True Positives (AI correctly)**: 99,657
|
| 48 |
-
|
| 49 |
-
---
|
| 50 |
-
|
| 51 |
-
## Plots
|
| 52 |
-
|
| 53 |
-
### Training & validation
|
| 54 |
-
|
| 55 |
-
- Learning curves:
|
| 56 |
-
|
| 57 |
-

|
| 58 |
-
|
| 59 |
-
- Eval metrics over time:
|
| 60 |
-
|
| 61 |
-

|
| 62 |
-
|
| 63 |
-
### Validation set
|
| 64 |
-
|
| 65 |
-
- ROC:
|
| 66 |
-
|
| 67 |
-

|
| 68 |
-
|
| 69 |
-
- Precision–Recall:
|
| 70 |
-
|
| 71 |
-

|
| 72 |
-
|
| 73 |
-
- Calibration curve:
|
| 74 |
-
|
| 75 |
-

|
| 76 |
-
|
| 77 |
-
- F1 vs threshold:
|
| 78 |
-
|
| 79 |
-

|
| 80 |
-
|
| 81 |
-
### Test set
|
| 82 |
-
|
| 83 |
-
- ROC:
|
| 84 |
-
|
| 85 |
-

|
| 86 |
-
|
| 87 |
-
- Precision–Recall:
|
| 88 |
-
|
| 89 |
-

|
| 90 |
-
|
| 91 |
-
- Calibration curve:
|
| 92 |
-
|
| 93 |
-

|
| 94 |
-
|
| 95 |
-
- Confusion matrix:
|
| 96 |
-
|
| 97 |
-

|
| 98 |
-
|
| 99 |
-
---
|
| 100 |
-
|
| 101 |
-
## Usage
|
| 102 |
-
|
| 103 |
-
### Load base + LoRA adapter
|
| 104 |
-
|
| 105 |
-
```python
|
| 106 |
-
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 107 |
-
from peft import PeftModel
|
| 108 |
-
import torch
|
| 109 |
-
import json
|
| 110 |
-
|
| 111 |
-
base_model_id = "microsoft/deberta-v3-large"
|
| 112 |
-
adapter_id = "<your-username>/<your-private-repo>"
|
| 113 |
-
|
| 114 |
-
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
|
| 115 |
-
|
| 116 |
-
base_model = AutoModelForSequenceClassification.from_pretrained(
|
| 117 |
-
base_model_id,
|
| 118 |
-
num_labels=1, # single logit for BCEWithLogitsLoss
|
| 119 |
-
)
|
| 120 |
-
model = PeftModel.from_pretrained(base_model, adapter_id)
|
| 121 |
-
model.eval()
|
| 122 |
-
````
|
| 123 |
-
|
| 124 |
-
### Inference with threshold
|
| 125 |
-
|
| 126 |
-
```python
|
| 127 |
-
# load threshold
|
| 128 |
-
with open("threshold.json") as f:
|
| 129 |
-
thr = json.load(f)["threshold"] # 0.9033
|
| 130 |
-
|
| 131 |
-
def predict_proba(texts):
|
| 132 |
-
enc = tokenizer(
|
| 133 |
-
texts,
|
| 134 |
-
padding=True,
|
| 135 |
-
truncation=True,
|
| 136 |
-
max_length=512,
|
| 137 |
-
return_tensors="pt",
|
| 138 |
-
)
|
| 139 |
-
with torch.no_grad():
|
| 140 |
-
logits = model(**enc).logits.squeeze(-1)
|
| 141 |
-
probs = torch.sigmoid(logits)
|
| 142 |
-
return probs.cpu().numpy()
|
| 143 |
-
|
| 144 |
-
def predict_label(texts, threshold=thr):
|
| 145 |
-
probs = predict_proba(texts)
|
| 146 |
-
return (probs >= threshold).astype(int)
|
| 147 |
-
|
| 148 |
-
# example
|
| 149 |
-
texts = ["Some example text to classify"]
|
| 150 |
-
probs = predict_proba(texts)
|
| 151 |
-
labels = predict_label(texts)
|
| 152 |
-
print(probs, labels) # label 1 = AI, 0 = Human
|
| 153 |
-
```
|
| 154 |
-
|
| 155 |
-
---
|
| 156 |
-
|
| 157 |
-
## Notes
|
| 158 |
-
|
| 159 |
-
* Classifier head is **trainable** together with LoRA layers (unfrozen after applying PEFT).
|
| 160 |
-
* Training used:
|
| 161 |
-
|
| 162 |
-
* `bf16=True`
|
| 163 |
-
* `optim="adamw_torch_fused"`
|
| 164 |
-
* cosine-with-restarts scheduler
|
| 165 |
-
* LR scaled down from HPO to account for full-dataset (~14k steps).
|
| 166 |
-
* Threshold `0.9033` was chosen as the **max-F1** point on the validation set.
|
| 167 |
-
You can adjust it if you prefer fewer false positives or fewer false negatives.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.ipynb_checkpoints/predictions_test-checkpoint.csv
DELETED
|
The diff for this file is too large to render.
See raw diff
|
|
|
.ipynb_checkpoints/predictions_val-checkpoint.csv
DELETED
|
The diff for this file is too large to render.
See raw diff
|
|
|
.ipynb_checkpoints/results-checkpoint.json
DELETED
|
@@ -1,94 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"hyperparameters": {
|
| 3 |
-
"include_attention_output_dense": false,
|
| 4 |
-
"learning_rate_sampled": 0.00044569416489470884,
|
| 5 |
-
"weight_decay": 0.022491619139739856,
|
| 6 |
-
"warmup_ratio": 0.0463266472104081,
|
| 7 |
-
"lr_scheduler_num_cycles": 1,
|
| 8 |
-
"per_device_train_batch_size": 8,
|
| 9 |
-
"gradient_accumulation_steps": 4,
|
| 10 |
-
"num_train_epochs": 2,
|
| 11 |
-
"lora_r": 32,
|
| 12 |
-
"lora_alpha": 128,
|
| 13 |
-
"lora_dropout": 0.0,
|
| 14 |
-
"lora_target_modules": [
|
| 15 |
-
"query_proj",
|
| 16 |
-
"key_proj",
|
| 17 |
-
"value_proj"
|
| 18 |
-
],
|
| 19 |
-
"learning_rate": 4.456941648947089e-05,
|
| 20 |
-
"lr_scheduler_type": "cosine_with_restarts",
|
| 21 |
-
"max_grad_norm": 0.5,
|
| 22 |
-
"optim": "adamw_torch_fused"
|
| 23 |
-
},
|
| 24 |
-
"threshold_optimization": {
|
| 25 |
-
"max_f1": {
|
| 26 |
-
"threshold": 0.9032942056655884,
|
| 27 |
-
"metrics": {
|
| 28 |
-
"threshold": 0.9032942056655884,
|
| 29 |
-
"auroc": 0.9969044529302581,
|
| 30 |
-
"average_precision": 0.9965060417039346,
|
| 31 |
-
"f1": 0.9734939759036144,
|
| 32 |
-
"accuracy": 0.9762551119595773,
|
| 33 |
-
"precision": 0.9854536098796707,
|
| 34 |
-
"recall": 0.9618211495185389,
|
| 35 |
-
"specificity": 0.9882255881198587,
|
| 36 |
-
"precision_human": 0.9689546846776094,
|
| 37 |
-
"recall_human": 0.9882255881198587,
|
| 38 |
-
"precision_ai": 0.9854536098796707,
|
| 39 |
-
"recall_ai": 0.9618211495185389,
|
| 40 |
-
"confusion_matrix": {
|
| 41 |
-
"true_negative": 123377,
|
| 42 |
-
"false_positive": 1470,
|
| 43 |
-
"false_negative": 3953,
|
| 44 |
-
"true_positive": 99586
|
| 45 |
-
}
|
| 46 |
-
}
|
| 47 |
-
},
|
| 48 |
-
"precision_at_95recall": {
|
| 49 |
-
"threshold": 5.1442217227304354e-05,
|
| 50 |
-
"metrics": {
|
| 51 |
-
"threshold": 5.1442217227304354e-05,
|
| 52 |
-
"auroc": 0.9969044529302581,
|
| 53 |
-
"average_precision": 0.9965060417039346,
|
| 54 |
-
"f1": 0.6238698501167432,
|
| 55 |
-
"accuracy": 0.45335090592242955,
|
| 56 |
-
"precision": 0.45335090592242955,
|
| 57 |
-
"recall": 1.0,
|
| 58 |
-
"specificity": 0.0,
|
| 59 |
-
"precision_human": 0.0,
|
| 60 |
-
"recall_human": 0.0,
|
| 61 |
-
"precision_ai": 0.45335090592242955,
|
| 62 |
-
"recall_ai": 1.0,
|
| 63 |
-
"confusion_matrix": {
|
| 64 |
-
"true_negative": 0,
|
| 65 |
-
"false_positive": 124847,
|
| 66 |
-
"false_negative": 0,
|
| 67 |
-
"true_positive": 103539
|
| 68 |
-
}
|
| 69 |
-
}
|
| 70 |
-
}
|
| 71 |
-
},
|
| 72 |
-
"test_metrics": {
|
| 73 |
-
"threshold": 0.9032942056655884,
|
| 74 |
-
"auroc": 0.9970131530896283,
|
| 75 |
-
"average_precision": 0.9966291954050931,
|
| 76 |
-
"f1": 0.9739500109946493,
|
| 77 |
-
"accuracy": 0.976658040956797,
|
| 78 |
-
"precision": 0.9856685063200997,
|
| 79 |
-
"recall": 0.9625068814649552,
|
| 80 |
-
"specificity": 0.9883938869665513,
|
| 81 |
-
"precision_human": 0.9695005538925684,
|
| 82 |
-
"recall_human": 0.9883938869665513,
|
| 83 |
-
"precision_ai": 0.9856685063200997,
|
| 84 |
-
"recall_ai": 0.9625068814649552,
|
| 85 |
-
"confusion_matrix": {
|
| 86 |
-
"true_negative": 123399,
|
| 87 |
-
"false_positive": 1449,
|
| 88 |
-
"false_negative": 3882,
|
| 89 |
-
"true_positive": 99657
|
| 90 |
-
}
|
| 91 |
-
},
|
| 92 |
-
"timestamp": "20251113_111139",
|
| 93 |
-
"seed": 42
|
| 94 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.ipynb_checkpoints/threshold-checkpoint.json
DELETED
|
@@ -1,9 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"threshold": 0.9032942056655884,
|
| 3 |
-
"method": "max_f1",
|
| 4 |
-
"validation_f1": 0.9734939759036144,
|
| 5 |
-
"alternative_thresholds": {
|
| 6 |
-
"max_f1": 0.9032942056655884,
|
| 7 |
-
"precision_at_95recall": 5.1442217227304354e-05
|
| 8 |
-
}
|
| 9 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
|
@@ -28,17 +28,19 @@ LoRA adapter for binary AI-text vs Human-text detection, trained on ~2.3M Englis
|
|
| 28 |
- **Head:** Single-logit + `BCEWithLogitsLoss`
|
| 29 |
- **Adapter type:** LoRA (`peft`)
|
| 30 |
- **Hardware:** H100 SXM, bf16, multi-GPU
|
| 31 |
-
- **Final decision threshold:** **0.
|
| 32 |
|
| 33 |
---
|
| 34 |
|
| 35 |
## Files in this repo
|
| 36 |
|
| 37 |
- `adapter/` – LoRA weights saved with `peft_model.save_pretrained(...)`
|
|
|
|
| 38 |
- `threshold.json` – chosen deployment threshold and validation F1
|
|
|
|
| 39 |
- `results.json` – hyperparameters, validation threshold search, test metrics
|
| 40 |
- `training_log_history.csv` – raw Trainer log history
|
| 41 |
-
- `
|
| 42 |
- `predictions_test.csv` – test probabilities and labels
|
| 43 |
- `figures/` – training and evaluation plots
|
| 44 |
- `README.md` – this file
|
|
@@ -47,24 +49,32 @@ LoRA adapter for binary AI-text vs Human-text detection, trained on ~2.3M Englis
|
|
| 47 |
|
| 48 |
## Metrics (test set)
|
| 49 |
|
| 50 |
-
Using threshold **0.
|
| 51 |
|
| 52 |
-
| Metric
|
| 53 |
-
|
| 54 |
-
| AUROC
|
| 55 |
-
| Average Precision (AP) | 0.
|
| 56 |
-
| F1
|
| 57 |
-
| Accuracy
|
| 58 |
-
| Precision
|
| 59 |
-
| Recall
|
| 60 |
-
| Specificity
|
| 61 |
|
| 62 |
Confusion matrix (test):
|
| 63 |
|
| 64 |
-
- **True Negatives (Human correctly)**: 123,
|
| 65 |
-
- **False Positives (Human → AI)**:
|
| 66 |
-
- **False Negatives (AI → Human)**: 3,
|
| 67 |
-
- **True Positives (AI correctly)**: 99,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
|
| 69 |
---
|
| 70 |
|
|
@@ -84,19 +94,19 @@ Confusion matrix (test):
|
|
| 84 |
|
| 85 |
- ROC:
|
| 86 |
|
| 87 |
-
![ROC (
|
| 88 |
|
| 89 |
- Precision–Recall:
|
| 90 |
|
| 91 |
-
![PR (
|
| 92 |
|
| 93 |
- Calibration curve:
|
| 94 |
|
| 95 |
-
![Calibration (
|
| 96 |
|
| 97 |
- F1 vs threshold:
|
| 98 |
|
| 99 |
-
![F1 vs threshold (
|
| 100 |
|
| 101 |
### Test set
|
| 102 |
|
|
@@ -129,7 +139,7 @@ import torch
|
|
| 129 |
import json
|
| 130 |
|
| 131 |
base_model_id = "microsoft/deberta-v3-large"
|
| 132 |
-
adapter_id = "stealthcode/ai-detection"
|
| 133 |
|
| 134 |
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
|
| 135 |
|
|
@@ -139,14 +149,14 @@ base_model = AutoModelForSequenceClassification.from_pretrained(
|
|
| 139 |
)
|
| 140 |
model = PeftModel.from_pretrained(base_model, adapter_id)
|
| 141 |
model.eval()
|
| 142 |
-
|
| 143 |
|
| 144 |
### Inference with threshold
|
| 145 |
|
| 146 |
```python
|
| 147 |
# load threshold
|
| 148 |
with open("threshold.json") as f:
|
| 149 |
-
thr = json.load(f)["threshold"] # 0.
|
| 150 |
|
| 151 |
def predict_proba(texts):
|
| 152 |
enc = tokenizer(
|
|
@@ -172,16 +182,54 @@ labels = predict_label(texts)
|
|
| 172 |
print(probs, labels) # label 1 = AI, 0 = Human
|
| 173 |
```
|
| 174 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 175 |
---
|
| 176 |
|
| 177 |
## Notes
|
| 178 |
|
| 179 |
-
|
| 180 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 181 |
|
| 182 |
-
|
| 183 |
-
* `optim="adamw_torch_fused"`
|
| 184 |
-
* cosine-with-restarts scheduler
|
| 185 |
-
* LR scaled down from HPO to account for full-dataset (~14k steps).
|
| 186 |
-
* Threshold `0.9033` was chosen as the **max-F1** point on the validation set.
|
| 187 |
You can adjust it if you prefer fewer false positives or fewer false negatives.
|
|
|
|
| 28 |
- **Head:** Single-logit + `BCEWithLogitsLoss`
|
| 29 |
- **Adapter type:** LoRA (`peft`)
|
| 30 |
- **Hardware:** H100 SXM, bf16, multi-GPU
|
| 31 |
+
- **Final decision threshold:** **0.9284** (max-F1 on calibration set)
|
| 32 |
|
| 33 |
---
|
| 34 |
|
| 35 |
## Files in this repo
|
| 36 |
|
| 37 |
- `adapter/` – LoRA weights saved with `peft_model.save_pretrained(...)`
|
| 38 |
+
- `merged_model/` – fully merged model (base + LoRA) for standalone use
|
| 39 |
- `threshold.json` – chosen deployment threshold and validation F1
|
| 40 |
+
- `calibration.json` – temperature scaling parameters and calibration metrics
|
| 41 |
- `results.json` – hyperparameters, validation threshold search, test metrics
|
| 42 |
- `training_log_history.csv` – raw Trainer log history
|
| 43 |
+
- `predictions_calib.csv` – calibration-set probabilities and labels
|
| 44 |
- `predictions_test.csv` – test probabilities and labels
|
| 45 |
- `figures/` – training and evaluation plots
|
| 46 |
- `README.md` – this file
|
|
|
|
| 49 |
|
| 50 |
## Metrics (test set)
|
| 51 |
|
| 52 |
+
Using threshold **0.9284**:
|
| 53 |
|
| 54 |
+
| Metric | Value |
|
| 55 |
+
| ---------------------- | ------ |
|
| 56 |
+
| AUROC | 0.9979 |
|
| 57 |
+
| Average Precision (AP) | 0.9977 |
|
| 58 |
+
| F1 | 0.9773 |
|
| 59 |
+
| Accuracy | 0.9797 |
|
| 60 |
+
| Precision | 0.9909 |
|
| 61 |
+
| Recall | 0.9640 |
|
| 62 |
+
| Specificity | 0.9927 |
|
| 63 |
|
| 64 |
Confusion matrix (test):
|
| 65 |
|
| 66 |
+
- **True Negatives (Human correctly)**: 123,936
|
| 67 |
+
- **False Positives (Human → AI)**: 912
|
| 68 |
+
- **False Negatives (AI → Human)**: 3,723
|
| 69 |
+
- **True Positives (AI correctly)**: 99,816
|
| 70 |
+
|
| 71 |
+
### Calibration
|
| 72 |
+
|
| 73 |
+
- **Method:** temperature scaling
|
| 74 |
+
- **Temperature (T):** 1.2807
|
| 75 |
+
- **Calibration set:** calibration
|
| 76 |
+
- Test ECE: 0.0119 → 0.0159 (after calibration)
|
| 77 |
+
- Test Brier: 0.01812 → 0.01829 (after calibration)
|
| 78 |
|
| 79 |
---
|
| 80 |
|
|
|
|
| 94 |
|
| 95 |
- ROC:
|
| 96 |
|
| 97 |
+

|
| 98 |
|
| 99 |
- Precision–Recall:
|
| 100 |
|
| 101 |
+

|
| 102 |
|
| 103 |
- Calibration curve:
|
| 104 |
|
| 105 |
+

|
| 106 |
|
| 107 |
- F1 vs threshold:
|
| 108 |
|
| 109 |
+

|
| 110 |
|
| 111 |
### Test set
|
| 112 |
|
|
|
|
| 139 |
import json
|
| 140 |
|
| 141 |
base_model_id = "microsoft/deberta-v3-large"
|
| 142 |
+
adapter_id = "stealthcode/ai-detection" # or local: "./adapter"
|
| 143 |
|
| 144 |
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
|
| 145 |
|
|
|
|
| 149 |
)
|
| 150 |
model = PeftModel.from_pretrained(base_model, adapter_id)
|
| 151 |
model.eval()
|
| 152 |
+
```
|
| 153 |
|
| 154 |
### Inference with threshold
|
| 155 |
|
| 156 |
```python
|
| 157 |
# load threshold
|
| 158 |
with open("threshold.json") as f:
|
| 159 |
+
thr = json.load(f)["threshold"] # 0.9284
|
| 160 |
|
| 161 |
def predict_proba(texts):
|
| 162 |
enc = tokenizer(
|
|
|
|
| 182 |
print(probs, labels) # label 1 = AI, 0 = Human
|
| 183 |
```
|
| 184 |
|
| 185 |
+
### Load merged model (no PEFT required)
|
| 186 |
+
|
| 187 |
+
```python
|
| 188 |
+
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 189 |
+
import torch, json
|
| 190 |
+
|
| 191 |
+
model_dir = "./merged_model"
|
| 192 |
+
tokenizer = AutoTokenizer.from_pretrained(model_dir)
|
| 193 |
+
model = AutoModelForSequenceClassification.from_pretrained(model_dir)
|
| 194 |
+
model.eval()
|
| 195 |
+
|
| 196 |
+
with open("threshold.json") as f:
|
| 197 |
+
thr = json.load(f)["threshold"] # 0.9284
|
| 198 |
+
|
| 199 |
+
def predict_proba(texts):
|
| 200 |
+
enc = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
|
| 201 |
+
with torch.no_grad():
|
| 202 |
+
logits = model(**enc).logits.squeeze(-1)
|
| 203 |
+
probs = torch.sigmoid(logits)
|
| 204 |
+
return probs.cpu().numpy()
|
| 205 |
+
```
|
| 206 |
+
|
| 207 |
+
### Optional: apply temperature scaling to logits
|
| 208 |
+
|
| 209 |
+
```python
|
| 210 |
+
import json
|
| 211 |
+
with open("calibration.json") as f:
|
| 212 |
+
T = json.load(f)["temperature"] # e.g., 1.2807
|
| 213 |
+
|
| 214 |
+
def predict_proba_calibrated(texts):
|
| 215 |
+
enc = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
|
| 216 |
+
with torch.no_grad():
|
| 217 |
+
logits = model(**enc).logits.squeeze(-1)
|
| 218 |
+
probs = torch.sigmoid(logits / T)
|
| 219 |
+
return probs.cpu().numpy()
|
| 220 |
+
```
|
| 221 |
+
|
| 222 |
---
|
| 223 |
|
| 224 |
## Notes
|
| 225 |
|
| 226 |
+
- Classifier head is **trainable** together with LoRA layers (unfrozen after applying PEFT).
|
| 227 |
+
- Training used:
|
| 228 |
+
|
| 229 |
+
- `bf16=True`
|
| 230 |
+
- `optim="adamw_torch_fused"`
|
| 231 |
+
- cosine-with-restarts scheduler
|
| 232 |
+
- LR scaled down from HPO to account for full-dataset (~14k steps).
|
| 233 |
|
| 234 |
+
- Threshold `0.9284` was chosen as the **max-F1** point on the calibration set.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 235 |
You can adjust it if you prefer fewer false positives or fewer false negatives.
|
adapter/.ipynb_checkpoints/README-checkpoint.md
DELETED
|
@@ -1,206 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
base_model: microsoft/deberta-v3-large
|
| 3 |
-
library_name: peft
|
| 4 |
-
tags:
|
| 5 |
-
- base_model:adapter:microsoft/deberta-v3-large
|
| 6 |
-
- lora
|
| 7 |
-
- transformers
|
| 8 |
-
---
|
| 9 |
-
|
| 10 |
-
# Model Card for Model ID
|
| 11 |
-
|
| 12 |
-
<!-- Provide a quick summary of what the model is/does. -->
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
## Model Details
|
| 17 |
-
|
| 18 |
-
### Model Description
|
| 19 |
-
|
| 20 |
-
<!-- Provide a longer summary of what this model is. -->
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
- **Developed by:** [More Information Needed]
|
| 25 |
-
- **Funded by [optional]:** [More Information Needed]
|
| 26 |
-
- **Shared by [optional]:** [More Information Needed]
|
| 27 |
-
- **Model type:** [More Information Needed]
|
| 28 |
-
- **Language(s) (NLP):** [More Information Needed]
|
| 29 |
-
- **License:** [More Information Needed]
|
| 30 |
-
- **Finetuned from model [optional]:** [More Information Needed]
|
| 31 |
-
|
| 32 |
-
### Model Sources [optional]
|
| 33 |
-
|
| 34 |
-
<!-- Provide the basic links for the model. -->
|
| 35 |
-
|
| 36 |
-
- **Repository:** [More Information Needed]
|
| 37 |
-
- **Paper [optional]:** [More Information Needed]
|
| 38 |
-
- **Demo [optional]:** [More Information Needed]
|
| 39 |
-
|
| 40 |
-
## Uses
|
| 41 |
-
|
| 42 |
-
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 43 |
-
|
| 44 |
-
### Direct Use
|
| 45 |
-
|
| 46 |
-
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 47 |
-
|
| 48 |
-
[More Information Needed]
|
| 49 |
-
|
| 50 |
-
### Downstream Use [optional]
|
| 51 |
-
|
| 52 |
-
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 53 |
-
|
| 54 |
-
[More Information Needed]
|
| 55 |
-
|
| 56 |
-
### Out-of-Scope Use
|
| 57 |
-
|
| 58 |
-
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 59 |
-
|
| 60 |
-
[More Information Needed]
|
| 61 |
-
|
| 62 |
-
## Bias, Risks, and Limitations
|
| 63 |
-
|
| 64 |
-
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 65 |
-
|
| 66 |
-
[More Information Needed]
|
| 67 |
-
|
| 68 |
-
### Recommendations
|
| 69 |
-
|
| 70 |
-
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 71 |
-
|
| 72 |
-
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 73 |
-
|
| 74 |
-
## How to Get Started with the Model
|
| 75 |
-
|
| 76 |
-
Use the code below to get started with the model.
|
| 77 |
-
|
| 78 |
-
[More Information Needed]
|
| 79 |
-
|
| 80 |
-
## Training Details
|
| 81 |
-
|
| 82 |
-
### Training Data
|
| 83 |
-
|
| 84 |
-
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 85 |
-
|
| 86 |
-
[More Information Needed]
|
| 87 |
-
|
| 88 |
-
### Training Procedure
|
| 89 |
-
|
| 90 |
-
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 91 |
-
|
| 92 |
-
#### Preprocessing [optional]
|
| 93 |
-
|
| 94 |
-
[More Information Needed]
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
#### Training Hyperparameters
|
| 98 |
-
|
| 99 |
-
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 100 |
-
|
| 101 |
-
#### Speeds, Sizes, Times [optional]
|
| 102 |
-
|
| 103 |
-
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 104 |
-
|
| 105 |
-
[More Information Needed]
|
| 106 |
-
|
| 107 |
-
## Evaluation
|
| 108 |
-
|
| 109 |
-
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 110 |
-
|
| 111 |
-
### Testing Data, Factors & Metrics
|
| 112 |
-
|
| 113 |
-
#### Testing Data
|
| 114 |
-
|
| 115 |
-
<!-- This should link to a Dataset Card if possible. -->
|
| 116 |
-
|
| 117 |
-
[More Information Needed]
|
| 118 |
-
|
| 119 |
-
#### Factors
|
| 120 |
-
|
| 121 |
-
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 122 |
-
|
| 123 |
-
[More Information Needed]
|
| 124 |
-
|
| 125 |
-
#### Metrics
|
| 126 |
-
|
| 127 |
-
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 128 |
-
|
| 129 |
-
[More Information Needed]
|
| 130 |
-
|
| 131 |
-
### Results
|
| 132 |
-
|
| 133 |
-
[More Information Needed]
|
| 134 |
-
|
| 135 |
-
#### Summary
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
## Model Examination [optional]
|
| 140 |
-
|
| 141 |
-
<!-- Relevant interpretability work for the model goes here -->
|
| 142 |
-
|
| 143 |
-
[More Information Needed]
|
| 144 |
-
|
| 145 |
-
## Environmental Impact
|
| 146 |
-
|
| 147 |
-
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 148 |
-
|
| 149 |
-
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 150 |
-
|
| 151 |
-
- **Hardware Type:** [More Information Needed]
|
| 152 |
-
- **Hours used:** [More Information Needed]
|
| 153 |
-
- **Cloud Provider:** [More Information Needed]
|
| 154 |
-
- **Compute Region:** [More Information Needed]
|
| 155 |
-
- **Carbon Emitted:** [More Information Needed]
|
| 156 |
-
|
| 157 |
-
## Technical Specifications [optional]
|
| 158 |
-
|
| 159 |
-
### Model Architecture and Objective
|
| 160 |
-
|
| 161 |
-
[More Information Needed]
|
| 162 |
-
|
| 163 |
-
### Compute Infrastructure
|
| 164 |
-
|
| 165 |
-
[More Information Needed]
|
| 166 |
-
|
| 167 |
-
#### Hardware
|
| 168 |
-
|
| 169 |
-
[More Information Needed]
|
| 170 |
-
|
| 171 |
-
#### Software
|
| 172 |
-
|
| 173 |
-
[More Information Needed]
|
| 174 |
-
|
| 175 |
-
## Citation [optional]
|
| 176 |
-
|
| 177 |
-
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 178 |
-
|
| 179 |
-
**BibTeX:**
|
| 180 |
-
|
| 181 |
-
[More Information Needed]
|
| 182 |
-
|
| 183 |
-
**APA:**
|
| 184 |
-
|
| 185 |
-
[More Information Needed]
|
| 186 |
-
|
| 187 |
-
## Glossary [optional]
|
| 188 |
-
|
| 189 |
-
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 190 |
-
|
| 191 |
-
[More Information Needed]
|
| 192 |
-
|
| 193 |
-
## More Information [optional]
|
| 194 |
-
|
| 195 |
-
[More Information Needed]
|
| 196 |
-
|
| 197 |
-
## Model Card Authors [optional]
|
| 198 |
-
|
| 199 |
-
[More Information Needed]
|
| 200 |
-
|
| 201 |
-
## Model Card Contact
|
| 202 |
-
|
| 203 |
-
[More Information Needed]
|
| 204 |
-
### Framework versions
|
| 205 |
-
|
| 206 |
-
- PEFT 0.17.1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
adapter/.ipynb_checkpoints/adapter_config-checkpoint.json
DELETED
|
@@ -1,41 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"alpha_pattern": {},
|
| 3 |
-
"auto_mapping": null,
|
| 4 |
-
"base_model_name_or_path": "microsoft/deberta-v3-large",
|
| 5 |
-
"bias": "none",
|
| 6 |
-
"corda_config": null,
|
| 7 |
-
"eva_config": null,
|
| 8 |
-
"exclude_modules": null,
|
| 9 |
-
"fan_in_fan_out": false,
|
| 10 |
-
"inference_mode": true,
|
| 11 |
-
"init_lora_weights": true,
|
| 12 |
-
"layer_replication": null,
|
| 13 |
-
"layers_pattern": null,
|
| 14 |
-
"layers_to_transform": null,
|
| 15 |
-
"loftq_config": {},
|
| 16 |
-
"lora_alpha": 128,
|
| 17 |
-
"lora_bias": false,
|
| 18 |
-
"lora_dropout": 0.0,
|
| 19 |
-
"megatron_config": null,
|
| 20 |
-
"megatron_core": "megatron.core",
|
| 21 |
-
"modules_to_save": [
|
| 22 |
-
"classifier",
|
| 23 |
-
"score"
|
| 24 |
-
],
|
| 25 |
-
"peft_type": "LORA",
|
| 26 |
-
"qalora_group_size": 16,
|
| 27 |
-
"r": 32,
|
| 28 |
-
"rank_pattern": {},
|
| 29 |
-
"revision": null,
|
| 30 |
-
"target_modules": [
|
| 31 |
-
"query_proj",
|
| 32 |
-
"key_proj",
|
| 33 |
-
"value_proj"
|
| 34 |
-
],
|
| 35 |
-
"target_parameters": null,
|
| 36 |
-
"task_type": "SEQ_CLS",
|
| 37 |
-
"trainable_token_indices": null,
|
| 38 |
-
"use_dora": false,
|
| 39 |
-
"use_qalora": false,
|
| 40 |
-
"use_rslora": false
|
| 41 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
adapter/README.md
CHANGED
|
@@ -203,4 +203,4 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
|
|
| 203 |
[More Information Needed]
|
| 204 |
### Framework versions
|
| 205 |
|
| 206 |
-
- PEFT 0.
|
|
|
|
| 203 |
[More Information Needed]
|
| 204 |
### Framework versions
|
| 205 |
|
| 206 |
+
- PEFT 0.18.0
|
adapter/adapter_config.json
CHANGED
|
@@ -1,9 +1,12 @@
|
|
| 1 |
{
|
|
|
|
| 2 |
"alpha_pattern": {},
|
|
|
|
| 3 |
"auto_mapping": null,
|
| 4 |
"base_model_name_or_path": "microsoft/deberta-v3-large",
|
| 5 |
"bias": "none",
|
| 6 |
"corda_config": null,
|
|
|
|
| 7 |
"eva_config": null,
|
| 8 |
"exclude_modules": null,
|
| 9 |
"fan_in_fan_out": false,
|
|
@@ -19,18 +22,21 @@
|
|
| 19 |
"megatron_config": null,
|
| 20 |
"megatron_core": "megatron.core",
|
| 21 |
"modules_to_save": [
|
|
|
|
|
|
|
| 22 |
"classifier",
|
| 23 |
"score"
|
| 24 |
],
|
| 25 |
"peft_type": "LORA",
|
|
|
|
| 26 |
"qalora_group_size": 16,
|
| 27 |
"r": 32,
|
| 28 |
"rank_pattern": {},
|
| 29 |
"revision": null,
|
| 30 |
"target_modules": [
|
| 31 |
"query_proj",
|
| 32 |
-
"
|
| 33 |
-
"
|
| 34 |
],
|
| 35 |
"target_parameters": null,
|
| 36 |
"task_type": "SEQ_CLS",
|
|
|
|
| 1 |
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
"auto_mapping": null,
|
| 6 |
"base_model_name_or_path": "microsoft/deberta-v3-large",
|
| 7 |
"bias": "none",
|
| 8 |
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
"eva_config": null,
|
| 11 |
"exclude_modules": null,
|
| 12 |
"fan_in_fan_out": false,
|
|
|
|
| 22 |
"megatron_config": null,
|
| 23 |
"megatron_core": "megatron.core",
|
| 24 |
"modules_to_save": [
|
| 25 |
+
"classifier",
|
| 26 |
+
"pooler.dense",
|
| 27 |
"classifier",
|
| 28 |
"score"
|
| 29 |
],
|
| 30 |
"peft_type": "LORA",
|
| 31 |
+
"peft_version": "0.18.0",
|
| 32 |
"qalora_group_size": 16,
|
| 33 |
"r": 32,
|
| 34 |
"rank_pattern": {},
|
| 35 |
"revision": null,
|
| 36 |
"target_modules": [
|
| 37 |
"query_proj",
|
| 38 |
+
"value_proj",
|
| 39 |
+
"key_proj"
|
| 40 |
],
|
| 41 |
"target_parameters": null,
|
| 42 |
"task_type": "SEQ_CLS",
|
adapter/adapter_model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2320198a394f889f1be50d439b5217d996e84da64a4a65671bf653607879cfcb
|
| 3 |
+
size 23099012
|
calibration.json
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"temperature": 1.2806789875030518,
|
| 3 |
+
"method": "temperature_scaling",
|
| 4 |
+
"calibration_set": "calibration",
|
| 5 |
+
"calibration_metrics": {
|
| 6 |
+
"temperature": 1.2806789875030518,
|
| 7 |
+
"optimization_method": "LBFGS_logspace",
|
| 8 |
+
"uncalibrated_nll": 0.06460460661246972,
|
| 9 |
+
"calibrated_nll": 0.06279846573841724,
|
| 10 |
+
"uncalibrated_ece": 0.012124871567496009,
|
| 11 |
+
"calibrated_ece": 0.016240862688628014,
|
| 12 |
+
"uncalibrated_brier": 0.01822748637167701,
|
| 13 |
+
"calibrated_brier": 0.018437309998858068,
|
| 14 |
+
"nll_improvement": 0.001806140874052481,
|
| 15 |
+
"ece_improvement": -0.004115991121132005,
|
| 16 |
+
"brier_improvement": -0.00020982362718105843
|
| 17 |
+
},
|
| 18 |
+
"test_metrics": {
|
| 19 |
+
"ece_before": 0.011862174308705089,
|
| 20 |
+
"ece_after": 0.015908939599173937,
|
| 21 |
+
"ece_improvement": -0.004046765290468848,
|
| 22 |
+
"brier_before": 0.01812282837704726,
|
| 23 |
+
"brier_after": 0.018294590049400802,
|
| 24 |
+
"brier_improvement": -0.0001717616723535438
|
| 25 |
+
}
|
| 26 |
+
}
|
merged_model/added_tokens.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"[MASK]": 128000
|
| 3 |
+
}
|
merged_model/config.json
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"DebertaV2ForSequenceClassification"
|
| 4 |
+
],
|
| 5 |
+
"attention_probs_dropout_prob": 0.1,
|
| 6 |
+
"bos_token_id": 1,
|
| 7 |
+
"dtype": "float32",
|
| 8 |
+
"eos_token_id": 2,
|
| 9 |
+
"hidden_act": "gelu",
|
| 10 |
+
"hidden_dropout_prob": 0.1,
|
| 11 |
+
"hidden_size": 1024,
|
| 12 |
+
"id2label": {
|
| 13 |
+
"0": "LABEL_0"
|
| 14 |
+
},
|
| 15 |
+
"initializer_range": 0.02,
|
| 16 |
+
"intermediate_size": 4096,
|
| 17 |
+
"label2id": {
|
| 18 |
+
"LABEL_0": 0
|
| 19 |
+
},
|
| 20 |
+
"layer_norm_eps": 1e-07,
|
| 21 |
+
"legacy": true,
|
| 22 |
+
"max_position_embeddings": 512,
|
| 23 |
+
"max_relative_positions": -1,
|
| 24 |
+
"model_type": "deberta-v2",
|
| 25 |
+
"norm_rel_ebd": "layer_norm",
|
| 26 |
+
"num_attention_heads": 16,
|
| 27 |
+
"num_hidden_layers": 24,
|
| 28 |
+
"pad_token_id": 0,
|
| 29 |
+
"pooler_dropout": 0,
|
| 30 |
+
"pooler_hidden_act": "gelu",
|
| 31 |
+
"pooler_hidden_size": 1024,
|
| 32 |
+
"pos_att_type": [
|
| 33 |
+
"p2c",
|
| 34 |
+
"c2p"
|
| 35 |
+
],
|
| 36 |
+
"position_biased_input": false,
|
| 37 |
+
"position_buckets": 256,
|
| 38 |
+
"relative_attention": true,
|
| 39 |
+
"share_att_key": true,
|
| 40 |
+
"transformers_version": "4.57.1",
|
| 41 |
+
"type_vocab_size": 0,
|
| 42 |
+
"use_cache": false,
|
| 43 |
+
"vocab_size": 128100
|
| 44 |
+
}
|
merged_model/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f780611b3aca302b2c564ebcb57391dcced23c391e62f96ee4509fa71076185f
|
| 3 |
+
size 1740300340
|
merged_model/special_tokens_map.json
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token": {
|
| 3 |
+
"content": "[CLS]",
|
| 4 |
+
"lstrip": false,
|
| 5 |
+
"normalized": false,
|
| 6 |
+
"rstrip": false,
|
| 7 |
+
"single_word": false
|
| 8 |
+
},
|
| 9 |
+
"cls_token": {
|
| 10 |
+
"content": "[CLS]",
|
| 11 |
+
"lstrip": false,
|
| 12 |
+
"normalized": false,
|
| 13 |
+
"rstrip": false,
|
| 14 |
+
"single_word": false
|
| 15 |
+
},
|
| 16 |
+
"eos_token": {
|
| 17 |
+
"content": "[SEP]",
|
| 18 |
+
"lstrip": false,
|
| 19 |
+
"normalized": false,
|
| 20 |
+
"rstrip": false,
|
| 21 |
+
"single_word": false
|
| 22 |
+
},
|
| 23 |
+
"mask_token": {
|
| 24 |
+
"content": "[MASK]",
|
| 25 |
+
"lstrip": false,
|
| 26 |
+
"normalized": false,
|
| 27 |
+
"rstrip": false,
|
| 28 |
+
"single_word": false
|
| 29 |
+
},
|
| 30 |
+
"pad_token": {
|
| 31 |
+
"content": "[PAD]",
|
| 32 |
+
"lstrip": false,
|
| 33 |
+
"normalized": false,
|
| 34 |
+
"rstrip": false,
|
| 35 |
+
"single_word": false
|
| 36 |
+
},
|
| 37 |
+
"sep_token": {
|
| 38 |
+
"content": "[SEP]",
|
| 39 |
+
"lstrip": false,
|
| 40 |
+
"normalized": false,
|
| 41 |
+
"rstrip": false,
|
| 42 |
+
"single_word": false
|
| 43 |
+
},
|
| 44 |
+
"unk_token": {
|
| 45 |
+
"content": "[UNK]",
|
| 46 |
+
"lstrip": false,
|
| 47 |
+
"normalized": true,
|
| 48 |
+
"rstrip": false,
|
| 49 |
+
"single_word": false
|
| 50 |
+
}
|
| 51 |
+
}
|
classifier_head.pt → merged_model/spm.model
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c679fbf93643d19aab7ee10c0b99e460bdbc02fedf34b92b05af343b4af586fd
|
| 3 |
+
size 2464616
|
merged_model/tokenizer_config.json
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"added_tokens_decoder": {
|
| 3 |
+
"0": {
|
| 4 |
+
"content": "[PAD]",
|
| 5 |
+
"lstrip": false,
|
| 6 |
+
"normalized": false,
|
| 7 |
+
"rstrip": false,
|
| 8 |
+
"single_word": false,
|
| 9 |
+
"special": true
|
| 10 |
+
},
|
| 11 |
+
"1": {
|
| 12 |
+
"content": "[CLS]",
|
| 13 |
+
"lstrip": false,
|
| 14 |
+
"normalized": false,
|
| 15 |
+
"rstrip": false,
|
| 16 |
+
"single_word": false,
|
| 17 |
+
"special": true
|
| 18 |
+
},
|
| 19 |
+
"2": {
|
| 20 |
+
"content": "[SEP]",
|
| 21 |
+
"lstrip": false,
|
| 22 |
+
"normalized": false,
|
| 23 |
+
"rstrip": false,
|
| 24 |
+
"single_word": false,
|
| 25 |
+
"special": true
|
| 26 |
+
},
|
| 27 |
+
"3": {
|
| 28 |
+
"content": "[UNK]",
|
| 29 |
+
"lstrip": false,
|
| 30 |
+
"normalized": true,
|
| 31 |
+
"rstrip": false,
|
| 32 |
+
"single_word": false,
|
| 33 |
+
"special": true
|
| 34 |
+
},
|
| 35 |
+
"128000": {
|
| 36 |
+
"content": "[MASK]",
|
| 37 |
+
"lstrip": false,
|
| 38 |
+
"normalized": false,
|
| 39 |
+
"rstrip": false,
|
| 40 |
+
"single_word": false,
|
| 41 |
+
"special": true
|
| 42 |
+
}
|
| 43 |
+
},
|
| 44 |
+
"bos_token": "[CLS]",
|
| 45 |
+
"clean_up_tokenization_spaces": false,
|
| 46 |
+
"cls_token": "[CLS]",
|
| 47 |
+
"do_lower_case": false,
|
| 48 |
+
"eos_token": "[SEP]",
|
| 49 |
+
"extra_special_tokens": {},
|
| 50 |
+
"mask_token": "[MASK]",
|
| 51 |
+
"model_max_length": 1000000000000000019884624838656,
|
| 52 |
+
"pad_token": "[PAD]",
|
| 53 |
+
"sep_token": "[SEP]",
|
| 54 |
+
"sp_model_kwargs": {},
|
| 55 |
+
"split_by_punct": false,
|
| 56 |
+
"tokenizer_class": "DebertaV2Tokenizer",
|
| 57 |
+
"unk_token": "[UNK]",
|
| 58 |
+
"vocab_type": "spm"
|
| 59 |
+
}
|
predictions_calib.csv
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
predictions_test.csv
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
predictions_val.csv
DELETED
|
The diff for this file is too large to render.
See raw diff
|
|
|
results.json
CHANGED
|
@@ -16,79 +16,105 @@
|
|
| 16 |
"key_proj",
|
| 17 |
"value_proj"
|
| 18 |
],
|
| 19 |
-
"learning_rate":
|
| 20 |
"lr_scheduler_type": "cosine_with_restarts",
|
| 21 |
"max_grad_norm": 0.5,
|
| 22 |
"optim": "adamw_torch_fused"
|
| 23 |
},
|
| 24 |
"threshold_optimization": {
|
| 25 |
"max_f1": {
|
| 26 |
-
"threshold": 0.
|
| 27 |
"metrics": {
|
| 28 |
-
"threshold": 0.
|
| 29 |
-
"auroc": 0.
|
| 30 |
-
"average_precision": 0.
|
| 31 |
-
"f1": 0.
|
| 32 |
-
"accuracy": 0.
|
| 33 |
-
"precision": 0.
|
| 34 |
-
"recall": 0.
|
| 35 |
-
"specificity": 0.
|
| 36 |
-
"precision_human": 0.
|
| 37 |
-
"recall_human": 0.
|
| 38 |
-
"precision_ai": 0.
|
| 39 |
-
"recall_ai": 0.
|
| 40 |
"confusion_matrix": {
|
| 41 |
-
"true_negative":
|
| 42 |
-
"false_positive":
|
| 43 |
-
"false_negative":
|
| 44 |
-
"true_positive":
|
| 45 |
}
|
| 46 |
}
|
| 47 |
},
|
| 48 |
"precision_at_95recall": {
|
| 49 |
-
"threshold":
|
| 50 |
"metrics": {
|
| 51 |
-
"threshold":
|
| 52 |
-
"auroc": 0.
|
| 53 |
-
"average_precision": 0.
|
| 54 |
-
"f1": 0.
|
| 55 |
-
"accuracy": 0.
|
| 56 |
-
"precision": 0.
|
| 57 |
"recall": 1.0,
|
| 58 |
"specificity": 0.0,
|
| 59 |
"precision_human": 0.0,
|
| 60 |
"recall_human": 0.0,
|
| 61 |
-
"precision_ai": 0.
|
| 62 |
"recall_ai": 1.0,
|
| 63 |
"confusion_matrix": {
|
| 64 |
"true_negative": 0,
|
| 65 |
-
"false_positive":
|
| 66 |
"false_negative": 0,
|
| 67 |
-
"true_positive":
|
| 68 |
}
|
| 69 |
}
|
| 70 |
}
|
| 71 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
"test_metrics": {
|
| 73 |
-
"threshold": 0.
|
| 74 |
-
"auroc": 0.
|
| 75 |
-
"average_precision": 0.
|
| 76 |
-
"f1": 0.
|
| 77 |
-
"accuracy": 0.
|
| 78 |
-
"precision": 0.
|
| 79 |
-
"recall": 0.
|
| 80 |
-
"specificity": 0.
|
| 81 |
-
"precision_human": 0.
|
| 82 |
-
"recall_human": 0.
|
| 83 |
-
"precision_ai": 0.
|
| 84 |
-
"recall_ai": 0.
|
| 85 |
"confusion_matrix": {
|
| 86 |
-
"true_negative":
|
| 87 |
-
"false_positive":
|
| 88 |
-
"false_negative":
|
| 89 |
-
"true_positive":
|
| 90 |
}
|
| 91 |
},
|
| 92 |
-
"timestamp": "
|
| 93 |
"seed": 42
|
| 94 |
}
|
|
|
|
| 16 |
"key_proj",
|
| 17 |
"value_proj"
|
| 18 |
],
|
| 19 |
+
"learning_rate": 0.0001554357238163802,
|
| 20 |
"lr_scheduler_type": "cosine_with_restarts",
|
| 21 |
"max_grad_norm": 0.5,
|
| 22 |
"optim": "adamw_torch_fused"
|
| 23 |
},
|
| 24 |
"threshold_optimization": {
|
| 25 |
"max_f1": {
|
| 26 |
+
"threshold": 0.9284088015556335,
|
| 27 |
"metrics": {
|
| 28 |
+
"threshold": 0.9284088015556335,
|
| 29 |
+
"auroc": 0.9978600960936826,
|
| 30 |
+
"average_precision": 0.997597673288253,
|
| 31 |
+
"f1": 0.9773827668313225,
|
| 32 |
+
"accuracy": 0.9797765846236365,
|
| 33 |
+
"precision": 0.9912838341196921,
|
| 34 |
+
"recall": 0.9638661853653825,
|
| 35 |
+
"specificity": 0.9929714251386692,
|
| 36 |
+
"precision_human": 0.9707053998766749,
|
| 37 |
+
"recall_human": 0.9929714251386692,
|
| 38 |
+
"precision_ai": 0.9912838341196921,
|
| 39 |
+
"recall_ai": 0.9638661853653825,
|
| 40 |
"confusion_matrix": {
|
| 41 |
+
"true_negative": 99176,
|
| 42 |
+
"false_positive": 702,
|
| 43 |
+
"false_negative": 2993,
|
| 44 |
+
"true_positive": 79838
|
| 45 |
}
|
| 46 |
}
|
| 47 |
},
|
| 48 |
"precision_at_95recall": {
|
| 49 |
+
"threshold": 1.9947297005273867e-06,
|
| 50 |
"metrics": {
|
| 51 |
+
"threshold": 1.9947297005273867e-06,
|
| 52 |
+
"auroc": 0.9978600960936826,
|
| 53 |
+
"average_precision": 0.997597673288253,
|
| 54 |
+
"f1": 0.6238683437523537,
|
| 55 |
+
"accuracy": 0.45334931503100556,
|
| 56 |
+
"precision": 0.45334931503100556,
|
| 57 |
"recall": 1.0,
|
| 58 |
"specificity": 0.0,
|
| 59 |
"precision_human": 0.0,
|
| 60 |
"recall_human": 0.0,
|
| 61 |
+
"precision_ai": 0.45334931503100556,
|
| 62 |
"recall_ai": 1.0,
|
| 63 |
"confusion_matrix": {
|
| 64 |
"true_negative": 0,
|
| 65 |
+
"false_positive": 99878,
|
| 66 |
"false_negative": 0,
|
| 67 |
+
"true_positive": 82831
|
| 68 |
}
|
| 69 |
}
|
| 70 |
}
|
| 71 |
},
|
| 72 |
+
"calibration": {
|
| 73 |
+
"temperature": 1.2806789875030518,
|
| 74 |
+
"method": "temperature_scaling",
|
| 75 |
+
"calibration_set": "calibration",
|
| 76 |
+
"calibration_metrics": {
|
| 77 |
+
"temperature": 1.2806789875030518,
|
| 78 |
+
"optimization_method": "LBFGS_logspace",
|
| 79 |
+
"uncalibrated_nll": 0.06460460661246972,
|
| 80 |
+
"calibrated_nll": 0.06279846573841724,
|
| 81 |
+
"uncalibrated_ece": 0.012124871567496009,
|
| 82 |
+
"calibrated_ece": 0.016240862688628014,
|
| 83 |
+
"uncalibrated_brier": 0.01822748637167701,
|
| 84 |
+
"calibrated_brier": 0.018437309998858068,
|
| 85 |
+
"nll_improvement": 0.001806140874052481,
|
| 86 |
+
"ece_improvement": -0.004115991121132005,
|
| 87 |
+
"brier_improvement": -0.00020982362718105843
|
| 88 |
+
},
|
| 89 |
+
"test_metrics": {
|
| 90 |
+
"ece_before": 0.011862174308705089,
|
| 91 |
+
"ece_after": 0.015908939599173937,
|
| 92 |
+
"ece_improvement": -0.004046765290468848,
|
| 93 |
+
"brier_before": 0.01812282837704726,
|
| 94 |
+
"brier_after": 0.018294590049400802,
|
| 95 |
+
"brier_improvement": -0.0001717616723535438
|
| 96 |
+
}
|
| 97 |
+
},
|
| 98 |
"test_metrics": {
|
| 99 |
+
"threshold": 0.9284088015556335,
|
| 100 |
+
"auroc": 0.997910815020985,
|
| 101 |
+
"average_precision": 0.9976513211537581,
|
| 102 |
+
"f1": 0.9773091101352641,
|
| 103 |
+
"accuracy": 0.9797054998752118,
|
| 104 |
+
"precision": 0.9909459137479152,
|
| 105 |
+
"recall": 0.9640425346970707,
|
| 106 |
+
"specificity": 0.9926951172625913,
|
| 107 |
+
"precision_human": 0.9708363687636594,
|
| 108 |
+
"recall_human": 0.9926951172625913,
|
| 109 |
+
"precision_ai": 0.9909459137479152,
|
| 110 |
+
"recall_ai": 0.9640425346970707,
|
| 111 |
"confusion_matrix": {
|
| 112 |
+
"true_negative": 123936,
|
| 113 |
+
"false_positive": 912,
|
| 114 |
+
"false_negative": 3723,
|
| 115 |
+
"true_positive": 99816
|
| 116 |
}
|
| 117 |
},
|
| 118 |
+
"timestamp": "20251115_090814",
|
| 119 |
"seed": 42
|
| 120 |
}
|
threshold.json
CHANGED
|
@@ -1,9 +1,9 @@
|
|
| 1 |
{
|
| 2 |
-
"threshold": 0.
|
| 3 |
"method": "max_f1",
|
| 4 |
-
"
|
| 5 |
"alternative_thresholds": {
|
| 6 |
-
"max_f1": 0.
|
| 7 |
-
"precision_at_95recall":
|
| 8 |
}
|
| 9 |
}
|
|
|
|
| 1 |
{
|
| 2 |
+
"threshold": 0.9284088015556335,
|
| 3 |
"method": "max_f1",
|
| 4 |
+
"calibration_f1": 0.9773827668313225,
|
| 5 |
"alternative_thresholds": {
|
| 6 |
+
"max_f1": 0.9284088015556335,
|
| 7 |
+
"precision_at_95recall": 1.9947297005273867e-06
|
| 8 |
}
|
| 9 |
}
|
training_log_history.csv
CHANGED
|
@@ -1,144 +1,52 @@
|
|
| 1 |
loss,grad_norm,learning_rate,epoch,step,eval_loss,eval_auroc,eval_ap,eval_f1,eval_max_f1,eval_best_threshold,eval_accuracy,eval_precision_human,eval_recall_human,eval_precision_ai,eval_recall_ai,eval_runtime,eval_samples_per_second,eval_steps_per_second,train_runtime,train_samples_per_second,train_steps_per_second,total_flos,train_loss
|
| 2 |
-
0.
|
| 3 |
-
,,,0.
|
| 4 |
-
0.
|
| 5 |
-
,,,0.
|
| 6 |
-
0.
|
| 7 |
-
,,,0.
|
| 8 |
-
0.
|
| 9 |
-
,,,0.
|
| 10 |
-
0.
|
| 11 |
-
,,,0.
|
| 12 |
-
0.
|
| 13 |
-
,,,0.
|
| 14 |
-
0.
|
| 15 |
-
,,,0.
|
| 16 |
-
0.
|
| 17 |
-
,,,0.
|
| 18 |
-
0.
|
| 19 |
-
,,,0.
|
| 20 |
-
0.
|
| 21 |
-
,,,0.
|
| 22 |
-
0.
|
| 23 |
-
,,,0.
|
| 24 |
-
0.
|
| 25 |
-
,,,0.
|
| 26 |
-
0.
|
| 27 |
-
,,,
|
| 28 |
-
0.
|
| 29 |
-
,,,
|
| 30 |
-
0.
|
| 31 |
-
,,,
|
| 32 |
-
0.
|
| 33 |
-
,,,
|
| 34 |
-
0.
|
| 35 |
-
,,,
|
| 36 |
-
0.
|
| 37 |
-
,,,
|
| 38 |
-
0.
|
| 39 |
-
,,,
|
| 40 |
-
0.
|
| 41 |
-
,,,
|
| 42 |
-
0.
|
| 43 |
-
,,,
|
| 44 |
-
0.
|
| 45 |
-
,,,
|
| 46 |
-
0.
|
| 47 |
-
,,,
|
| 48 |
-
0.
|
| 49 |
-
,,,
|
| 50 |
-
0.
|
| 51 |
-
,,,
|
| 52 |
-
0.
|
| 53 |
-
,,,0.728571928964237,5200,0.10578533262014389,0.9959060523948745,0.9954666417692407,0.9599800834007594,0.9701210620572613,0.9343951344490051,0.963399682992828,0.9733206559721748,0.9593422348955121,0.9518095165761592,0.9682921411255662,245.8669,928.901,14.516,,,,,
|
| 54 |
-
0.0739,0.8520299196243286,3.2529380394336615e-05,0.7565939262320922,5400,,,,,,,,,,,,,,,,,,,
|
| 55 |
-
,,,0.7565939262320922,5400,0.10391142964363098,0.9957951879085571,0.9952825044933503,0.962313827231671,0.9695487450431454,0.9416541457176208,0.9656546373245295,0.9726057276729814,0.964332342787572,0.957428705270504,0.9672490559112992,245.8211,929.074,14.519,,,,,
|
| 56 |
-
0.0775,0.6676751971244812,3.1605427502971364e-05,0.7846159234999475,5600,,,,,,,,,,,,,,,,,,,
|
| 57 |
-
,,,0.7846159234999475,5600,0.08965875953435898,0.9963213227367876,0.9958643558038499,0.966500156857067,0.9713820883230001,0.8991213440895081,0.969608469871183,0.9726449130121062,0.9717334016836608,0.9659546963031566,0.9670462337863027,245.7698,929.268,14.522,,,,,
|
| 58 |
-
0.0727,1.1384795904159546,3.0661624608404376e-05,0.8126379207678027,5800,,,,,,,,,,,,,,,,,,,
|
| 59 |
-
,,,0.8126379207678027,5800,0.09947814047336578,0.9960502170307564,0.9955943884802091,0.9634165700569075,0.9701781488936472,0.9124361872673035,0.966701111276523,0.9725823095030796,0.966326783983596,0.9597094171091219,0.9671524739470151,245.8302,929.04,14.518,,,,,
|
| 60 |
-
0.0723,0.6806196570396423,2.969998169398137e-05,0.840659918035658,6000,,,,,,,,,,,,,,,,,,,
|
| 61 |
-
,,,0.840659918035658,6000,0.11319559812545776,0.9956365087859012,0.9951325050889013,0.958895197822541,0.9684056849185914,0.9525741338729858,0.9623751018013363,0.9730618357016131,0.9576842054674922,0.9499298657972552,0.9680313698219994,245.806,929.131,14.52,,,,,
|
| 62 |
-
0.0703,0.6376020908355713,2.872254673630171e-05,0.8686819153035132,6200,,,,,,,,,,,,,,,,,,,
|
| 63 |
-
,,,0.8686819153035132,6200,0.09632854163646698,0.9961434762558163,0.9956792827654228,0.9641803258959732,0.9704082233158977,0.9343951344490051,0.9674191938209873,0.9726870118367018,0.9675683036036108,0.961140542823141,0.9672393977148708,245.8119,929.109,14.519,,,,,
|
| 64 |
-
0.0717,0.6587705016136169,2.773140134371577e-05,0.8967039125713685,6400,,,,,,,,,,,,,,,,,,,
|
| 65 |
-
,,,0.8967039125713685,6400,0.11494402587413788,0.9958696310682444,0.9953992776112017,0.9580610203799094,0.9703862409011201,0.9585376977920532,0.9615344197980612,0.9739004034167933,0.9552332054434628,0.9472397387002983,0.9691324042148369,245.8451,928.983,14.517,,,,,
|
| 66 |
-
0.071,1.1992093324661255,2.6728656323197893e-05,0.9247259098392238,6600,,,,,,,,,,,,,,,,,,,
|
| 67 |
-
,,,0.9247259098392238,6600,0.1079399436712265,0.9959413616290814,0.9954836777985783,0.9598131601491273,0.9701123540493861,0.9504110217094421,0.963233298013013,0.9734663674212855,0.9588776662635066,0.9512949435537426,0.9684853050541342,245.9034,928.763,14.514,,,,,
|
| 68 |
-
0.0717,0.6620305180549622,2.5716447185036114e-05,0.952747907107079,6800,,,,,,,,,,,,,,,,,,,
|
| 69 |
-
,,,0.952747907107079,6800,0.09787657111883163,0.9961755673418714,0.9957160605725904,0.963615903975994,0.9705563424993778,0.8991213440895081,0.9668718748084384,0.9729949910063963,0.9662146467275946,0.9596011838059937,0.9676643583577202,245.9366,928.638,14.512,,,,,
|
| 70 |
-
0.0711,1.2997543811798096,2.4696929594912076e-05,0.9807699043749343,7000,,,,,,,,,,,,,,,,,,,
|
| 71 |
-
,,,0.9807699043749343,7000,0.1146920770406723,0.995944675317155,0.9954875225794212,0.9577217544780213,0.9696940029268117,0.954647421836853,0.9611928927342307,0.9742170251042603,0.9542640191594511,0.9461808173729452,0.9695477066612581,245.8587,928.932,14.516,,,,,
|
| 72 |
-
0.0724,0.7974486351013184,2.3672274783056795e-05,1.0086868191530352,7200,,,,,,,,,,,,,,,,,,,
|
| 73 |
-
,,,1.0086868191530352,7200,0.1015845537185669,0.9961790497526729,0.9957323299850726,0.9629252207518783,0.9700351292055995,0.9184802770614624,0.9662282276496807,0.9727491403386985,0.9652614800515832,0.9584976076555024,0.9673939288577251,245.8262,929.055,14.518,,,,,
|
| 74 |
-
0.0649,0.8229542374610901,2.2644664920259076e-05,1.0367088164208904,7400,,,,,,,,,,,,,,,,,,,
|
| 75 |
-
,,,1.0367088164208904,7400,0.08645018190145493,0.9965600272968401,0.9961185117789384,0.9678489558648573,0.971378867175356,0.8255897164344788,0.9708913856365977,0.9722255515337472,0.9745929017116951,0.9692739865355742,0.9664281092148852,245.8743,928.873,14.516,,,,,
|
| 76 |
-
0.068,0.7686835527420044,2.1616288470574255e-05,1.0647308136887457,7600,,,,,,,,,,,,,,,,,,,
|
| 77 |
-
,,,1.0647308136887457,7600,0.0841926634311676,0.9967842136955064,0.9963728317459107,0.9687837112215133,0.9733571550938489,0.8933094143867493,0.971745203296174,0.972796830822804,0.9755941272117071,0.9704690831556503,0.9671041829648731,245.8638,928.913,14.516,,,,,
|
| 78 |
-
0.0684,0.49581971764564514,2.0589335530630446e-05,1.092752810956601,7800,,,,,,,,,,,,,,,,,,,
|
| 79 |
-
,,,1.092752810956601,7800,0.097724050283432,0.9964498048620039,0.996013417423547,0.9642599451633075,0.9715029456084876,0.9334307909011841,0.9674673578940916,0.9733220461966381,0.966991597715604,0.9605082845396786,0.9680410280184278,245.8522,928.956,14.517,,,,,
|
| 80 |
-
0.0665,1.0024373531341553,1.9565993165457813e-05,1.1207748082244562,8000,,,,,,,,,,,,,,,,,,,
|
| 81 |
-
,,,1.1207748082244562,8000,0.08990959823131561,0.9965416445649657,0.996111897432367,0.9669079477905634,0.9719970667313506,0.8918110132217407,0.9699937824560174,0.9725960460123683,0.9725103526716701,0.9668565910188315,0.9669593100184471,246.7329,925.641,14.465,,,,,
|
| 82 |
-
0.0676,0.9732509851455688,1.8548440750774307e-05,1.1487968054923114,8200,,,,,,,,,,,,,,,,,,,
|
| 83 |
-
,,,1.1487968054923114,8200,0.0953540951013565,0.9965252478256964,0.9961004574885287,0.9655225599969152,0.9714322090330724,0.9399133324623108,0.9686802168259,0.9728337725782605,0.9697870193116375,0.9637063408063119,0.9673456378755831,246.7142,925.711,14.466,,,,,
|
| 84 |
-
0.0638,1.3985579013824463,1.753884533164692e-05,1.1768188027601667,8400,,,,,,,,,,,,,,,,,,,
|
| 85 |
-
,,,1.1768188027601667,8400,0.09055861830711365,0.9965989400880163,0.9961690844094493,0.966675668897934,0.9727147099462821,0.9353464841842651,0.9697442049862951,0.9733802149812554,0.9712127644236546,0.9653813922575301,0.9679734206434291,246.574,926.237,14.474,,,,,
|
| 86 |
-
0.0641,0.5897226929664612,1.6539357007413156e-05,1.204840800028022,8600,,,,,,,,,,,,,,,,,,,
|
| 87 |
-
,,,1.204840800028022,8600,0.10399724543094635,0.9962708327892441,0.9957835972645783,0.9633992144268537,0.9704696278600408,0.9532750844955444,0.96666608285972,0.9730076498499081,0.9658141565275897,0.9591430376596274,0.9676933329470054,246.6251,926.045,14.471,,,,,
|
| 88 |
-
0.0693,0.9277693629264832,1.5552104352691164e-05,1.2328627972958772,8800,,,,,,,,,,,,,,,,,,,
|
| 89 |
-
,,,1.2328627972958772,8800,0.08372964709997177,0.9967832506763049,0.9963809846481838,0.9686703149419614,0.9731677699865736,0.896251380443573,0.9716488751499655,0.9725353686967074,0.9756902448597082,0.9705722652083697,0.9667758042863076,246.6238,926.05,14.471,,,,,
|
| 90 |
-
0.0621,0.4265595078468323,1.457918988423039e-05,1.2608847945637325,9000,,,,,,,,,,,,,,,,,,,
|
| 91 |
-
,,,1.2608847945637325,9000,0.08751900494098663,0.9967499914503182,0.9963140806446376,0.9678585059253784,0.972755227251117,0.8887588381767273,0.9708694928760957,0.9730257576000128,0.9737038134676844,0.9682654757762054,0.9674518780362955,246.5692,926.255,14.475,,,,,
|
| 92 |
-
0.066,0.5132720470428467,1.3622685583256601e-05,1.2889067918315877,9200,,,,,,,,,,,,,,,,,,,
|
| 93 |
-
,,,1.2889067918315877,9200,0.09265980124473572,0.9966772555928973,0.9962576579748246,0.9657953308217793,0.9729735020893797,0.9381240010261536,0.9689122800872207,0.9734461323190002,0.9695787644076349,0.9634930552218004,0.9681086353934266,246.6667,925.889,14.469,,,,,
|
| 94 |
-
0.0638,0.6131060719490051,1.26846284828475e-05,1.316928789099443,9400,,,,,,,,,,,,,,,,,,,
|
| 95 |
-
,,,1.316928789099443,9400,0.0880228728055954,0.996786233571905,0.9963835217229717,0.9674853643116038,0.9736705691338173,0.9161096215248108,0.9705016944996628,0.973477863476741,0.9725343820836704,0.9669207022959676,0.9680506862148562,246.5183,926.446,14.478,,,,,
|
| 96 |
-
0.0631,0.5040121674537659,1.1767016329735986e-05,1.3449507863672983,9600,,,,,,,,,,,,,,,,,,,
|
| 97 |
-
,,,1.3449507863672983,9600,0.088343046605587,0.9967945716541701,0.9963785207323399,0.9681004779370901,0.9732178678012516,0.9207897186279297,0.9710971775853161,0.9730067043218075,0.9741523624916898,0.9687886877127824,0.9674132452505819,246.6015,926.134,14.473,,,,,
|
| 98 |
-
0.0626,0.5754002332687378,1.0871803329780279e-05,1.3729727836351535,9800,,,,,,,,,,,,,,,,,,,
|
| 99 |
-
,,,1.3729727836351535,9800,0.08908010274171829,0.9968349029617605,0.9964338452298704,0.9673004536241675,0.9733966350967752,0.9161096215248108,0.9703309309677476,0.9734012268954734,0.9722940879636676,0.9666380532595171,0.9679637624470007,246.5625,926.28,14.475,,,,,
|
| 100 |
-
0.0635,0.37719520926475525,1.0000895986161401e-05,1.4009947809030088,10000,,,,,,,,,,,,,,,,,,,
|
| 101 |
-
,,,1.4009947809030088,10000,0.10576409846544266,0.9964103740665244,0.9959816504324808,0.9620018891174369,0.9717239996277691,0.9585376977920532,0.9652999746043979,0.9738979588528072,0.9623138721795478,0.9552007160268129,0.9689006075005554,246.5201,926.44,14.478,,,,,
|
| 102 |
-
0.0647,0.4855269193649292,9.156149039171426e-06,1.429016778170864,10200,,,,,,,,,,,,,,,,,,,
|
| 103 |
-
,,,1.429016778170864,10200,0.08731859922409058,0.9969044529302581,0.9965060417039346,0.967862178187226,0.9734939759031145,0.9019206762313843,0.9708563572197946,0.973457141254797,0.9732232252276787,0.9677220017572826,0.9680023952327143,246.5275,926.412,14.477,,,,,
|
| 104 |
-
0.0658,0.9444574117660522,8.339361516239216e-06,1.4570387754387193,10400,,,,,,,,,,,,,,,,,,,
|
| 105 |
-
,,,1.4570387754387193,10400,0.0880136489868164,0.9968816941218545,0.9965015373119736,0.9673096035276295,0.9733884580565982,0.9161096215248108,0.9703309309677476,0.9736215071361525,0.9720618036476647,0.9663771580601317,0.9682438501434242,246.6147,926.084,14.472,,,,,
|
| 106 |
-
0.0611,0.45777803659439087,7.5522729006059505e-06,1.4850607727065746,10600,,,,,,,,,,,,,,,,,,,
|
| 107 |
-
,,,1.4850607727065746,10600,0.09273924678564072,0.9967055123563264,0.9962876686774951,0.9661673703082528,0.9724540697185579,0.9399133324623108,0.969262564255252,0.9734555945770013,0.9702275585316428,0.9642434561773109,0.9680989771969982,246.5306,926.4,14.477,,,,,
|
| 108 |
-
0.0627,0.8916956186294556,6.796559426809692e-06,1.5130827699744298,10800,,,,,,,,,,,,,,,,,,,
|
| 109 |
-
,,,1.5130827699744298,10800,0.08739963173866272,0.9968748801914394,0.9964844262023231,0.9675924808764267,0.9735564214519877,0.9241418242454529,0.9705980226458715,0.9735888389993586,0.9725984605156712,0.9669997877799857,0.9681859009648538,246.7364,925.627,14.465,,,,,
|
| 110 |
-
0.0623,0.7013267278671265,6.0738305108685545e-06,1.541104767242285,11000,,,,,,,,,,,,,,,,,,,
|
| 111 |
-
,,,1.541104767242285,11000,0.0974632278084755,0.9966048359068608,0.9961832096074739,0.964546783625731,0.9723199624672457,0.9362850189208984,0.9677213139159143,0.9737315407011913,0.9670396565396044,0.9605831585198809,0.9685432542327046,246.5852,926.195,14.474,,,,,
|
| 112 |
-
0.063,0.8154935240745544,5.385625322764794e-06,1.5691267645101403,11200,,,,,,,,,,,,,,,,,,,
|
| 113 |
-
,,,1.5691267645101403,11200,0.09752331674098969,0.9966611901489429,0.9962433071485012,0.964554077507007,0.9729103642409853,0.9433475732803345,0.9677169353638139,0.9739989347794509,0.966751303595601,0.9602653444116859,0.9688812911076985,246.5136,926.464,14.478,,,,,
|
| 114 |
-
0.0631,0.685742974281311,4.73340950852946e-06,1.5971487617779956,11400,,,,,,,,,,,,,,,,,,,
|
| 115 |
-
,,,1.5971487617779956,11400,0.09248074889183044,0.9967045142155668,0.996290326082087,0.9660411899313501,0.9726085083041068,0.9399133324623108,0.9691355862443407,0.9736550060313631,0.9697790095076374,0.9637337075627668,0.968359748500565,246.5731,926.241,14.474,,,,,
|
| 116 |
-
0.0627,1.1395540237426758,4.118572068908318e-06,1.6251707590458508,11600,,,,,,,,,,,,,,,,,,,
|
| 117 |
-
,,,1.6251707590458508,11600,0.08698944747447968,0.9968948298167076,0.996494789063069,0.9677260527890946,0.973689873045802,0.9161096215248108,0.9707250006567828,0.9735494265034746,0.9728788036556746,0.9673244873341376,0.9681279517862834,246.6038,926.125,14.473,,,,,
|
| 118 |
-
0.0621,1.5019831657409668,3.5424224012565447e-06,1.653192756313706,11800,,,,,,,,,,,,,,,,,,,
|
| 119 |
-
,,,1.653192756313706,11800,0.10022038221359253,0.9966304956476687,0.9962084684159175,0.9636517328825022,0.9727368874608185,0.9626730680465698,0.9668631177042376,0.973980940371978,0.965165362403582,0.9584499708605222,0.9689102656969838,246.5696,926.254,14.475,,,,,
|
| 120 |
-
0.0621,0.6207771897315979,3.006187510961881e-06,1.6812147535815614,12000,,,,,,,,,,,,,,,,,,,
|
| 121 |
-
,,,1.6812147535815614,12000,0.08885731548070908,0.9968981157711279,0.9965016737672114,0.9672419696290188,0.9739953876205015,0.9433475732803345,0.970256495582041,0.9738992902790713,0.9716292742316596,0.9658865453144563,0.968601203411275,246.5364,926.378,14.477,,,,,
|
| 122 |
-
0.0625,1.3990130424499512,2.511009398335074e-06,1.7092367508494168,12200,,,,,,,,,,,,,,,,,,,
|
| 123 |
-
,,,1.7092367508494168,12200,0.09371371567249298,0.9967498807864655,0.9963397310167501,0.965798508289155,0.9732076432118419,0.9481545686721802,0.9688991444309196,0.9738417951772319,0.9691382251876297,0.9630024389775499,0.9686108616077034,246.5834,926.202,14.474,,,,,
|
| 124 |
-
0.0623,0.5540758967399597,2.057942626532536e-06,1.737258748117272,12400,,,,,,,,,,,,,,,,,,,
|
| 125 |
-
,,,1.737258748117272,12400,0.09393420815467834,0.9967897724943766,0.9963785511182874,0.9659367279417076,0.9731286363853603,0.939024806022644,0.9690305009939313,0.9738176083423182,0.9694105585236329,0.963315530623223,0.9685722288219898,246.6306,926.025,14.471,,,,,
|
| 126 |
-
0.0643,0.9720720052719116,1.6479520756908518e-06,1.7652807453851274,12600,,,,,,,,,,,,,,,,,,,
|
| 127 |
-
,,,1.7652807453851274,12600,0.09308448433876038,0.9967791736358621,0.9963694636590227,0.9660367906669813,0.9731835264636404,0.9441768527030945,0.9691268291401399,0.9737688830257887,0.9696428428396358,0.9635815044009686,0.968504621446991,246.6887,925.807,14.468,,,,,
|
| 128 |
-
0.0624,1.0696525573730469,1.2819108880559477e-06,1.7933027426529826,12800,,,,,,,,,,,,,,,,,,,
|
| 129 |
-
,,,1.7933027426529826,12800,0.09151949733495712,0.9968253090517087,0.9964204520696328,0.9665868246157702,0.9733586501189067,0.9399133324623108,0.9696434982879861,0.9738089880416613,0.9705719801036469,0.964657444638975,0.9685239378398478,246.5498,926.328,14.476,,,,,
|
| 130 |
-
0.0625,1.4033018350601196,9.605986084832452e-07,1.8213247399208379,13000,,,,,,,,,,,,,,,,,,,
|
| 131 |
-
,,,1.8213247399208379,13000,0.09507396817207336,0.9967674833401138,0.9963561447668665,0.9654810607883717,0.9731590960292014,0.9591543078422546,0.9685970243359926,0.9739035882566147,0.9685054506716221,0.9622760982817012,0.9687074435719873,246.7376,925.623,14.465,,,,,
|
| 132 |
-
0.063,0.45164167881011963,6.846995242687752e-07,1.8493467371886931,13200,,,,,,,,,,,,,,,,,,,
|
| 133 |
-
,,,1.8493467371886931,13200,0.09227791428565979,0.9968329623022687,0.9964266292094927,0.9663162762913218,0.9733341164718503,0.9465966820716858,0.969385163714063,0.9738956307751695,0.9699952742156399,0.9639942713790021,0.968649494393417,246.5226,926.43,14.477,,,,,
|
| 134 |
-
0.0618,0.3481653332710266,4.548012078469415e-07,1.8773687344565484,13400,,,,,,,,,,,,,,,,,,,
|
| 135 |
-
,,,1.8773687344565484,13400,0.09297340363264084,0.9968072979972056,0.9963973889545064,0.9661836679640834,0.9732662175973241,0.9425067901611328,0.969262564255252,0.9738363909822815,0.9698270683316379,0.9637972956089685,0.9685818870184182,246.5722,926.244,14.474,,,,,
|
| 136 |
-
0.0631,0.3175712525844574,2.7139326545848965e-07,1.9053907317244037,13600,,,,,,,,,,,,,,,,,,,
|
| 137 |
-
,,,1.9053907317244037,13600,0.09512791037559509,0.9967728122242903,0.9963611581808621,0.9655092748572913,0.9731316412589667,0.9489172697067261,0.968623295648595,0.9739124831868813,0.9685454996916225,0.9623226227369108,0.9687171017684157,246.586,926.192,14.474,,,,,
|
| 138 |
-
0.0606,0.5379019975662231,1.348662944535554e-07,1.933412728992259,13800,,,,,,,,,,,,,,,,,,,
|
| 139 |
-
,,,1.933412728992259,13800,0.09340985864400864,0.9968055931317871,0.9963962951212402,0.9660171262895287,0.9732650683786642,0.9504110217094421,0.969104936379638,0.9738516867673443,0.9695146859756342,0.9634372448244393,0.9686108616077034,246.589,926.181,14.473,,,,,
|
| 140 |
-
0.061,1.021728754043579,4.551105145043261e-08,1.9614347262601142,14000,,,,,,,,,,,,,,,,,,,
|
| 141 |
-
,,,1.9614347262601142,14000,0.09326184540987015,0.9968091848139563,0.9964000739066909,0.9660543477213837,0.9732996051448428,0.9465966820716858,0.9691399647964412,0.9738533696972622,0.9695787644076349,0.9635112934372208,0.9686108616077034,246.5091,926.481,14.478,,,,,
|
| 142 |
-
0.0623,1.5442456007003784,3.5178331215701367e-09,1.9894567235279694,14200,,,,,,,,,,,,,,,,,,,
|
| 143 |
-
,,,1.9894567235279694,14200,0.09308414906263351,0.9968106062700608,0.9964016493909656,0.9661896251908728,0.9732898767747101,0.9473810195922852,0.9692669428073525,0.9738670913565948,0.9698030389196376,0.9637709014030367,0.9686205198041318,246.5533,926.315,14.476,,,,,
|
| 144 |
-
,,,2.0,14276,,,,,,,,,,,,,,,27534.7527,132.711,0.518,3.3997885998922465e+18,0.09163027936652716
|
|
|
|
| 1 |
loss,grad_norm,learning_rate,epoch,step,eval_loss,eval_auroc,eval_ap,eval_f1,eval_max_f1,eval_best_threshold,eval_accuracy,eval_precision_human,eval_recall_human,eval_precision_ai,eval_recall_ai,eval_runtime,eval_samples_per_second,eval_steps_per_second,train_runtime,train_samples_per_second,train_steps_per_second,total_flos,train_loss
|
| 2 |
+
0.276,3.159646987915039,0.00013013829896707,0.07783918424534911,500,,,,,,,,,,,,,,,,,,,
|
| 3 |
+
,,,0.07783918424534911,500,0.12149354815483093,0.9924457584277557,0.9916298325594288,0.9536510818288485,0.9557720332927327,0.6680145263671875,0.9579702783883426,0.9616493887295509,0.9614568231515375,0.9535359777528871,0.9537662136972542,251.8553,906.814,14.171,,,,,
|
| 4 |
+
0.1082,0.4662734270095825,0.00015502115157402368,0.15567836849069822,1000,,,,,,,,,,,,,,,,,,,
|
| 5 |
+
,,,0.15567836849069822,1000,0.11359784007072449,0.9941361028997843,0.9936079408729138,0.9563341131667457,0.964144751321268,0.8459424376487732,0.9600369549797273,0.9707662766667209,0.9556737446634681,0.9475350777398559,0.9652981002327625,252.3501,905.036,14.143,,,,,
|
| 6 |
+
0.0901,0.56740403175354,0.00015336171321936976,0.23351755273604732,1500,,,,,,,,,,,,,,,,,,,
|
| 7 |
+
,,,0.23351755273604732,1500,0.09792134165763855,0.995637180466362,0.9951736481792155,0.9632415978730987,0.9687276503605232,0.8856314420700073,0.9665828903698125,0.971504195528524,0.9672399016396068,0.9607059479089608,0.9657906682506109,252.1455,905.771,14.155,,,,,
|
| 8 |
+
0.0828,0.6910482048988342,0.0001504606098364759,0.31135673698139643,2000,,,,,,,,,,,,,,,,,,,
|
| 9 |
+
,,,0.31135673698139643,2000,0.07930342108011246,0.9965300160027056,0.996129497193991,0.9701504169589276,0.9709891509313444,0.6976089477539062,0.9731682327288013,0.968767028089932,0.9825946959077911,0.9786452037697653,0.9618018331256821,252.2542,905.38,14.148,,,,,
|
| 10 |
+
0.0786,0.7298774719238281,0.00014636546193317465,0.38919592122674557,2500,,,,,,,,,,,,,,,,,,,
|
| 11 |
+
,,,0.38919592122674557,2500,0.10587891191244125,0.9961595063461319,0.99574080839367,0.9633412681237827,0.9702281512250107,0.9585376977920532,0.9666222973387161,0.9727536699467656,0.965998382019592,0.9593414171599334,0.9673746124648683,252.014,906.243,14.162,,,,,
|
| 12 |
+
0.075,0.5279271006584167,0.00014114348980363213,0.46703510547209465,3000,,,,,,,,,,,,,,,,,,,
|
| 13 |
+
,,,0.46703510547209465,3000,0.08700015395879745,0.9963549115948438,0.9959508112193701,0.9664618832348054,0.9711828125530761,0.8791467547416687,0.9695953342148819,0.9720765232989261,0.9723101075716677,0.9666019379957298,0.9663218690541728,252.0033,906.282,14.163,,,,,
|
| 14 |
+
0.0718,0.8794483542442322,0.00013488041013280436,0.5448742897174438,3500,,,,,,,,,,,,,,,,,,,
|
| 15 |
+
,,,0.5448742897174438,3500,0.10059615969657898,0.9964150386194646,0.9960424913733611,0.9614705825931823,0.9711960816065123,0.954647421836853,0.9648227124254551,0.9732758550835028,0.9620655682555448,0.9548853558398507,0.9681472681791402,252.084,905.992,14.158,,,,,
|
| 16 |
+
0.0695,0.40731295943260193,0.00012767902898967842,0.6227134739627929,4000,,,,,,,,,,,,,,,,,,,
|
| 17 |
+
,,,0.6227134739627929,4000,0.10083704441785812,0.9961781307127794,0.995731264522598,0.9619906765054659,0.9692610702277147,0.9149009585380554,0.9653350030212009,0.9728877169710597,0.9634352447395612,0.9564208797922713,0.9676257255720067,252.2327,905.457,14.15,,,,,
|
| 18 |
+
0.0669,0.264863520860672,0.00011965755430477945,0.7005526582081419,4500,,,,,,,,,,,,,,,,,,,
|
| 19 |
+
,,,0.7005526582081419,4500,0.094916433095932,0.9967265965858536,0.9963598631028794,0.9650556403576777,0.972462604745369,0.9161096215248108,0.9682248474074593,0.9731915784051766,0.9685535094956227,0.962298576833695,0.9678285476970031,251.9541,906.459,14.165,,,,,
|
| 20 |
+
0.0672,0.7680786848068237,0.00011094765553198254,0.7783918424534911,5000,,,,,,,,,,,,,,,,,,,
|
| 21 |
+
,,,0.7783918424534911,5000,0.09323982149362564,0.9967850366539563,0.9963962011157831,0.9673139455667273,0.9710460087467818,0.8643104434013367,0.9703790950408518,0.972494317999936,0.9733433722876801,0.9678236488446292,0.9668047788755928,251.9538,906.46,14.165,,,,,
|
| 22 |
+
0.0647,0.4233705997467041,0.0001016923023445425,0.8562310266988402,5500,,,,,,,,,,,,,,,,,,,
|
| 23 |
+
,,,0.8562310266988402,5500,0.09126096963882446,0.9967433808226673,0.9963771581479701,0.9657864214107987,0.9726255234214075,0.9111796617507935,0.9688728731183173,0.9742069565497575,0.9687056957716245,0.9625206246882314,0.9690744550362665,252.0396,906.151,14.16,,,,,
|
| 24 |
+
0.0627,0.25091952085494995,9.204341784232336e-05,0.9340702109441893,6000,,,,,,,,,,,,,,,,,,,
|
| 25 |
+
,,,0.9340702109441893,6000,0.06995870172977448,0.9975342235086746,0.9972417957598282,0.973204830514238,0.9758824625579795,0.8918110132217407,0.9758479066142408,0.9732840473716358,0.9827949410077935,0.9790068315757582,0.9674711944291523,252.1038,905.921,14.157,,,,,
|
| 26 |
+
0.0606,0.3678501546382904,8.215938479193825e-05,1.011831556005293,6500,,,,,,,,,,,,,,,,,,,
|
| 27 |
+
,,,1.011831556005293,6500,0.07501858472824097,0.9975025002706578,0.997203865754845,0.971139887346844,0.9754012996088349,0.8902942538261414,0.9738863152732654,0.9744957013881682,0.9778208527237339,0.9731459660760525,0.9691420624112653,252.3023,905.208,14.146,,,,,
|
| 28 |
+
0.0538,0.6192397475242615,7.220244583391773e-05,1.0896707402506423,7000,,,,,,,,,,,,,,,,,,,
|
| 29 |
+
,,,1.0896707402506423,7000,0.09348879754543304,0.997100493345388,0.9967079581489795,0.9689502265506753,0.9725104874267961,0.9184802770614624,0.9718853169633865,0.97321942331053,0.9754099017197049,0.9702686474655716,0.967635383768435,252.0093,906.26,14.162,,,,,
|
| 30 |
+
0.0536,0.6146565675735474,6.233604033151736e-05,1.1675099244959912,7500,,,,,,,,,,,,,,,,,,,
|
| 31 |
+
,,,1.1675099244959912,7500,0.08755695074796677,0.9973925286764659,0.9970545509180517,0.9708156623418074,0.974796319089823,0.9324532747268677,0.9735754380741376,0.9747314921365554,0.9769878331077239,0.9721743341404359,0.9694607828934025,251.9972,906.304,14.163,,,,,
|
| 32 |
+
0.0563,0.24420885741710663,5.272212157577683e-05,1.2453491087413404,8000,,,,,,,,,,,,,,,,,,,
|
| 33 |
+
,,,1.2453491087413404,8000,0.07455883920192719,0.9975485047929438,0.9972585615075422,0.9726402255038238,0.9758848582753115,0.9136765599250793,0.9752655591848888,0.9750665582603982,0.9798072841157577,0.97550810243656,0.969789161571968,252.079,906.01,14.158,,,,,
|
| 34 |
+
0.0521,0.4137882590293884,4.351849838388919e-05,1.3231882929866896,8500,,,,,,,,,,,,,,,,,,,
|
| 35 |
+
,,,1.3231882929866896,8500,0.07783409208059311,0.9975517384131479,0.9972440455972116,0.9723962677736611,0.9749894769810089,0.9111796617507935,0.975077281444572,0.9738866219645087,0.9807043821637684,0.9765353333658013,0.9682921411255662,252.489,904.538,14.135,,,,,
|
| 36 |
+
0.0526,0.3627403974533081,3.4876244727530656e-05,1.4010274772320386,9000,,,,,,,,,,,,,,,,,,,
|
| 37 |
+
,,,1.4010274772320386,9000,0.0745643824338913,0.9975749102093014,0.9972690713005937,0.9723900247831475,0.9756176280729699,0.9046505093574524,0.9750247388193672,0.975305785387727,0.9791024213637493,0.9746829301427421,0.9701078820541053,252.119,905.866,14.156,,,,,
|
| 38 |
+
0.052,0.7762022614479065,2.693721991111627e-05,1.4788666614773878,9500,,,,,,,,,,,,,,,,,,,
|
| 39 |
+
,,,1.4788666614773878,9500,0.07452459633350372,0.9977328074904859,0.9974410894856215,0.9730508384452147,0.975759591492858,0.9124361872673035,0.9755895720403177,0.976881986981624,0.978501686063742,0.9740254712964038,0.9720781541254986,252.2439,905.417,14.149,,,,,
|
| 40 |
+
0.0507,0.3820905387401581,1.9831740005311437e-05,1.5567058457227367,10000,,,,,,,,,,,,,,,,,,,
|
| 41 |
+
,,,1.5567058457227367,10000,0.08428945392370224,0.9975299683849125,0.9972155352602694,0.9700319035460719,0.9748023112122028,0.9334307909011841,0.9728135700086695,0.9755487501803781,0.9746970291636964,0.9695218431614696,0.9705425008933831,252.2555,905.376,14.148,,,,,
|
| 42 |
+
0.0515,0.36162489652633667,1.3676438758331925e-05,1.634545029968086,10500,,,,,,,,,,,,,,,,,,,
|
| 43 |
+
,,,1.634545029968086,10500,0.07442453503608704,0.9977306493713021,0.997440190110606,0.9729125537103704,0.9761234031726127,0.9136765599250793,0.9754888653420087,0.9760021075993326,0.9792385880317509,0.9748654545454546,0.9709674615362327,252.2157,905.519,14.151,,,,,
|
| 44 |
+
0.0512,0.5945746302604675,8.572353097359252e-06,1.7123842142134351,11000,,,,,,,,,,,,,,,,,,,
|
| 45 |
+
,,,1.7123842142134351,11000,0.07524814456701279,0.9977685405428793,0.9974825248399177,0.9727850366057699,0.9761874492694766,0.9207897186279297,0.975357508778997,0.9763919857424856,0.978581784103743,0.9741039521978714,0.9714696877505095,252.2195,905.505,14.15,,,,,
|
| 46 |
+
0.0509,0.9120739698410034,4.603264645836933e-06,1.7902233984587843,11500,,,,,,,,,,,,,,,,,,,
|
| 47 |
+
,,,1.7902233984587843,11500,0.06851697713136673,0.9979216958335029,0.997654853113855,0.9747033543129303,0.9769461620177022,0.9124361872673035,0.9771395794838563,0.9764685264549843,0.9818417743317821,0.9779586201532299,0.9714696877505095,252.8119,903.383,14.117,,,,,
|
| 48 |
+
0.048,0.6627203822135925,1.834324480010042e-06,1.8680625827041333,12000,,,,,,,,,,,,,,,,,,,
|
| 49 |
+
,,,1.8680625827041333,12000,0.07031949609518051,0.9978716433195517,0.9975978286587303,0.9741437319971922,0.9765590576618682,0.9241418242454529,0.9766141532318093,0.9766512444160816,0.980664333143768,0.9765690214120707,0.9717304590540763,252.5244,904.411,14.133,,,,,
|
| 50 |
+
0.0488,0.7994762659072876,3.1098369880601253e-07,1.9459017669494822,12500,,,,,,,,,,,,,,,,,,,
|
| 51 |
+
,,,1.9459017669494822,12500,0.07445533573627472,0.9977876009119543,0.997504157410398,0.972836356080046,0.9761660160257996,0.929440438747406,0.9753881586436997,0.9769268924908395,0.9780771664517369,0.9735279325286289,0.9721457615004974,252.7495,903.606,14.121,,,,,
|
| 52 |
+
,,,2.0,12848,,,,,,,,,,,,,,,15236.3158,215.85,0.843,3.0700924448014336e+18,0.07117979241486355
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|