metadata
language: en
tags:
- jailbreak-detection
- deberta-v3
- text-classification
- fine-tuned
model-index:
- name: predict_llama3.2_3b_harmful_behaviors_1
results:
- task:
type: text-classification
name: Jailbreak Detection
metrics:
- name: F1
type: f1
value: 0.4706
- name: PR-AUC
type: pr_auc
value: 0.3991
- name: ROC-AUC
type: roc_auc
value: 0.7376
- name: Precision
type: precision
value: 0.3077
- name: Recall
type: recall
value: 1
Jailbreak Prediction Model: llama3.2:3b (fine-tuned on harmful_behaviors_1)
Incrementally fine-tuned from yonad2008/predict_llama3.2_3b
on data from harmful_behaviors_1.csv.
Evaluation Results
| Metric | Value |
|---|---|
| F1 | 0.4706 |
| PR-AUC | 0.3991 |
| ROC-AUC | 0.7376 |
| Precision | 0.3077 |
| Recall | 1.0000 |
| Best Threshold | 0.15 |
Fine-Tuning Details
- Base model:
yonad2008/predict_llama3.2_3b - Target model:
llama3.2:3b - Dataset:
harmful_behaviors_1.csv - Frozen layers: 9/12 transformer layers
- Epochs: 3
- Learning Rate: 5e-06
- Max Length: 512
- Input format: turns only
Dataset Size (before turn expansion)
Original rows (after cleaning and balancing): 284 (unsafe: 71, safe: 213)