metadata
language: en
tags:
- jailbreak-detection
- deberta-v3
- text-classification
- fine-tuned
model-index:
- name: predict_llama2_7b_harmful_behaviors_1
results:
- task:
type: text-classification
name: Jailbreak Detection
metrics:
- name: F1
type: f1
value: 0.2913
- name: PR-AUC
type: pr_auc
value: 0.1851
- name: ROC-AUC
type: roc_auc
value: 0.5363
- name: Precision
type: precision
value: 0.1974
- name: Recall
type: recall
value: 0.5556
Jailbreak Prediction Model: llama2:7b (fine-tuned on harmful_behaviors_1)
Incrementally fine-tuned from yonad2008/predict_llama2_7b
on data from harmful_behaviors_1.csv.
Evaluation Results
| Metric | Value |
|---|---|
| F1 | 0.2913 |
| PR-AUC | 0.1851 |
| ROC-AUC | 0.5363 |
| Precision | 0.1974 |
| Recall | 0.5556 |
| Best Threshold | 0.10 |
Fine-Tuning Details
- Base model:
yonad2008/predict_llama2_7b - Target model:
llama2:7b - Dataset:
harmful_behaviors_1.csv - Frozen layers: 11/12 transformer layers
- Epochs: 3
- Learning Rate: 5e-06
- Max Length: 512
- Input format: turns only
Dataset Size (before turn expansion)
Original rows (after cleaning and balancing): 236 (unsafe: 59, safe: 177)