yonad2008/predict_qwen2.5_7b-instruct

Jailbreak Prediction Model: qwen2.5:7b-instruct

Fine-tuned DeBERTa-v3-base for detecting unsafe/jailbreak prompts in multi-turn conversations.

Evaluation Results (best fold: 3)

Metric	Value
F1	0.7731
PR-AUC	0.8692
ROC-AUC	0.9591
Precision	0.8364
Recall	0.7188
Best Threshold	0.10

Training Details

• Base model: microsoft/deberta-v3-base
• Target model: qwen2.5:7b-instruct
• Datasets: HarmBench
• K-Folds: 5
• Epochs: 5
• Learning Rate: 2e-05
• Max Length: 512
• Input format: turns only

Dataset Size (before turn expansion)

Original rows (after cleaning and balancing): 1536 (unsafe: 308, safe: 1228)