experiments_deep_baseline/reports/experiment_report.md

Layer 2 Experiment Report

Generated at: 2026-03-31T17:18:41.607272Z

Dataset

• Dataset id: xTRam1/safe-guard-prompt-injection
• Total rows after dedup: 10135
• Label counts: {'0': 7066, '1': 3069}
• Train/Val/Test sizes: {'train': 8108, 'val': 1013, 'test': 1014}

Configs Executed

config	owner	model	max_len	lr	batch	grad_acc	epochs	loss	lora
config1_modernbert_large	Deep	answerdotai/ModernBERT-large	384	1.5e-05	16	4	4	ce	no

Per-Run Results

run_id	owner	model	seed	f1_inj	pr_auc	recall_inj	precision_inj	fpr_safe	best_thr	lat_ms	train_min
config1_modernbert_large_seed42	Deep	answerdotai/ModernBERT-large	42	0.9984	1.0000	1.0000	0.9968	0.0014	0.47	649.09	2220.27

Aggregated Ranking

config	owner	model	runs	f1_inj mean±std	pr_auc mean±std	recall_inj mean±std	precision_inj mean±std	fpr_safe mean±std	lat_ms
config1_modernbert_large	Deep	answerdotai/ModernBERT-large	1	0.9984 ± 0.0000	1.0000 ± 0.0000	1.0000 ± 0.0000	0.9968 ± 0.0000	0.0014 ± 0.0000	649.09

Winner

• Config: config1_modernbert_large
• Owner: Deep
• Model: answerdotai/ModernBERT-large
• F1 injection (mean): 0.9984
• PR-AUC (mean): 1.0000
• FPR safe (mean): 0.0014

Selection Rule Used

1. Maximize mean injection-class F1
2. Break ties with mean PR-AUC
3. Then minimize mean FPR on safe prompts