Add safeagents judge bench v2 results
#7
by ybelkada HF Staff - opened
Hi! This adds vllm-sr/d3-mini's results on the safeagents judge bench v2, a benchmark of real agent tool calls that safeagents' scanner flags as suspicious, each labelled block or allow and with a kind of misuse (reward hacking, harness introspection, credential misuse, ...).
On the 661 test calls: balanced accuracy 0.856, AUROC 0.986, false-block rate 0.288, category macro F1 0.512. Run zero-shot, one System One request per call, on NVIDIA A100 80GB (HF Jobs). Per-call verdicts are in the dataset's results/ folder.
The file is named after the benchmark, so it doesn't touch any existing .eval_results file. v2 keeps every earlier call and adds new ones (see the dataset card); these numbers replace the earlier ones in this file.