precisionalgorithms's picture
Raw eval output for both runs
3f64c14 verified
|
Raw
History Blame Contribute Delete
668 Bytes

Eval results

Raw output from the harness so the numbers in the model card can be checked rather than taken on trust.

  • v3_base.json / v3_tuned.json: the released model against its base, 120 generated cases, both bf16, identical serving settings. This is the pair the model card reports.
  • v4_base.json / v4_tuned.json: the later run that tried to fix the refusal rate by rebalancing the training data and made it worse. Kept because the negative result is the useful part.

Reproduce with:

python -m pat.cli run --url <endpoint> --model <name> --generated 120 --gen-seed 1 --out out.json
python -m pat.cli compare v3_base.json v3_tuned.json