File size: 668 Bytes
3f64c14
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Eval results

Raw output from the harness so the numbers in the model card can be checked rather than
taken on trust.

- `v3_base.json` / `v3_tuned.json`: the released model against its base, 120 generated
  cases, both bf16, identical serving settings. This is the pair the model card reports.
- `v4_base.json` / `v4_tuned.json`: the later run that tried to fix the refusal rate by
  rebalancing the training data and made it worse. Kept because the negative result is the
  useful part.

Reproduce with:

    python -m pat.cli run --url <endpoint> --model <name> --generated 120 --gen-seed 1 --out out.json
    python -m pat.cli compare v3_base.json v3_tuned.json