AgentreBench commited on
Commit
7ca7fe9
·
verified ·
1 Parent(s): ae8c826

Fix eval chart labels

Browse files
Files changed (2) hide show
  1. README.md +4 -4
  2. assets/xref-9b-eval-chart.svg +4 -4
README.md CHANGED
@@ -70,13 +70,13 @@ Held-out mixed PE/ELF evaluation, 40 binaries total, balanced by format and beni
70
 
71
  | Checkpoint / workflow | Strict accuracy | Binary accuracy | Notes |
72
  |---|---:|---:|---|
73
- | SFT baseline | 55.0% | 62.5% | Initial supervised model. |
74
- | SFT + IPO | 57.5% | 57.5% | Preference-tuned checkpoint; improved answer rate but still weak on ELF malware. |
75
- | SFT + IPO + release static workflow | 67.5% | 67.5% | Same IPO family with deeper static-analysis workflow and final-answer forcing. |
76
 
77
  Observed gains:
78
 
79
- - strict accuracy improved from 55.0% to 67.5% in the release workflow (+12.5 percentage points);
80
  - labeled ELF malware performance improved substantially under the deeper static workflow;
81
  - benign ELF calibration regressed in that run, so unknown/benign decisions still need analyst review.
82
 
 
70
 
71
  | Checkpoint / workflow | Strict accuracy | Binary accuracy | Notes |
72
  |---|---:|---:|---|
73
+ | Base model | 55.0% | 62.5% | Untuned base under the same holdout triage harness. |
74
+ | SFT | 57.5% | 57.5% | Supervised checkpoint on PE/ELF reverse-engineering tasks. |
75
+ | SFT + IPO | 67.5% | 67.5% | Preference-optimized release workflow with deeper static-analysis settings and final-answer forcing. |
76
 
77
  Observed gains:
78
 
79
+ - strict accuracy improved from 55.0% on the base model to 67.5% with SFT + IPO in the release workflow (+12.5 percentage points);
80
  - labeled ELF malware performance improved substantially under the deeper static workflow;
81
  - benign ELF calibration regressed in that run, so unknown/benign decisions still need analyst review.
82
 
assets/xref-9b-eval-chart.svg CHANGED