codybum commited on
Commit
bb42310
·
verified ·
1 Parent(s): 235623d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -111,7 +111,8 @@ the *official* IFEval checker. GRPO lifted the **official IFEval strict-avg from
111
 
112
  ## Evaluation & benchmarks
113
 
114
- **Benchmarks are official-suite only:** IFEval via the EleutherAI lm-evaluation-harness (`--apply_chat_template`),
 
115
  BFCL via the official `bfcl_eval`. Peer strict-avg values are the Open LLM Leaderboard's "IFEval" (strict). Our 61.6
116
  is measured on our own copy of that harness; it calibrates cleanly (we measure Qwen2.5-3B at 64.0 vs the leaderboard's
117
  64.7, a 0.7-pt gap).
 
111
 
112
  ## Evaluation & benchmarks
113
 
114
+ **Benchmarks are official-suite only:** IFEval via the EleutherAI lm-evaluation-harness 0.4.12 (`task ifeval, apply_chat_template=True
115
+ - greedy, task-default max_gen_toks=1280`),
116
  BFCL via the official `bfcl_eval`. Peer strict-avg values are the Open LLM Leaderboard's "IFEval" (strict). Our 61.6
117
  is measured on our own copy of that harness; it calibrates cleanly (we measure Qwen2.5-3B at 64.0 vs the leaderboard's
118
  64.7, a 0.7-pt gap).