legolasyiu commited on
Commit
748d84b
·
verified ·
1 Parent(s): a9306ca

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -166,14 +166,14 @@ For further details, questions, or feedback, please email episteme.ai@proton.me
166
 
167
  ## Benchmark
168
 
169
- hf (pretrained=EpistemeAI/ReasoningCore-3B-R01), gen_kwargs: (None), limit: None, num_fewshot: 1, batch_size: 8
170
  | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
171
  |-------------|------:|------|-----:|--------|---|-----:|---|-----:|
172
- |arc_challenge| 1|none | 1|acc |↑ |0.4113|± |0.0144|
173
- | | |none | 1|acc_norm|↑ |0.4548|± |0.0146|
174
- |hellaswag | 1|none | 1|acc |↑ |0.5103|± |0.0050|
175
- | | |none | 1|acc_norm|↑ |0.6965|± |0.0046|
176
- |winogrande | 1|none | 1|acc |↑ |0.6654|± |0.0133|
177
 
178
 
179
  # Uploaded model
 
166
 
167
  ## Benchmark
168
 
169
+ hf (pretrained=EpistemeAI/ReasoningCore-3B-R01), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 8
170
  | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
171
  |-------------|------:|------|-----:|--------|---|-----:|---|-----:|
172
+ |arc_challenge| 1|none | 5|acc |↑ |0.4352|± |0.0145|
173
+ | | |none | 5|acc_norm|↑ |0.4889|± |0.0146|
174
+ |hellaswag | 1|none | 5|acc |↑ |0.5147|± |0.0050|
175
+ | | |none | 5|acc_norm|↑ |0.7087|± |0.0045|
176
+ |winogrande | 1|none | 5|acc |↑ |0.6811|± |0.0131|
177
 
178
 
179
  # Uploaded model