Update README.md
Browse files
README.md
CHANGED
|
@@ -166,14 +166,14 @@ For further details, questions, or feedback, please email episteme.ai@proton.me
|
|
| 166 |
|
| 167 |
## Benchmark
|
| 168 |
|
| 169 |
-
hf (pretrained=EpistemeAI/ReasoningCore-3B-R01), gen_kwargs: (None), limit: None, num_fewshot:
|
| 170 |
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
|
| 171 |
|-------------|------:|------|-----:|--------|---|-----:|---|-----:|
|
| 172 |
-
|arc_challenge| 1|none |
|
| 173 |
-
| | |none |
|
| 174 |
-
|hellaswag | 1|none |
|
| 175 |
-
| | |none |
|
| 176 |
-
|winogrande | 1|none |
|
| 177 |
|
| 178 |
|
| 179 |
# Uploaded model
|
|
|
|
| 166 |
|
| 167 |
## Benchmark
|
| 168 |
|
| 169 |
+
hf (pretrained=EpistemeAI/ReasoningCore-3B-R01), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 8
|
| 170 |
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
|
| 171 |
|-------------|------:|------|-----:|--------|---|-----:|---|-----:|
|
| 172 |
+
|arc_challenge| 1|none | 5|acc |↑ |0.4352|± |0.0145|
|
| 173 |
+
| | |none | 5|acc_norm|↑ |0.4889|± |0.0146|
|
| 174 |
+
|hellaswag | 1|none | 5|acc |↑ |0.5147|± |0.0050|
|
| 175 |
+
| | |none | 5|acc_norm|↑ |0.7087|± |0.0045|
|
| 176 |
+
|winogrande | 1|none | 5|acc |↑ |0.6811|± |0.0131|
|
| 177 |
|
| 178 |
|
| 179 |
# Uploaded model
|