Update README.md
Browse files
README.md
CHANGED
|
@@ -108,11 +108,11 @@ Code to duplicate the benchmark (Using +std for final result)
|
|
| 108 |
|
| 109 |
```
|
| 110 |
|
| 111 |
-
hf (pretrained=EpistemeAI/metatune-gpt20b-R1.
|
| 112 |
-
| Tasks |Version| Filter |n-shot| Metric |metatune R1.
|
| 113 |
|-------------------------|------:|----------------|:-----|-----------|:------------|:-----------|:----------|
|
| 114 |
|gsm8k_cot_llama | 3|flexible- extrac| 0|exact_match| +1.0(0.9) |+1.0(0.9) |0.91 |
|
| 115 |
-
|
| 116 |
# Uploaded finetuned model
|
| 117 |
|
| 118 |
- **Developed by:** EpistemeAI
|
|
|
|
| 108 |
|
| 109 |
```
|
| 110 |
|
| 111 |
+
hf (pretrained=EpistemeAI/metatune-gpt20b-R1.09,parallelize=True,dtype=bfloat16), gen_kwargs: (temperature=0.9,top_p=0.9,max_new_tokens=2048), limit: 10.0, num_fewshot: 0, batch_size: auto:4
|
| 112 |
+
| Tasks |Version| Filter |n-shot| Metric |metatune R1.09(high)| metatune R1.1|metatune R0|
|
| 113 |
|-------------------------|------:|----------------|:-----|-----------|:------------|:-----------|:----------|
|
| 114 |
|gsm8k_cot_llama | 3|flexible- extrac| 0|exact_match| +1.0(0.9) |+1.0(0.9) |0.91 |
|
| 115 |
+
|gpqa_diamond_cot_zeroshot| 1|flexible-extract| 0|exact_match| +0.8528 |0.933 | |
|
| 116 |
# Uploaded finetuned model
|
| 117 |
|
| 118 |
- **Developed by:** EpistemeAI
|