Update README.md
Browse files
README.md
CHANGED
|
@@ -67,7 +67,11 @@ outputs = pipe(
|
|
| 67 |
print(outputs[0]["generated_text"][-1])
|
| 68 |
```
|
| 69 |
|
| 70 |
-
## Benchmark
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
|
| 72 |
# Uploaded finetuned model
|
| 73 |
|
|
|
|
| 67 |
print(outputs[0]["generated_text"][-1])
|
| 68 |
```
|
| 69 |
|
| 70 |
+
## Benchmark
|
| 71 |
+
These benchmark are current benchmark and not final benchmark, due to recursive fine tuning techniques self improves over time:
|
| 72 |
+
hf (pretrained=EpistemeAI/metatune-gpt20b-R0,parallelize=True,dtype=bfloat16), gen_kwargs: (temperature=1,top_p=1,max_new_tokens=1000), limit: 30.0, num_fewshot: 5, batch_size: 1
|
| 73 |
+
| Tasks |Version| Filter |n-shot| Metric | |Value |
|
| 74 |
+
|gsm8k_cot | 3|flexible-extract| 5|exact_match|↑ |0.9022|
|
| 75 |
|
| 76 |
# Uploaded finetuned model
|
| 77 |
|