Update README.md
Browse files
README.md
CHANGED
|
@@ -62,9 +62,9 @@ Our data encompasses examples of a length up to 16384 tokens, further enhancing
|
|
| 62 |
|
| 63 |
## Evaluation
|
| 64 |
|
| 65 |
-
We performed benchmarks using lighteval. The accuracy numbers obtained this way differ greatly from the base model's official benchmarks and those performed with different benchmark suites.
|
| 66 |
Thus, we have run the same benchmarks using lighteval on the [base model](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) under the exact same conditions as well for comparison.
|
| 67 |
-
As of 2025-01-24, We are working on running these benchmarks again using a different suite as well as running more German-specific benchmarks
|
| 68 |
|
| 69 |
### English Benchmarks
|
| 70 |
| Benchmark | Mistral-Nemo-Instruct 2407 | educa-ai-nemo-sft |
|
|
|
|
| 62 |
|
| 63 |
## Evaluation
|
| 64 |
|
| 65 |
+
**IMPORTANT:** We performed benchmarks using lighteval. The accuracy numbers obtained this way differ greatly from the base model's official benchmarks and those performed with different benchmark suites.
|
| 66 |
Thus, we have run the same benchmarks using lighteval on the [base model](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) under the exact same conditions as well for comparison.
|
| 67 |
+
**As of 2025-01-24, We are working on running these benchmarks again using a different suite as well as running more German-specific benchmarks.**
|
| 68 |
|
| 69 |
### English Benchmarks
|
| 70 |
| Benchmark | Mistral-Nemo-Instruct 2407 | educa-ai-nemo-sft |
|