LenDigLearn commited on
Commit
c5a8a5e
·
verified ·
1 Parent(s): 1fc3a30

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -62,9 +62,9 @@ Our data encompasses examples of a length up to 16384 tokens, further enhancing
62
 
63
  ## Evaluation
64
 
65
- We performed benchmarks using lighteval. The accuracy numbers obtained this way differ greatly from the base model's official benchmarks and those performed with different benchmark suites.
66
  Thus, we have run the same benchmarks using lighteval on the [base model](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) under the exact same conditions as well for comparison.
67
- As of 2025-01-24, We are working on running these benchmarks again using a different suite as well as running more German-specific benchmarks.
68
 
69
  ### English Benchmarks
70
  | Benchmark | Mistral-Nemo-Instruct 2407 | educa-ai-nemo-sft |
 
62
 
63
  ## Evaluation
64
 
65
+ **IMPORTANT:** We performed benchmarks using lighteval. The accuracy numbers obtained this way differ greatly from the base model's official benchmarks and those performed with different benchmark suites.
66
  Thus, we have run the same benchmarks using lighteval on the [base model](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) under the exact same conditions as well for comparison.
67
+ **As of 2025-01-24, We are working on running these benchmarks again using a different suite as well as running more German-specific benchmarks.**
68
 
69
  ### English Benchmarks
70
  | Benchmark | Mistral-Nemo-Instruct 2407 | educa-ai-nemo-sft |