|
|
--- |
|
|
base_model: |
|
|
- utter-project/EuroLLM-22B-Instruct-2512 |
|
|
license: apache-2.0 |
|
|
--- |
|
|
|
|
|
# EuroLLM-22B-Instruct-2512-FP8-Dynamic |
|
|
|
|
|
FP8 W8A8 quantized version of [utter-project/EuroLLM-22B-Instruct-2512](https://huggingface.co/utter-project/EuroLLM-22B-Instruct-2512). |
|
|
|
|
|
|
|
|
#### Accuracy Comparison |
|
|
|
|
|
``` |
|
|
lm_eval --model vllm --model_args pretrained=EuroLLM-22B-Instruct-2512-FP8-Dynamic,add_bos_token=True --task gsm8k --num_fewshot 5 --batch_size auto --limit 250 |
|
|
``` |
|
|
| Model | Filter | Exact Match | StdErr | |
|
|
|------|--------|-------------:|-------:| |
|
|
| **EuroLLM-22B-Instruct-2512** | flexible-extract | 0.576 | ±0.0313 | |
|
|
| **EuroLLM-22B-Instruct-2512** | strict-match | 0.536 | ±0.0316 | |
|
|
| **EuroLLM-22B-Instruct-2512-FP8-Dynamic** | flexible-extract | 0.520 | ±0.0317 | |
|
|
| **EuroLLM-22B-Instruct-2512-FP8-Dynamic** | strict-match | 0.504 | ±0.0317 | |
