Fix accuracy benchmark commands
Browse files
README.md
CHANGED
|
@@ -106,23 +106,29 @@ scp -r LFM2-2.6B-a16w4_return_logits sima@<modalix-ip>:/media/nvme/llima/models/
|
|
| 106 |
|
| 107 |
### Modalix Backend Accuracy
|
| 108 |
|
| 109 |
-
Run
|
| 110 |
|
| 111 |
```bash
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
--
|
| 115 |
-
--
|
|
|
|
|
|
|
|
|
|
| 116 |
```
|
| 117 |
|
| 118 |
-
The `--board_model`
|
| 119 |
|
| 120 |
-
###
|
| 121 |
|
| 122 |
-
Run the
|
| 123 |
|
| 124 |
```bash
|
| 125 |
-
llima-benchmark accuracy LiquidAI/LFM2-2.6B
|
|
|
|
|
|
|
|
|
|
| 126 |
```
|
| 127 |
|
| 128 |
See [MOLE accuracy benchmarking](https://developer.sima.ai/software/genai-llima/mole) for the full accuracy benchmarking workflow.
|
|
|
|
| 106 |
|
| 107 |
### Modalix Backend Accuracy
|
| 108 |
|
| 109 |
+
Run the benchmark from a host machine. The positional `model_id` must be the source Hugging Face model because MOLE uses its tokenizer and configuration; `--board_model` selects this compiled artifact on Modalix.
|
| 110 |
|
| 111 |
```bash
|
| 112 |
+
MODALIX_IP="<modalix-ip>"
|
| 113 |
+
llima-benchmark accuracy LiquidAI/LFM2-2.6B \
|
| 114 |
+
--backend modalix \
|
| 115 |
+
--output ./results/modalix \
|
| 116 |
+
--board_ip "$MODALIX_IP" \
|
| 117 |
+
--board_model LFM2-2.6B-a16w4_return_logits \
|
| 118 |
+
--task hellaswag piqa triviaqa wikitext winogrande
|
| 119 |
```
|
| 120 |
|
| 121 |
+
The compiled `--board_model` must already exist on Modalix and contain both `devkit/` and `elf_files/`. The CLI starts the benchmark server by default. Add `--board_venv_path /path/to/venv` when `llima` is installed in a non-default virtual environment, or `--no-board_start_server` when connecting to a server that is already running.
|
| 122 |
|
| 123 |
+
### Hugging Face Backend Accuracy
|
| 124 |
|
| 125 |
+
Run the same task suite against the source model on the host:
|
| 126 |
|
| 127 |
```bash
|
| 128 |
+
llima-benchmark accuracy LiquidAI/LFM2-2.6B \
|
| 129 |
+
--backend hf \
|
| 130 |
+
--output ./results/hf \
|
| 131 |
+
--task hellaswag piqa triviaqa wikitext winogrande
|
| 132 |
```
|
| 133 |
|
| 134 |
See [MOLE accuracy benchmarking](https://developer.sima.ai/software/genai-llima/mole) for the full accuracy benchmarking workflow.
|