Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,9 @@ base_model: databricks/dbrx-instruct
|
|
| 11 |
**Apple/Metal support task is still open, missing clamp op implementation - this means some quants here won't work on Metal.**
|
| 12 |
|
| 13 |
**All quants in this repo have been tested successfully running the following command:**
|
| 14 |
-
`
|
|
|
|
|
|
|
| 15 |
|
| 16 |
* GGUF importance matrix (imatrix) quants for https://huggingface.co/databricks/dbrx-instruct
|
| 17 |
* The importance matrix is trained for ~100K tokens (200 batches of 512 tokens) using [wiki.train.raw](https://huggingface.co/datasets/wikitext).
|
|
|
|
| 11 |
**Apple/Metal support task is still open, missing clamp op implementation - this means some quants here won't work on Metal.**
|
| 12 |
|
| 13 |
**All quants in this repo have been tested successfully running the following command:**
|
| 14 |
+
```
|
| 15 |
+
./build/bin/main -ngl 41 -s 0 -e -p "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWrite an essay about AI.<|im_end|>\n<|im_start|>assistant\n" -m ggml-dbrx-instruct-16x12b-<<quant-to-test>>.gguf
|
| 16 |
+
```
|
| 17 |
|
| 18 |
* GGUF importance matrix (imatrix) quants for https://huggingface.co/databricks/dbrx-instruct
|
| 19 |
* The importance matrix is trained for ~100K tokens (200 batches of 512 tokens) using [wiki.train.raw](https://huggingface.co/datasets/wikitext).
|