Add README for kat-dev-72b
Browse files
README.md
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
# Kat-Dev 72B (GGUF)
|
| 3 |
+
|
| 4 |
+
Quantized builds of the KAT-Dev 72B coding model for Ollama / llama.cpp runtimes. Each variant ships with the matching Modelfile generated from the Ollama registry export.
|
| 5 |
+
|
| 6 |
+
## Variants
|
| 7 |
+
|
| 8 |
+
| Variant | Size | Blob |
|
| 9 |
+
| --- | --- | --- |
|
| 10 |
+
| `iq2_m` | 27.32 GB | `sha256-cbe26a3c280f1f1070b070ac3ab9bd1c3ddc23d422bb5ba902580b107765ca9c` |
|
| 11 |
+
| `iq2_xxs` | 23.74 GB | `sha256-a49c7526f165f7320c434ceee55f72e93654a30a0ecde701a87e023d619c17b7` |
|
| 12 |
+
| `iq3_m` | 33.07 GB | `sha256-14d07184013c2ce3d8be24188512382ed972fda2901cb2f5b5a9e8ebd0c7e4b9` |
|
| 13 |
+
| `iq4_xs` | 36.98 GB | `sha256-c4cb9c6e6847031c418b076d68fb93852140a183afc171e4f62e3a84c58001f6` |
|
| 14 |
+
|
| 15 |
+
## Usage with Ollama
|
| 16 |
+
|
| 17 |
+
Example for the `iq4_xs` quantization:
|
| 18 |
+
|
| 19 |
+
```bash
|
| 20 |
+
ollama create kat-dev-72b-iq4_xs -f modelfiles/kat-dev-72b--iq4_xs.Modelfile
|
| 21 |
+
ollama run kat-dev-72b-iq4_xs
|
| 22 |
+
```
|
| 23 |
+
|
| 24 |
+
You can swap `iq4_xs` for any other variant listed above.
|
| 25 |
+
|
| 26 |
+
## Source
|
| 27 |
+
|
| 28 |
+
Originally published on my Ollama profile: https://ollama.com/richardyoung/kat-dev-72b
|