Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,65 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- ru
|
| 5 |
+
tags:
|
| 6 |
+
- gguf
|
| 7 |
+
- instruct
|
| 8 |
+
---
|
| 9 |
+
# VokiLLM-0.5B-Instruct (GGUF)
|
| 10 |
+
|
| 11 |
+
Готовые файлы модели **VokiLLM-0.5B-Instruct** в формате **GGUF**: FP16 и квантовки для `llama.cpp`/LM Studio и других совместимых рантаймов.
|
| 12 |
+
|
| 13 |
+
## Кратко
|
| 14 |
+
|
| 15 |
+
- **Архитектура**: `qwen2`
|
| 16 |
+
- **Размер**: ~0.5B (лейбл в GGUF: `630M`)
|
| 17 |
+
- **Контекст**: 8192
|
| 18 |
+
- **Назначение**: Instruct / чат
|
| 19 |
+
|
| 20 |
+
## Файлы
|
| 21 |
+
|
| 22 |
+
| Файл | Квантование | Размер (bytes) |
|
| 23 |
+
|---|---:|---:|
|
| 24 |
+
| `vokillm-0.5b-instruct-fp16.gguf` | FP16 | 1266425856 |
|
| 25 |
+
| `vokillm-0.5b-instruct-q8_0.gguf` | Q8_0 | 675710976 |
|
| 26 |
+
| `vokillm-0.5b-instruct-q6_k.gguf` | Q6_K | 650379296 |
|
| 27 |
+
| `vokillm-0.5b-instruct-q5_k_m.gguf` | Q5_K_M | 522186752 |
|
| 28 |
+
| `vokillm-0.5b-instruct-q5_1.gguf` | Q5_1 | 521348096 |
|
| 29 |
+
| `vokillm-0.5b-instruct-q5_k_s.gguf` | Q5_K_S | 514810880 |
|
| 30 |
+
| `vokillm-0.5b-instruct-q5_0.gguf` | Q5_0 | 490475520 |
|
| 31 |
+
| `vokillm-0.5b-instruct-q4_k_m.gguf` | Q4_K_M | 491400192 |
|
| 32 |
+
| `vokillm-0.5b-instruct-q4_k_s.gguf` | Q4_K_S | 479064064 |
|
| 33 |
+
| `vokillm-0.5b-instruct-q4_1.gguf` | Q4_1 | 459602944 |
|
| 34 |
+
| `vokillm-0.5b-instruct-q3_k_l.gguf` | Q3_K_L | 445933568 |
|
| 35 |
+
| `vokillm-0.5b-instruct-iq4_nl.gguf` | IQ4_NL | 430880768 |
|
| 36 |
+
| `vokillm-0.5b-instruct-q4_0.gguf` | Q4_0 | 428730368 |
|
| 37 |
+
| `vokillm-0.5b-instruct-iq4_xs.gguf` | IQ4_XS | 428020736 |
|
| 38 |
+
| `vokillm-0.5b-instruct-q3_k_m.gguf` | Q3_K_M | 432041984 |
|
| 39 |
+
| `vokillm-0.5b-instruct-iq3_xxs.gguf` | IQ3_XXS | 416919296 |
|
| 40 |
+
| `vokillm-0.5b-instruct-iq3_s.gguf` | IQ3_S | 415182848 |
|
| 41 |
+
| `vokillm-0.5b-instruct-q2_k.gguf` | Q2_K | 415182848 |
|
| 42 |
+
| `vokillm-0.5b-instruct-q3_k_s.gguf` | Q3_K_S | 414838784 |
|
| 43 |
+
| `vokillm-0.5b-instruct-iq2_s.gguf` | IQ2_S | 402312928 |
|
| 44 |
+
| `vokillm-0.5b-instruct-iq2_xs.gguf` | IQ2_XS | 400985056 |
|
| 45 |
+
| `vokillm-0.5b-instruct-iq2_xxs.gguf` | IQ2_XXS | 398125024 |
|
| 46 |
+
| `vokillm-0.5b-instruct-iq1_s.gguf` | IQ1_S | 392404960 |
|
| 47 |
+
|
| 48 |
+
## Как запустить
|
| 49 |
+
|
| 50 |
+
### llama.cpp (CLI)
|
| 51 |
+
|
| 52 |
+
Пример (Windows):
|
| 53 |
+
|
| 54 |
+
```bash
|
| 55 |
+
llama-cli.exe -m "vokillm-0.5b-instruct-q4_k_m.gguf" -p "Привет! Коротко объясни, что такое GGUF."
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
### LM Studio
|
| 59 |
+
|
| 60 |
+
- Открой LM Studio → **Models** → **Add model** → выбери нужный `*.gguf`
|
| 61 |
+
- Рекомендуемый стартовый вариант: `q4_k_m` (баланс скорость/качество)
|
| 62 |
+
|
| 63 |
+
## Про квантовки IQ2/IQ1
|
| 64 |
+
|
| 65 |
+
Квантовки `IQ2_*` и `IQ1_S` сделаны с **importance matrix (imatrix)** (это повышает качество для “экстремальных” квантовок).
|