Update README.md
Browse files
README.md
CHANGED
|
@@ -91,7 +91,7 @@ It is more heavy then JiRack Coder 7B INT8
|
|
| 91 |
|
| 92 |
### Important Memory Notes
|
| 93 |
|
| 94 |
-
Even though the
|
| 95 |
|
| 96 |
- KV-cache consumption during generation (especially with long context)
|
| 97 |
- ONNX Runtime overhead and temporary buffers
|
|
|
|
| 91 |
|
| 92 |
### Important Memory Notes
|
| 93 |
|
| 94 |
+
Even though the 8B INT4 model itself takes approximately **5–6 GB**, we recommend **at least 24GB VRAM** for the following reasons:
|
| 95 |
|
| 96 |
- KV-cache consumption during generation (especially with long context)
|
| 97 |
- ONNX Runtime overhead and temporary buffers
|