Update README.md
Browse files
README.md
CHANGED
|
@@ -54,32 +54,32 @@ For commercial licensing, cluster deployment, or enterprise use of the JiRack Co
|
|
| 54 |
|
| 55 |
## Hardware Recommendations for AMD Systems
|
| 56 |
|
| 57 |
-
|
| 58 |
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
-
|
| 62 |
-
|-----------------------|----------------------------------|--------------------------------------|----------|----------------------|---------------------------|
|
| 63 |
-
| **Good Performance** | Ryzen 7 7700 / 9700X | RX 7900 XTX / 7900 XT (24GB) | 32GB+ | 45-70 tokens/s | Best balance |
|
| 64 |
-
| **Very Good** | Ryzen 9 7950X / 9950X | RX 7900 XTX (24GB) | 64GB | 55-85 tokens/s | Strong choice |
|
| 65 |
-
| **Enterprise / Fast** | EPYC 7003/9004 series | Instinct MI300X or 2x RX 7900 XTX | 128GB+ | 80-120+ tokens/s | For 32B model too |
|
| 66 |
-
| **Budget / Decent** | Ryzen 5 7600 / 9600X | RX 7800 XT (16GB) | 32GB | 35-50 tokens/s | Acceptable |
|
| 67 |
|
| 68 |
-
|
| 69 |
|
| 70 |
-
|
| 71 |
-
-
|
| 72 |
-
-
|
| 73 |
-
-
|
| 74 |
|
| 75 |
-
**
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
|
| 80 |
---
|
| 81 |
|
| 82 |
-
|
| 83 |
|
| 84 |
## 📧 Contact & Licensing
|
| 85 |
For joint venture opportunities, hardware integration, or licensing inquiries:
|
|
|
|
| 54 |
|
| 55 |
## Hardware Recommendations for AMD Systems
|
| 56 |
|
| 57 |
+
### Recommended Hardware for JiRack Coder 7B INT8
|
| 58 |
|
| 59 |
+
| Use Case | CPU | GPU (ROCm) | VRAM / RAM | Expected Speed | Recommendation |
|
| 60 |
+
|-----------------------|----------------------------------|-----------------------------------|----------------|---------------------|--------------------|
|
| 61 |
+
| **Recommended** | Ryzen 7 7700 / 9700X | RX 7900 XTX / 7900 XT | 24GB VRAM | 50-75 tokens/s | Best choice |
|
| 62 |
+
| **High Performance** | Ryzen 9 7950X / 9950X | RX 7900 XTX | 24GB+ VRAM | 65-90 tokens/s | Excellent |
|
| 63 |
+
| **Enterprise** | EPYC 7003/9004 series | MI300X or 2x RX 7900 XTX | 48GB+ VRAM | 90-140 tokens/s | For 32B model |
|
| 64 |
+
| **Budget Option** | Ryzen 5 7600 / 9600X | RX 7800 XT (16GB) | 16GB VRAM | 35-50 tokens/s | Acceptable |
|
| 65 |
|
| 66 |
+
### Important Memory Notes
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
+
Even though the 7B INT8 model itself takes approximately **8–9 GB**, we recommend **at least 24GB VRAM** for the following reasons:
|
| 69 |
|
| 70 |
+
- KV-cache consumption during generation (especially with long context)
|
| 71 |
+
- ONNX Runtime overhead and temporary buffers
|
| 72 |
+
- System stability and to avoid Out of Memory errors
|
| 73 |
+
- Room for larger context windows
|
| 74 |
|
| 75 |
+
**Minimum recommended:** 24GB VRAM (RX 7900 series)
|
| 76 |
+
**Ideal:** 24–32GB VRAM
|
| 77 |
+
|
| 78 |
+
For pure CPU inference (no GPU), we recommend at least **64GB system RAM** (Ryzen 9 7950X/9950X).
|
| 79 |
|
| 80 |
---
|
| 81 |
|
| 82 |
+
|
| 83 |
|
| 84 |
## 📧 Contact & Licensing
|
| 85 |
For joint venture opportunities, hardware integration, or licensing inquiries:
|