kgrabko commited on
Commit
e61f1c7
·
verified ·
1 Parent(s): 3f32acf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -18
README.md CHANGED
@@ -54,32 +54,32 @@ For commercial licensing, cluster deployment, or enterprise use of the JiRack Co
54
 
55
  ## Hardware Recommendations for AMD Systems
56
 
57
- Here's a clear and practical hardware recommendation for running **JiRack Coder 7B** (INT8 / INT4) on AMD platforms:
58
 
59
- ### Best AMD Hardware Recommendations
 
 
 
 
 
60
 
61
- | Use Case | Recommended CPU | Recommended GPU (ROCm) | RAM | Expected Speed | Notes |
62
- |-----------------------|----------------------------------|--------------------------------------|----------|----------------------|---------------------------|
63
- | **Good Performance** | Ryzen 7 7700 / 9700X | RX 7900 XTX / 7900 XT (24GB) | 32GB+ | 45-70 tokens/s | Best balance |
64
- | **Very Good** | Ryzen 9 7950X / 9950X | RX 7900 XTX (24GB) | 64GB | 55-85 tokens/s | Strong choice |
65
- | **Enterprise / Fast** | EPYC 7003/9004 series | Instinct MI300X or 2x RX 7900 XTX | 128GB+ | 80-120+ tokens/s | For 32B model too |
66
- | **Budget / Decent** | Ryzen 5 7600 / 9600X | RX 7800 XT (16GB) | 32GB | 35-50 tokens/s | Acceptable |
67
 
68
- ### Key Recommendations (AMD)
69
 
70
- **Top Pick (Best Price/Performance):**
71
- - **CPU**: Ryzen 7 9700X or Ryzen 9 7950X
72
- - **GPU**: Radeon RX 7900 XTX (24GB VRAM) strongly recommended
73
- - **RAM**: 64GB DDR5
74
 
75
- **Important Notes:**
76
- - ROCm support is decent on RX 7000 series, but less stable than NVIDIA.
77
- - For pure CPU inference, Ryzen 9 7950X / 9950X with 64GB+ RAM performs excellently.
78
- - The **INT4** version runs noticeably faster than INT8 on AMD CPUs.
79
 
80
  ---
81
 
82
- Would you like me to add recommendations based on your budget (e.g. under $1500, mid-range, high-end)?
83
 
84
  ## 📧 Contact & Licensing
85
  For joint venture opportunities, hardware integration, or licensing inquiries:
 
54
 
55
  ## Hardware Recommendations for AMD Systems
56
 
57
+ ### Recommended Hardware for JiRack Coder 7B INT8
58
 
59
+ | Use Case | CPU | GPU (ROCm) | VRAM / RAM | Expected Speed | Recommendation |
60
+ |-----------------------|----------------------------------|-----------------------------------|----------------|---------------------|--------------------|
61
+ | **Recommended** | Ryzen 7 7700 / 9700X | RX 7900 XTX / 7900 XT | 24GB VRAM | 50-75 tokens/s | Best choice |
62
+ | **High Performance** | Ryzen 9 7950X / 9950X | RX 7900 XTX | 24GB+ VRAM | 65-90 tokens/s | Excellent |
63
+ | **Enterprise** | EPYC 7003/9004 series | MI300X or 2x RX 7900 XTX | 48GB+ VRAM | 90-140 tokens/s | For 32B model |
64
+ | **Budget Option** | Ryzen 5 7600 / 9600X | RX 7800 XT (16GB) | 16GB VRAM | 35-50 tokens/s | Acceptable |
65
 
66
+ ### Important Memory Notes
 
 
 
 
 
67
 
68
+ Even though the 7B INT8 model itself takes approximately **8–9 GB**, we recommend **at least 24GB VRAM** for the following reasons:
69
 
70
+ - KV-cache consumption during generation (especially with long context)
71
+ - ONNX Runtime overhead and temporary buffers
72
+ - System stability and to avoid Out of Memory errors
73
+ - Room for larger context windows
74
 
75
+ **Minimum recommended:** 24GB VRAM (RX 7900 series)
76
+ **Ideal:** 24–32GB VRAM
77
+
78
+ For pure CPU inference (no GPU), we recommend at least **64GB system RAM** (Ryzen 9 7950X/9950X).
79
 
80
  ---
81
 
82
+
83
 
84
  ## 📧 Contact & Licensing
85
  For joint venture opportunities, hardware integration, or licensing inquiries: