Spaces:
Sleeping
Sleeping
Commit ·
2caec2a
1
Parent(s): bb2afa9
Add documentation for running on RTX 4050 6GB
Browse files- PROJECT_DOCS.md +13 -0
PROJECT_DOCS.md
CHANGED
|
@@ -109,3 +109,16 @@ To make this run on the cloud:
|
|
| 109 |
|
| 110 |
---
|
| 111 |
**Developed by Ihtesham | Powered by Open Source AI**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 109 |
|
| 110 |
---
|
| 111 |
**Developed by Ihtesham | Powered by Open Source AI**
|
| 112 |
+
|
| 113 |
+
---
|
| 114 |
+
|
| 115 |
+
## 8. Local Hardware Recommendations (RTX 4050)
|
| 116 |
+
If you run this application on an **RTX 4050 (6GB VRAM) + 16GB RAM**:
|
| 117 |
+
|
| 118 |
+
| Model | Size | VRAM Usage | Speed (Tokens/s) | Recommendation |
|
| 119 |
+
| :--- | :--- | :--- | :--- | :--- |
|
| 120 |
+
| **Qwen-0.5B** | 0.5B | ~0.8 GB | **100+** (Instant) | ⚡ Overkill Speed. Low Memory usage. |
|
| 121 |
+
| **Qwen-1.5B** | 1.5B | ~2.5 GB | **70+** (Very Fast) | ✅ **Perfect Balance.** Best for 6GB cards. |
|
| 122 |
+
| **Qwen-7B (4-bit)** | 7B | ~5.5 GB | **30+** (Fast) | 🧠 **Smartest.** Maxes out your VRAM. |
|
| 123 |
+
|
| 124 |
+
**Conclusion**: Your RTX 4050 is **10x more powerful** than the Free Tier CPU. You should upgrade to the **1.5B Model** locally for better intelligence without sacrificing speed.
|