aivisionslab commited on
Commit
795ffd0
·
verified ·
1 Parent(s): 471c462

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +187 -11
README.md CHANGED
@@ -1,20 +1,196 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # AIVisionsLab — RX580 Vulkan Stack
2
 
3
- Modular Local AI Infrastructure optimized for legacy hardware.
4
- *“O hardware não morre, ele se transforma.”*
 
 
 
 
 
 
 
 
 
5
 
6
  ---
7
 
8
- ## 🛠 Acesso Rápido
9
- - **[Documentação Master](https://setup-ia-local-rx580-vulkan.web.app/)** — O portal do projeto.
10
- - **[Leia o Manifesto do Laboratório](MANIFESTO.md)** — A filosofia por trás do código.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
  ---
13
 
14
- ## Hardware Configuration
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  ```bash
16
- GPU : RX 580 8GB
17
- Backend : Vulkan
18
- Inference : llama.cpp
19
- Generation : Flux GGUF
20
- UI : OpenWebUI
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - pt
4
+ - en
5
+ - es
6
+ - fr
7
+ - ar
8
+ tags:
9
+ - llama.cpp
10
+ - vulkan
11
+ - local-ai
12
+ - gguf
13
+ - amd
14
+ - rx580
15
+ - stable-diffusion
16
+ - ollama
17
+ - openwebui
18
+ - hardware-revival
19
+ - offline
20
+ - inference
21
+ license: mit
22
+ library_name: llama-cpp-python
23
+ pipeline_tag: text-generation
24
+ ---
25
+
26
  # AIVisionsLab — RX580 Vulkan Stack
27
 
28
+ > *"O hardware não morre, ele se transforma."*
29
+
30
+ Complete local AI infrastructure for **AMD RX 580 8GB (Polaris/GCN4)** using **Vulkan** as the compute backend — no CUDA, no ROCm, no cloud, no new hardware.
31
+
32
+ ---
33
+
34
+ ## What This Is
35
+
36
+ This is not a fine-tuned model. This is a **documented stack** — configuration files, build scripts, benchmarks, and guides — for running local AI inference on a 2017-era AMD GPU that mainstream frameworks consider unsupported.
37
+
38
+ AMD dropped official ROCm support for GCN4/Polaris on Windows. This project proves that `llama.cpp` + Vulkan fills that gap completely.
39
 
40
  ---
41
 
42
+ ## Stack
43
+
44
+ | Component | Tool | Notes |
45
+ |-----------|------|-------|
46
+ | LLM Inference | `llama.cpp` (Vulkan build) | Compiled with `-DLLAMA_VULKAN=ON` |
47
+ | Model serving | `Ollama` | Auto-detects Vulkan on AMD |
48
+ | Chat interface | `OpenWebUI` | Docker, runs at localhost:3000 |
49
+ | Image generation | `stable-diffusion.cpp` | Vulkan backend, SD 1.5 native |
50
+ | Image interface | `ComfyUI` | WSL2, CPU stable mode |
51
+ | Advanced images | `Flux.1 Schnell` | CPU+GPU hybrid mode |
52
+
53
+ ---
54
+
55
+ ## Hardware Tested
56
+
57
+ ```
58
+ GPU : AMD RX 580 8GB (Polaris 20 / GCN4)
59
+ CPU : Intel Xeon E5-2670 v3 (12c/24t @ 2.3GHz)
60
+ RAM : 32GB DDR4 ECC
61
+ Storage : NVMe SSD (models)
62
+ OS : Windows 11 + WSL2 (Ubuntu 22.04)
63
+ Driver : AMD Adrenalin (latest)
64
+ ```
65
 
66
  ---
67
 
68
+ ## Recommended Models
69
+
70
+ ### LLMs (GGUF Q4_K_M)
71
+
72
+ | Model | Size | Speed on RX 580 | Use Case |
73
+ |-------|------|-----------------|----------|
74
+ | Llama 3.2 3B | ~2GB | ~18 tok/s | Fast general use |
75
+ | Mistral 7B | ~4GB | ~9 tok/s | Best quality/speed |
76
+ | Qwen2.5 7B | ~4GB | ~8 tok/s | Portuguese / multilingual |
77
+ | Phi-3 Mini | ~2GB | ~20 tok/s | Low RAM machines |
78
+ | CodeLlama 7B | ~4GB | ~9 tok/s | Code generation |
79
+
80
+ ### Image Generation
81
+
82
+ | Model | Backend | Speed | Notes |
83
+ |-------|---------|-------|-------|
84
+ | SD 1.5 512x512 | Vulkan GPU | ~8s/img | 20 steps, native |
85
+ | SDXL | Vulkan GPU | ~45s/img | Possible but slow |
86
+ | Flux.1 Schnell | CPU+GPU | ~3min/img | High quality |
87
+
88
+ ---
89
+
90
+ ## Quick Start
91
+
92
+ ### 1. Install Ollama
93
+ ```bash
94
+ # Download from https://ollama.com
95
+ ollama pull llama3.2
96
+ ollama run llama3.2
97
+ ```
98
+
99
+ ### 2. Run OpenWebUI
100
+ ```bash
101
+ docker run -d \
102
+ -p 3000:8080 \
103
+ --add-host=host.docker.internal:host-gateway \
104
+ -v open-webui:/app/backend/data \
105
+ --name open-webui \
106
+ --restart always \
107
+ ghcr.io/open-webui/open-webui:main
108
+ ```
109
+ Open `http://localhost:3000` — connect Ollama at `http://host.docker.internal:11434`
110
+
111
+ ### 3. Build llama.cpp with Vulkan
112
+ ```bash
113
+ git clone https://github.com/ggerganov/llama.cpp
114
+ cd llama.cpp
115
+ cmake -B build -DLLAMA_VULKAN=ON
116
+ cmake --build build --config Release -j8
117
+
118
+ ./build/bin/llama-server \
119
+ --model ./models/mistral-7b-q4_k_m.gguf \
120
+ --n-gpu-layers 35 \
121
+ --port 8080
122
+ ```
123
+
124
+ ### 4. Build stable-diffusion.cpp with Vulkan
125
  ```bash
126
+ git clone https://github.com/leejet/stable-diffusion.cpp
127
+ cd stable-diffusion.cpp
128
+ cmake -B build -DSD_VULKAN=ON
129
+ cmake --build build --config Release
130
+
131
+ ./build/sd-server \
132
+ --model ./models/v1-5-pruned-emaonly.safetensors \
133
+ --port 7860
134
+ ```
135
+
136
+ ---
137
+
138
+ ## Windows Firewall Fix
139
+
140
+ Docker can't reach Ollama by default. Run as Administrator:
141
+
142
+ ```powershell
143
+ New-NetFirewallRule `
144
+ -DisplayName "Allow Docker to Ollama" `
145
+ -Direction Inbound `
146
+ -Action Allow `
147
+ -Protocol TCP `
148
+ -LocalPort 11434 `
149
+ -RemoteAddress 172.16.0.0/12
150
+ ```
151
+
152
+ ---
153
+
154
+ ## Startup Script
155
+
156
+ ```batch
157
+ @echo off
158
+ :: Start Ollama if not running
159
+ tasklist /FI "IMAGENAME eq ollama.exe" 2>NUL | find /I "ollama.exe" >NUL
160
+ if errorlevel 1 start "" "%LOCALAPPDATA%\Programs\Ollama\ollama.exe" serve
161
+
162
+ :: Start OpenWebUI
163
+ timeout /t 3 >NUL
164
+ docker start open-webui
165
+
166
+ echo AI Stack ready at http://localhost:3000
167
+ start http://localhost:3000
168
+ ```
169
+
170
+ ---
171
+
172
+ ## Why Vulkan?
173
+
174
+ | | CUDA | ROCm | **Vulkan** |
175
+ |---|---|---|---|
176
+ | NVIDIA support | ✅ | ❌ | ✅ |
177
+ | AMD support (modern) | ❌ | ✅ | ✅ |
178
+ | AMD RX 580 Windows | ❌ | ❌ | **✅** |
179
+ | Open standard | ❌ | ✅ | ✅ |
180
+ | Setup complexity | Medium | High | **Low** |
181
+
182
+ ---
183
+
184
+ ## Links
185
+
186
+ - 📖 [Full Documentation Portal](https://setup-ia-local-rx580-vulkan.web.app/)
187
+ - 💻 [GitHub Repository](https://github.com/aivisionslab-studios/rx580-local-ai-guide)
188
+ - 🎥 [YouTube Channel](https://youtube.com/@aivisionslab-hub)
189
+
190
+ ---
191
+
192
+ ## License
193
+
194
+ MIT — use it, fork it, share it.
195
+
196
+ *Built in São Paulo, Brazil. Tested on hardware from 2014–2017.*