Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -143,6 +143,23 @@ python -m sglang.launch_server \
|
|
| 143 |
--tool-call-parser qwen3_coder
|
| 144 |
```
|
| 145 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 146 |
## Related Models
|
| 147 |
|
| 148 |
| Variant | Description | Link |
|
|
|
|
| 143 |
--tool-call-parser qwen3_coder
|
| 144 |
```
|
| 145 |
|
| 146 |
+
### Convert to GGUF
|
| 147 |
+
|
| 148 |
+
You can quantize this model yourself using [llama.cpp](https://github.com/ggerganov/llama.cpp):
|
| 149 |
+
|
| 150 |
+
```bash
|
| 151 |
+
# Pull the model
|
| 152 |
+
git lfs install
|
| 153 |
+
git clone https://huggingface.co/clarkkitchen22/Pokemon-Red-Qwen3-80B
|
| 154 |
+
|
| 155 |
+
# Build llama.cpp and convert
|
| 156 |
+
git clone https://github.com/ggerganov/llama.cpp
|
| 157 |
+
cd llama.cpp && make -j
|
| 158 |
+
|
| 159 |
+
# Convert and quantize
|
| 160 |
+
python convert_hf_to_gguf.py ../Pokemon-Red-Qwen3-80B --outtype q4_k_m
|
| 161 |
+
```
|
| 162 |
+
|
| 163 |
## Related Models
|
| 164 |
|
| 165 |
| Variant | Description | Link |
|