b4rtaz commited on
Commit
0bc5c51
·
verified ·
1 Parent(s): 5d3d33f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +31 -3
README.md CHANGED
@@ -1,3 +1,31 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: distributed-llama
4
+ tags:
5
+ - distributed-inference
6
+ - text-generation
7
+ ---
8
+
9
+
10
+ This is converted [Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B) model to Distributed Llama format. The model is quantized to Q40.
11
+
12
+ ## 🚀 How to Run?
13
+
14
+ * ⏬ Download [Distributed Llama repository](https://github.com/b4rtaz/distributed-llama).
15
+ * 🔨 Build Distributed Llama:
16
+
17
+ ```
18
+ make dllama
19
+ ```
20
+
21
+ * Download this model: `python launch.py qwen3_14b_q40`
22
+
23
+ ## 🚀 Run Distributed Llama:
24
+
25
+ ```
26
+ ./dllama chat --model dllama_model_qwen3_14b_q40 --tokenizer dllama_tokenizer_qwen3_14b.t --buffer-float-type q80 --nthreads 4 --max-seq-len 8192
27
+ ```
28
+
29
+ ## 🎩 License
30
+
31
+ [Apache 2.0](https://huggingface.co/Qwen/Qwen3-14B/blob/main/LICENSE)