justindal commited on
Commit
41dac46
·
verified ·
1 Parent(s): bdc2b91

update readme

Browse files
Files changed (1) hide show
  1. README.md +26 -0
README.md CHANGED
@@ -14,3 +14,29 @@ tags:
14
  - qwen-coder
15
  - mlx
16
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  - qwen-coder
15
  - mlx
16
  ---
17
+
18
+ # mlx-community/Qwen2.5-Coder-7B-Instruct
19
+
20
+ The Model [mlx-community/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/mlx-community/Qwen2.5-Coder-7B-Instruct) was converted to MLX format from [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) using mlx-lm version **0.31.1**.
21
+
22
+ ## Use with mlx
23
+
24
+ ```bash
25
+ pip install mlx-lm
26
+ ```
27
+
28
+ ```python
29
+ from mlx_lm import load, generate
30
+
31
+ model, tokenizer = load("mlx-community/Qwen2.5-Coder-7B-Instruct")
32
+
33
+ prompt = "hello"
34
+
35
+ if tokenizer.chat_template is not None:
36
+ messages = [{"role": "user", "content": prompt}]
37
+ prompt = tokenizer.apply_chat_template(
38
+ messages, add_generation_prompt=True, return_dict=False,
39
+ )
40
+
41
+ response = generate(model, tokenizer, prompt=prompt, verbose=True)
42
+ ```