ZhafranR commited on
Commit
ef741c2
·
verified ·
1 Parent(s): 50666c8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -0
README.md CHANGED
@@ -10,6 +10,31 @@ base_model: XeAI/LLaMa_3.2_3B_Instruct_Text2SQL
10
  This model was converted to GGUF format from [`XeAI/LLaMa_3.2_3B_Instruct_Text2SQL`](https://huggingface.co/XeAI/LLaMa_3.2_3B_Instruct_Text2SQL) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
11
  Refer to the [original model card](https://huggingface.co/XeAI/LLaMa_3.2_3B_Instruct_Text2SQL) for more details on the model.
12
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  ## Use with llama.cpp
14
  Install llama.cpp through brew (works on Mac and Linux)
15
 
 
10
  This model was converted to GGUF format from [`XeAI/LLaMa_3.2_3B_Instruct_Text2SQL`](https://huggingface.co/XeAI/LLaMa_3.2_3B_Instruct_Text2SQL) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
11
  Refer to the [original model card](https://huggingface.co/XeAI/LLaMa_3.2_3B_Instruct_Text2SQL) for more details on the model.
12
 
13
+ ## Use with llama-cpp-python
14
+
15
+ ```python
16
+ from llama_cpp import Llama
17
+
18
+ # Load the model
19
+ model = Llama(
20
+ model_path="path_to_your_model.gguf",
21
+ n_ctx=2048,
22
+ n_batch=512,
23
+ n_threads=6
24
+ )
25
+
26
+ # Generate text
27
+ output = model.create_completion(
28
+ "Your prompt here",
29
+ max_tokens=512,
30
+ temperature=0.7,
31
+ top_p=0.95,
32
+ top_k=40,
33
+ repeat_penalty=1.1
34
+ )
35
+ print(output['choices'][0]['text'])
36
+ ```
37
+
38
  ## Use with llama.cpp
39
  Install llama.cpp through brew (works on Mac and Linux)
40