Update README.md
Browse files
README.md
CHANGED
|
@@ -10,6 +10,31 @@ base_model: XeAI/LLaMa_3.2_3B_Instruct_Text2SQL
|
|
| 10 |
This model was converted to GGUF format from [`XeAI/LLaMa_3.2_3B_Instruct_Text2SQL`](https://huggingface.co/XeAI/LLaMa_3.2_3B_Instruct_Text2SQL) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
|
| 11 |
Refer to the [original model card](https://huggingface.co/XeAI/LLaMa_3.2_3B_Instruct_Text2SQL) for more details on the model.
|
| 12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
## Use with llama.cpp
|
| 14 |
Install llama.cpp through brew (works on Mac and Linux)
|
| 15 |
|
|
|
|
| 10 |
This model was converted to GGUF format from [`XeAI/LLaMa_3.2_3B_Instruct_Text2SQL`](https://huggingface.co/XeAI/LLaMa_3.2_3B_Instruct_Text2SQL) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
|
| 11 |
Refer to the [original model card](https://huggingface.co/XeAI/LLaMa_3.2_3B_Instruct_Text2SQL) for more details on the model.
|
| 12 |
|
| 13 |
+
## Use with llama-cpp-python
|
| 14 |
+
|
| 15 |
+
```python
|
| 16 |
+
from llama_cpp import Llama
|
| 17 |
+
|
| 18 |
+
# Load the model
|
| 19 |
+
model = Llama(
|
| 20 |
+
model_path="path_to_your_model.gguf",
|
| 21 |
+
n_ctx=2048,
|
| 22 |
+
n_batch=512,
|
| 23 |
+
n_threads=6
|
| 24 |
+
)
|
| 25 |
+
|
| 26 |
+
# Generate text
|
| 27 |
+
output = model.create_completion(
|
| 28 |
+
"Your prompt here",
|
| 29 |
+
max_tokens=512,
|
| 30 |
+
temperature=0.7,
|
| 31 |
+
top_p=0.95,
|
| 32 |
+
top_k=40,
|
| 33 |
+
repeat_penalty=1.1
|
| 34 |
+
)
|
| 35 |
+
print(output['choices'][0]['text'])
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
## Use with llama.cpp
|
| 39 |
Install llama.cpp through brew (works on Mac and Linux)
|
| 40 |
|