Update README.md
Browse filesUpdated description
README.md
CHANGED
|
@@ -3,39 +3,34 @@ license: apache-2.0
|
|
| 3 |
base_model: syntheticlab/fix-json
|
| 4 |
tags:
|
| 5 |
- llama-cpp
|
| 6 |
-
- gguf
|
|
|
|
|
|
|
| 7 |
---
|
| 8 |
|
| 9 |
-
# fix-json
|
| 10 |
|
| 11 |
-
**Model creator:** [syntheticlab](https://huggingface.co/syntheticlab)
|
| 12 |
-
**
|
| 13 |
-
**GGUF quantization:**
|
| 14 |
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
## Use with Ollama
|
| 20 |
-
|
| 21 |
-
```bash
|
| 22 |
-
ollama run "hf.co/CronoBJS/fix-json-GGUF:<quantization>"
|
| 23 |
-
```
|
| 24 |
|
| 25 |
-
|
| 26 |
|
| 27 |
-
|
| 28 |
-
lms load "CronoBJS/fix-json-GGUF"
|
| 29 |
-
```
|
| 30 |
|
| 31 |
-
|
| 32 |
|
| 33 |
-
|
| 34 |
-
llama-cli --hf-repo "CronoBJS/fix-json-GGUF" --hf-file "fix-json-Q8_0.gguf" -p "The meaning to life and the universe is"
|
| 35 |
-
```
|
| 36 |
|
| 37 |
-
##
|
| 38 |
|
|
|
|
| 39 |
```bash
|
| 40 |
-
llama-
|
| 41 |
-
|
|
|
|
|
|
| 3 |
base_model: syntheticlab/fix-json
|
| 4 |
tags:
|
| 5 |
- llama-cpp
|
| 6 |
+
- gguf
|
| 7 |
+
- lora
|
| 8 |
+
- merged-model
|
| 9 |
---
|
| 10 |
|
| 11 |
+
# fix-json (LoRA Merged GGUF)
|
| 12 |
|
| 13 |
+
**Model creator:** [syntheticlab](https://huggingface.co/syntheticlab)
|
| 14 |
+
**Base model required:** [syntheticlab/fix-json](https://huggingface.co/syntheticlab/fix-json) (Meta-Llama-3.1-8B-Instruct)
|
| 15 |
+
**GGUF conversion & quantization:** by [CronoBJS](https://huggingface.co/CronoBJS) using [`llama.cpp`](https://github.com/ggerganov/llama.cpp)
|
| 16 |
|
| 17 |
+
⚠️ **Important:**
|
| 18 |
+
This file is a **merged LoRA adapter in GGUF format**. It is **not** a standalone model.
|
| 19 |
+
You **must** also have the original base GGUF model (e.g., `Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf`) to run it.
|
| 20 |
+
Use the `--lora` or `--lora-scaled` flags in `llama.cpp` to apply it at runtime.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
+
---
|
| 23 |
|
| 24 |
+
## Special thanks
|
|
|
|
|
|
|
| 25 |
|
| 26 |
+
🙏 Thanks to [Georgi Gerganov](https://github.com/ggerganov) and the [llama.cpp team](https://github.com/ggerganov/llama.cpp) for making all of this possible.
|
| 27 |
|
| 28 |
+
---
|
|
|
|
|
|
|
| 29 |
|
| 30 |
+
## Running the Model
|
| 31 |
|
| 32 |
+
### 1️⃣ llama.cpp CLI
|
| 33 |
```bash
|
| 34 |
+
llama-cli -m "path/to/Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf" \
|
| 35 |
+
--lora "path/to/syntheticlab-fix-json-lora.gguf" \
|
| 36 |
+
-c 2048 -n 256 -p "Your prompt here"
|