---
license: cc-by-sa-4.0
language:
  - en
library_name: gguf
pipeline_tag: text-generation
base_model: defog/llama-3-sqlcoder-8b
tags:
  - text-to-sql
  - sql
  - gguf
  - llama-3
  - quantized
  - llama-cpp
  - code-generation
---

# llama-3-sqlcoder-8b — GGUF

GGUF (llama.cpp) build of [`defog/llama-3-sqlcoder-8b`](https://huggingface.co/defog/llama-3-sqlcoder-8b),
a Llama-3 8B model fine-tuned for **text-to-SQL** generation. This repo packages a
**Q4_K_M** quantization so the model runs efficiently on CPU/GPU through
[llama.cpp](https://github.com/ggerganov/llama.cpp), Ollama, LM Studio, and other
GGUF-compatible runtimes.

## Files

| File | Quant | Approx. size | Notes |
|------|-------|--------------|-------|
| `llama-3-sqlcoder-8b.Q4_K_M.gguf` | Q4_K_M | ~4.9 GB | 4-bit, good quality/size trade-off |

## Usage

**llama.cpp**

```bash
./llama-cli -m llama-3-sqlcoder-8b.Q4_K_M.gguf -p "Generate a SQL query to answer the question."
```

**Ollama (Modelfile)**

```
FROM ./llama-3-sqlcoder-8b.Q4_K_M.gguf
```

**Python (llama-cpp-python)**

```python
from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="omeryentur/llama-3-sqlcoder-8b-GGUF",
    filename="llama-3-sqlcoder-8b.Q4_K_M.gguf",
)
out = llm("### Task\nGenerate a SQL query to answer the question.\n### Question\nHow many users signed up in 2024?\n### SQL\n")
print(out["choices"][0]["text"])
```

Use the prompt format expected by the base `defog/llama-3-sqlcoder-8b` model for best results.

## Credits

- Base model: [defog/llama-3-sqlcoder-8b](https://huggingface.co/defog/llama-3-sqlcoder-8b)
- Quantization tooling: [llama.cpp](https://github.com/ggerganov/llama.cpp)

See more text-to-SQL work on this profile, including the
[Text-to-PostgreSQL dataset](https://huggingface.co/datasets/omeryentur/text-to-postgresql).