---
license: apache-2.0
base_model: junmingg/qwen2.5-coder-7b-text2sql
datasets:
  - b-mc2/sql-create-context
pipeline_tag: text-generation
tags:
  - text-to-sql
  - gguf
  - llama.cpp
  - ollama
  - qwen2.5
---

# Qwen2.5-Coder-7B Text-to-SQL — GGUF quants

GGUF quantizations of [`junmingg/qwen2.5-coder-7b-text2sql`](https://huggingface.co/junmingg/qwen2.5-coder-7b-text2sql)
for CPU/GPU inference via [llama.cpp](https://github.com/ggerganov/llama.cpp), [Ollama](https://ollama.com),
LM Studio, etc.

| File | Quant | Size | Notes |
|---|---|---|---|
| `qwen2.5-coder-7b-instruct.Q4_K_M.gguf` | Q4_K_M | ~4.7 GB | 4-bit, best size/quality balance (recommended) |
| `qwen2.5-coder-7b-instruct.Q6_K.gguf` | Q6_K | ~6.3 GB | 6-bit, near-Q8 quality, smaller |
| `qwen2.5-coder-7b-instruct.Q8_0.gguf` | Q8_0 | ~8.1 GB | 8-bit, near-lossless |

See the [main model card](https://huggingface.co/junmingg/qwen2.5-coder-7b-text2sql) for results
(exact 78.8% / semantic 86.2% / validity 99.2% vs base 3.8 / 67.0 / 100), training details, and the
**required system prompt** — the model expects the text-to-SQL system message + ChatML format.

## Quick start (Ollama)

```bash
ollama run hf.co/junmingg/qwen2.5-coder-7b-text2sql-GGUF:Q4_K_M
```

## Quick start (llama.cpp)

```bash
llama-cli -hf junmingg/qwen2.5-coder-7b-text2sql-GGUF:Q4_K_M
```

License: Apache-2.0 (base model) / data CC-BY-4.0.