README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
tags:
  - text-to-sql
  - nl2sql
  - sql
  - qlora
  - gguf
  - llama-cpp
  - edge
  - on-prem
language:
  - en
library_name: gguf
pipeline_tag: text-generation
datasets:
  - xlangai/spider
---

# TinySQL-1.5B

**A private, on-prem Natural-Language-to-SQL model that runs on a laptop CPU.**

TinySQL is [Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct)
fine-tuned with QLoRA for text-to-SQL and quantized to 4-bit GGUF (Q4_K_M) so it
runs offline via `llama.cpp` — no GPU, no cloud, no per-query cost. It converts an
English question + a database schema into a **validated, read-only SQL SELECT**.

The point is **not** to beat frontier models on raw accuracy. It is to be the
*compliant, $0/query, offline* option for regulated data (finance, health, legal,
government) where the schema + data **cannot** leave the premises.

---

## Measured results

All numbers below are **measured**, not estimated. Evaluation is **execution
accuracy** on the full **Spider dev split (1,034 examples)**: run the predicted SQL
and the gold SQL against the real SQLite database and compare returned rows
(order-insensitive), over a read-only connection.

### Fine-tuning lift (apples-to-apples)

Identical base model, identical Q4_K_M quantization, identical prompt template and
SELECT-only guardrail — **only the QLoRA adapter differs**.

| Model | Spider dev exec. acc | Valid (SELECT-only + executes) | Malformed outputs |
|-------|----------------------|--------------------------------|-------------------|
| Base Qwen2.5-Coder-1.5B | 50.87% | 76.9% | 54 |
| **TinySQL-1.5B** | **62.86%** | **87.99%** | **0** |

- **+11.99 points** execution accuracy from fine-tuning.
- **54 → 0** malformed outputs: the base model emitted non-SQL chatter and
  degenerate repetition loops; the fine-tune produces clean, parseable SELECTs.

### Performance (laptop CPU, Intel Core Ultra 5 235U)

| Metric | Value |
|--------|-------|
| Mean latency | 0.57 s / query |
| p95 latency | 0.75 s |
| Peak RAM | ~1.73 GB |
| Cost | $0 / query (self-hosted) |

> Larger models and cloud APIs achieve higher accuracy, but require GPU/cloud and
> send your schema + data off-premises. TinySQL trades peak accuracy for privacy,
> $0 cost, and offline operation — the axes that matter for regulated data.

---

## Intended use

- Private/on-prem "text-to-SQL copilot" for non-technical users to query a database
  in plain English.
- Embedded/offline analytics (edge devices, desktop apps) with no network.
- A cheap first-pass layer that handles routine queries locally, escalating only
  hard ones to a larger model.

**Read-only by design.** Generated SQL is validated to be a single SELECT before it
is shown or executed. It cannot INSERT/UPDATE/DELETE/DROP.

## Out of scope / limitations

- **Not for autonomous critical decisions** — ~63% accuracy means a human should
  verify before acting on results.
- **No writes** — SELECT-only.
- **Schema size** — trained/served at 2048-token context; very large schemas are
  pruned and accuracy drops.
- **SQLite dialect** — targets SQLite SQL.

---

## How to use

### With `llama-cpp-python`

```python
from llama_cpp import Llama

llm = Llama(model_path="tinysql-1.5b-q4_k_m.gguf", n_ctx=2048, verbose=False)

INSTRUCTION = ("You are a SQL expert. Given the database schema, write a single "
               "SQLite SELECT query that answers the question. Return ONLY the SQL.")

schema = """CREATE TABLE orders (
  id INTEGER PRIMARY KEY,
  status TEXT,
  customer_id INTEGER
);"""
question = "How many orders are completed?"

prompt = (f"### Instruction:\n{INSTRUCTION}\n\n"
          f"### Schema:\n{schema}\n\n"
          f"### Question:\n{question}\n\n"
          f"### SQL:\n")

out = llm(prompt, max_tokens=256, temperature=0.0, stop=["###"])
print(out["choices"][0]["text"].strip())
# -> SELECT count(*) FROM orders WHERE status = 'completed';
```

### With `llama.cpp` CLI

```bash
llama-cli -m tinysql-1.5b-q4_k_m.gguf -p "### Instruction:..." -n 256 --temp 0
```

**Always enforce SELECT-only + run against a read-only DB connection** before
executing generated SQL. Do not run model output with write permissions.

---

## Prompt format

The model was trained with (and expects) this exact template:

```
### Instruction:
You are a SQL expert. Given the database schema, write a single SQLite
SELECT query that answers the question. Return ONLY the SQL.

### Schema:
{CREATE TABLE statements}

### Question:
{natural-language question}

### Evidence:        # optional external-knowledge hint (BIRD-style)
{hint}

### SQL:
```

---

## Training

- **Base:** Qwen/Qwen2.5-Coder-1.5B-Instruct
- **Method:** QLoRA (4-bit), LoRA r=16, α=16
- **Data:** Spider + BIRD, ~15.4k instruction examples, SELECT-only, FK-aware schema
  pruning to fit 2048 tokens
- **Shipped checkpoint:** 500 steps (~0.26 epoch). A training-length ablation found
  accuracy peaks early: 500 steps = 62.86%, 1000 = 61.22%, 1800 = 40.81% (overfit).
  **Less was more.**
- **Export:** merged to 16-bit → GGUF → quantized Q4_K_M for CPU/edge.

---

## Datasets & attribution

- **Spider** (Yu et al., 2018) — CC BY-SA 4.0
- **BIRD** (Li et al., 2023) — CC BY-SA 4.0

Base model **Qwen2.5-Coder-1.5B-Instruct** is Apache-2.0. This fine-tune is released
under **Apache-2.0**; please also honor the CC BY-SA 4.0 attribution for Spider/BIRD.

## Citation

```bibtex
@misc{tinysql2026,
  title  = {TinySQL: Private On-Prem NL-to-SQL on a Laptop CPU},
  author = {Indirakumar},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Indirakumar01/tinysql-1.5b}},
  note   = {Fine-tuned Qwen2.5-Coder-1.5B, GGUF Q4_K_M}
}
```