TinySQL-1.5B

A private, on-prem Natural-Language-to-SQL model that runs on a laptop CPU.

TinySQL is Qwen2.5-Coder-1.5B-Instruct fine-tuned with QLoRA for text-to-SQL and quantized to 4-bit GGUF (Q4_K_M) so it runs offline via llama.cpp — no GPU, no cloud, no per-query cost. It converts an English question + a database schema into a validated, read-only SQL SELECT.

The point is not to beat frontier models on raw accuracy. It is to be the compliant, $0/query, offline option for regulated data (finance, health, legal, government) where the schema + data cannot leave the premises.

---

Measured results

All numbers below are measured, not estimated. Evaluation is execution accuracy on the full Spider dev split (1,034 examples): run the predicted SQL and the gold SQL against the real SQLite database and compare returned rows (order-insensitive), over a read-only connection.

Fine-tuning lift (apples-to-apples)

Identical base model, identical Q4_K_M quantization, identical prompt template and SELECT-only guardrail — only the QLoRA adapter differs.

Model	Spider dev exec. acc	Valid (SELECT-only + executes)	Malformed outputs
Base Qwen2.5-Coder-1.5B	50.87%	76.9%	54
TinySQL-1.5B	62.86%	87.99%	0

• +11.99 points execution accuracy from fine-tuning.
• 54 → 0 malformed outputs: the base model emitted non-SQL chatter and degenerate repetition loops; the fine-tune produces clean, parseable SELECTs.

Performance (laptop CPU, Intel Core Ultra 5 235U)

Metric	Value
Mean latency	0.57 s / query
p95 latency	0.75 s
Peak RAM	~1.73 GB
Cost	$0 / query (self-hosted)

Larger models and cloud APIs achieve higher accuracy, but require GPU/cloud and send your schema + data off-premises. TinySQL trades peak accuracy for privacy, $0 cost, and offline operation — the axes that matter for regulated data.

---

Intended use

• Private/on-prem "text-to-SQL copilot" for non-technical users to query a database in plain English.
• Embedded/offline analytics (edge devices, desktop apps) with no network.
• A cheap first-pass layer that handles routine queries locally, escalating only hard ones to a larger model.

Read-only by design. Generated SQL is validated to be a single SELECT before it is shown or executed. It cannot INSERT/UPDATE/DELETE/DROP.

Out of scope / limitations

• Not for autonomous critical decisions — ~63% accuracy means a human should verify before acting on results.
• No writes — SELECT-only.
• Schema size — trained/served at 2048-token context; very large schemas are pruned and accuracy drops.
• SQLite dialect — targets SQLite SQL.

---

How to use

With llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="tinysql-1.5b-q4_k_m.gguf", n_ctx=2048, verbose=False)

INSTRUCTION = ("You are a SQL expert. Given the database schema, write a single "
               "SQLite SELECT query that answers the question. Return ONLY the SQL.")

schema = """CREATE TABLE orders (
  id INTEGER PRIMARY KEY,
  status TEXT,
  customer_id INTEGER
);"""
question = "How many orders are completed?"

prompt = (f"### Instruction:\n{INSTRUCTION}\n\n"
          f"### Schema:\n{schema}\n\n"
          f"### Question:\n{question}\n\n"
          f"### SQL:\n")

out = llm(prompt, max_tokens=256, temperature=0.0, stop=["###"])
print(out["choices"][0]["text"].strip())
# -> SELECT count(*) FROM orders WHERE status = 'completed';

With llama.cpp CLI

llama-cli -m tinysql-1.5b-q4_k_m.gguf -p "### Instruction:..." -n 256 --temp 0

Always enforce SELECT-only + run against a read-only DB connection before executing generated SQL. Do not run model output with write permissions.

---

Prompt format

The model was trained with (and expects) this exact template:

### Instruction:
You are a SQL expert. Given the database schema, write a single SQLite
SELECT query that answers the question. Return ONLY the SQL.

### Schema:
{CREATE TABLE statements}

### Question:
{natural-language question}

### Evidence:        # optional external-knowledge hint (BIRD-style)
{hint}

### SQL:

---

Training

• Base: Qwen/Qwen2.5-Coder-1.5B-Instruct
• Method: QLoRA (4-bit), LoRA r=16, α=16
• Data: Spider + BIRD, ~15.4k instruction examples, SELECT-only, FK-aware schema pruning to fit 2048 tokens
• Shipped checkpoint: 500 steps (~0.26 epoch). A training-length ablation found accuracy peaks early: 500 steps = 62.86%, 1000 = 61.22%, 1800 = 40.81% (overfit). Less was more.
• Export: merged to 16-bit → GGUF → quantized Q4_K_M for CPU/edge.

---

Datasets & attribution

• Spider (Yu et al., 2018) — CC BY-SA 4.0
• BIRD (Li et al., 2023) — CC BY-SA 4.0

Base model Qwen2.5-Coder-1.5B-Instruct is Apache-2.0. This fine-tune is released under Apache-2.0; please also honor the CC BY-SA 4.0 attribution for Spider/BIRD.

Citation

@misc{tinysql2026,
  title  = {TinySQL: Private On-Prem NL-to-SQL on a Laptop CPU},
  author = {Indirakumar},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Indirakumar01/tinysql-1.5b}},
  note   = {Fine-tuned Qwen2.5-Coder-1.5B, GGUF Q4_K_M}
}