File size: 5,146 Bytes
bb678d9 34bd0e5 bb678d9 34bd0e5 bb678d9 34bd0e5 bb678d9 34bd0e5 bb678d9 34bd0e5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | ---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-7B-Instruct
tags:
- text-to-sql
- sql
- qlora
- lora
- fine-tuned
- qwen2
language:
- en
pipeline_tag: text-generation
---
# fg-sql-merged
A fine-tuned text-to-SQL model for the `fg_organization` HR/ERP database schema.
Given a natural-language question, it generates the corresponding MySQL query.
This repo is the **merged** model (base weights + LoRA adapter folded together,
full fp16 precision) — for the standalone adapter alone, see
[`yoonboom/fg-sql-lora-adapter`](https://huggingface.co/yoonboom/fg-sql-lora-adapter).
## Model Details
- **Base model:** [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)
- **Fine-tuning method:** QLoRA (4-bit base + LoRA adapter during training, merged
back to fp16 for this repo)
- **LoRA config:** r=16, alpha=32, dropout=0.05, target modules
`q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`
- **Precision:** fp16 (merged); a `q8_0` GGUF quantization is also available
for llama.cpp/Ollama serving at
[`yoonboom/fg-sql-gguf`](https://huggingface.co/yoonboom/fg-sql-gguf)
- **Language:** English (natural-language input), MySQL 8.0 dialect (SQL output)
## Intended Use
Translate natural-language questions about a fictional company's HR/ERP data
into MySQL queries against the `fg_organization` schema (18 tables covering
organization structure, workforce, projects/tasks, and finance). Intended to
run behind a guardrail layer that validates generated SQL against the real
schema before execution (see `sql_schema_guard.py` in the companion repo) —
**this model should not be trusted to execute SQL directly against a
production database without that validation step.**
### Out of Scope
- Any other database schema — this model is specifically tuned to
`fg_organization`'s tables/columns/ID conventions and will not generalize
to arbitrary SQL schemas.
- Recruitment/candidate data, IT asset tracking, employee training records,
and real external client identity — the underlying schema has no tables
for any of these; the model is trained to decline these rather than
fabricate a table.
- True cost/profit/margin calculations — the schema has no cost/rate column
anywhere, only revenue and hours logged as separate figures.
## Training Data
- 400 hand-written/generated question→SQL pairs (320 train / 40 val / 40 test),
stratified across ~10-11 categories: simple lookups, joins, aggregations,
date/time filtering, manager-hierarchy recursion, and mixed-complexity
queries (CTEs, EXISTS, correlated subqueries).
- Schema: `fg_organization`, 18 tables, MySQL 8.0 (`WITH RECURSIVE` support
required for hierarchy queries).
## Training Procedure
- **Hardware:** Google Colab free-tier, Tesla T4 GPU (16GB VRAM, fp16 only —
no bf16 support)
- **Method:** QLoRA — 4-bit quantized base (`bitsandbytes`, nf4,
double-quant) + LoRA adapter (`peft`), trained via `transformers.Trainer`
- **Hyperparameters:** 3 epochs, effective batch size 16 (per-device batch 1
× grad accumulation 16), learning rate 2e-4, cosine schedule, 3% warmup,
`paged_adamw_8bit` optimizer
- **Loss masking:** assistant-turn-only (system + user prompt tokens masked
to -100, loss computed only on the generated SQL)
### Training Results
| Step | Train loss | Val loss |
|---|---|---|
| 10 | 0.369 | 0.227 |
| 20 | 0.195 | 0.155 |
| 30 | 0.102 | 0.124 |
| 40 | 0.075 | 0.113 |
| 50 | 0.065 | 0.110 |
| 60 | 0.048 | 0.109 |
Val loss plateaus over the final 20 steps while train loss keeps falling —
3 epochs was about the right stopping point for this dataset size.
## Evaluation
- **Exact-match on held-out test set (40 examples):** ~30% strict
string-match; substantially higher (~68%) when accounting for
functionally-equivalent-but-differently-styled SQL (renamed aliases,
different but valid `ORDER BY` choices, extra helpful columns).
- **Known weak categories** (as of this checkpoint): resolving named
roles/titles without hardcoded lookups, descendant-direction recursive
hierarchy questions ("everyone under X" vs. "who X reports to"), and a
tendency toward schema hallucination under harder aggregation/CTE
questions (invented column/ID names that don't exist in the real schema).
- A static schema-fidelity checker (regex/alias-aware, no DB required) is
used alongside this model in production to catch hallucinated
tables/columns/IDs before execution.
## Limitations
- Trained on a relatively small dataset (400 pairs) for a 7B model — some
brittleness on question phrasings not well-represented in training data
should be expected, particularly for hierarchy-direction and role/title
resolution questions (see Evaluation above).
- This model can still generate SQL referencing non-existent
columns/tables/IDs when uncertain, rather than reliably declining —
always validate output against the real schema before execution.
- Specific to the `fg_organization` schema; not a general-purpose
text-to-SQL model.
## License
Apache 2.0 (inherited from the Qwen2.5-Coder-7B-Instruct base model).
|