Bhuvandesai's picture
Complete CPU GGUF serving + docs + minimal UI redesign
1d8be28
|
Raw
History Blame Contribute Delete
1.24 kB
metadata
title: Phi3 Text to SQL Studio
emoji: 🗄️
colorFrom: indigo
colorTo: purple
sdk: docker
pinned: false

Phi-3 Text-to-SQL Studio

A fine-tuned Phi-3-mini-4k-instruct model (QLoRA LoRA adapter) for natural-language → SQL, served on CPU via llama.cpp + a 4-bit Q4_K_M GGUF, behind a Flask web UI with a schema sidebar, generated SQL, and live SQLite execution.

Highlights

  • Trained only 0.12% of params (4.46M, a 9 MB adapter) with QLoRA in ~3 min on a 6 GB laptop GPU.
  • Held-out execution accuracy 75% (vs 41.7% for the base model), 100% valid SQL.
  • Q4_K_M GGUF is 68.6% smaller than f16 with no measured task-accuracy loss.

Note: runs on free cpu-basic (2 vCPU, no GPU). First load takes ~1–3 min (downloads the model); each query takes ~30 s–2 min to generate. Submit a question and wait — it completes.

A full write-up (fine-tuning + quantization + deployment deep dive, with all benchmarks) lives in docs/Phi3-Text-to-SQL-Finetuning-Quantization.md.