| --- |
| datasets: |
| - Shaleen123/text2sql-v1 |
| language: |
| - en |
| base_model: |
| - google/gemma-4-12B-it |
| pipeline_tag: text-generation |
| library_name: transformers |
| --- |
| |
| **A Gemma 4 12B model fine-tuned for text-to-SQL: turn natural-language questions into correct, executable SQL given a database schema.** |
| |
| --- |
| |
| ## Model Summary |
| |
| | | | |
| |---|---| |
| | **Model** | `Shaleen123/gemma4-12b-sql` | |
| | **Base model** | Gemma 4 12B | |
| | **Task** | Text-to-SQL / schema-grounded query generation | |
| | **Parameters** | ~12B | |
| | **Language** | English (natural-language questions), SQL (output) | |
| |
| This model takes a database schema (DDL) and a question, and returns a SQL query. It is intended to help analysts, developers, and applications that need to query relational data without hand-writing SQL. |
| |
| ## Intended Use |
| |
| **Good fits** |
| - Natural-language interfaces over relational databases (BI tools, chat-with-your-data apps) |
| - SQL autocomplete and drafting assistants for analysts and engineers |
| - Generating a first draft of a query that a human then reviews |
| - Research and benchmarking on text-to-SQL |
|
|
| ## How to Use |
| |
| ### Prompt format |
| |
| The model was trained with the following system prompt. Use it as-is for best results: |
| |
| ``` |
| You are the best SQL Agentic Model. Given a database schema and a natural language question, generate the correct SQL query. Return only the SQL query without explanations. |
| ``` |
| |
| Then provide the schema first, followed by the question, in the user turn. (TODO: confirm the user-turn template matches what was used in training.) |
| |
| ``` |
| ### Schema |
| CREATE TABLE customers ( |
| id INT PRIMARY KEY, |
| name TEXT, |
| country TEXT, |
| signup_date DATE |
| ); |
| CREATE TABLE orders ( |
| id INT PRIMARY KEY, |
| customer_id INT REFERENCES customers(id), |
| total NUMERIC, |
| created_at TIMESTAMP |
| ); |
| |
| ### Question |
| What are the top 5 countries by total order value in 2024? |
| |
| ### SQL |
| ``` |
| |
| ### Transformers |
| |
| ```python |
| import torch |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| |
| model_id = "Shaleen123/gemma4-12b-sql" |
| |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained( |
| model_id, |
| torch_dtype=torch.bfloat16, |
| device_map="auto", |
| ) |
| |
| schema = """ |
| CREATE TABLE customers (id INT PRIMARY KEY, name TEXT, country TEXT, signup_date DATE); |
| CREATE TABLE orders (id INT PRIMARY KEY, customer_id INT REFERENCES customers(id), |
| total NUMERIC, created_at TIMESTAMP); |
| """ |
| question = "What are the top 5 countries by total order value in 2024?" |
| |
| SYSTEM_PROMPT = ( |
| "You are the best SQL Agentic Model. Given a database schema and a natural language " |
| "question, generate the correct SQL query. Return only the SQL query without explanations." |
| ) |
| |
| messages = [ |
| {"role": "system", "content": SYSTEM_PROMPT}, |
| { |
| "role": "user", |
| "content": f"### Question\n{question}\n### Scheme\n{scheme}", |
| }, |
| ] |
| # If your chat template does not support a separate system role, |
| # prepend SYSTEM_PROMPT to the user message instead. |
| |
| inputs = tokenizer.apply_chat_template( |
| messages, add_generation_prompt=True, return_tensors="pt", return_dict=True |
| ).to(model.device) |
| |
| with torch.no_grad(): |
| output = model.generate(**inputs, max_new_tokens=256, do_sample=False) |
| |
| print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)) |
| ``` |
| |
| ### Recommended generation settings |
| |
| - `temperature = 1.0` and 'top_p = 0.95' for deterministic, reproducible SQL |
| - `max_new_tokens` of 10,000 covers most queries |
| - Stop on a semicolon or end-of-turn token to avoid trailing explanations |
| ### vLLM |
| |
| ```bash |
| vllm serve Shaleen123/gemma-4-12B-sql --dtype bfloat16 --max-model-len 8192 |
| ``` |
| |
| ```python |
| from openai import OpenAI |
| |
| client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") |
| |
| resp = client.chat.completions.create( |
| model="Shaleen123/gemma-4-12B-sql", |
| temperature=0, |
| messages=[ |
| {"role": "system", "content": "You are the best SQL Agentic Model. Given a database schema and a natural language question, generate the correct SQL query. Return only the SQL query without explanations."}, |
| {"role": "user", "content": "### Schema\nCREATE TABLE users (id INT, name TEXT, age INT);\n### Question\nHow many users are older than 30?"}, |
| ], |
| ) |
| print(resp.choices[0].message.content) |
| ``` |
| |
| ### Hardware notes |
| |
| A 12B model in bf16 needs roughly 24–28 GB of VRAM for weights alone. For smaller GPUs, use 8-bit or 4-bit quantization (e.g. bitsandbytes, AWQ, or GGUF). Expect a small accuracy drop at lower precision. (TODO: report measured numbers if you have them.) |
| |
| ## Training Details |
| |
| > TODO: fill in everything below with your actual setup. |
| |
| **Data** |
| - Sources: TODO (e.g. Spider, BIRD, WikiSQL, synthetic data, internal data) |
| - Size: TODO examples |
| - Preprocessing: TODO (dedup, schema formatting, dialect filtering, execution-based filtering) |
| - Dialects covered: TODO (e.g. SQLite, PostgreSQL, MySQL, BigQuery) |
| **Procedure** |
| - Method: TODO (SFT with LoRA r=?, alpha=?, target modules=?) |
| - Epochs / steps: TODO |
| - Learning rate & scheduler: TODO |
| - Batch size (effective): TODO |
| - Max sequence length: TODO |
| - Precision: TODO (bf16) |
| - Hardware & training time: TODO |
| - Frameworks: TODO (e.g. TRL, Unsloth, Axolotl, PEFT) |
| ## Evaluation |
| |
| > TODO: add real results. Suggested benchmarks and metrics below. |
| |
| | Benchmark | Metric | Base Gemma 4 12B | This model | |
| |---|---|---|---| |
| | Spider (dev) | Execution accuracy | TODO | TODO | |
| | Spider (dev) | Exact-set-match | TODO | TODO | |
| | BIRD (dev) | Execution accuracy | TODO | TODO | |
| | Custom / internal | TODO | TODO | TODO | |
| |
| Execution accuracy (does the query return the correct result?) is more meaningful than string match, since many different SQL queries are equivalent. Report the prompt format, decoding settings, and whether schema values or hints were provided, so results are reproducible. |
| |
| ## Limitations and Risks |
| |
| - **Hallucinated schema elements.** The model may reference tables or columns that don't exist, especially with large or ambiguous schemas. |
| - **Ambiguity.** Vague questions ("best customers") may be interpreted differently than intended. Clarify metrics and time ranges in the prompt. |
| - **Dialect drift.** Output may mix syntax across SQL dialects. State the target dialect explicitly. |
| - **Complex queries.** Deeply nested subqueries, window functions, recursive CTEs, and multi-hop joins are more error-prone. |
| - **Not safe by construction.** Generated SQL can be destructive or expensive (full scans, cross joins). It can also be vulnerable to prompt injection if user-supplied text is embedded in the prompt. |
| - **Inherited biases and limitations** from the base model and training data. |
| ### Safe deployment checklist |
| |
| 1. Connect with a **read-only** database role. |
| 2. Parse and validate SQL (allow-list `SELECT`; block DDL/DML) before execution. |
| 3. Enforce query timeouts and row limits. |
| 4. Run `EXPLAIN` or a dry run on large datasets. |
| 5. Log queries and keep a human in the loop for anything sensitive. |
| 6. Optionally add execution-feedback retries: on error, pass the error message back to the model for a corrected query. |
| ## License |
| |
| TODO. This model is a derivative of Gemma 4 and is subject to the base model's license and use policy. Review the terms on the base model's page before use or redistribution. Also check the licenses of your training datasets. |
| |
| ## Citation |
| |
| ```bibtex |
| @misc{shaleen2026gemma4sql, |
| author = {Shaleen}, |
| title = {gemma4-12b-sql: A Text-to-SQL Fine-tune of Gemma 4 12B}, |
| year = {2026}, |
| publisher = {Hugging Face}, |
| howpublished = {\url{https://huggingface.co/Shaleen123/gemma-4-12B-sql}} |
| } |
| ``` |
| |
| ## Acknowledgements |
| |
| Built on Gemma by Google. Thanks to the creators of the text-to-SQL datasets and open-source tooling used in training. (TODO: credit specific datasets and libraries.) |
| |
| ## Contact |
| |
| Questions, issues, or feedback: open a discussion on the model's Hugging Face page. (TODO: add other contact info if desired.) |
