File size: 2,384 Bytes
3f17d1a 6de629b 3f17d1a 728040d 3f17d1a 6de629b 3f17d1a 728040d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | ---
base_model: google/gemma-3-270m-it
library_name: transformers
model_name: text-to-sql-model
tags:
- generated_from_trainer
- trl
- sft
- peft
- lora
- gemma
licence: license
license: mit
datasets:
- sirunchained/text-to-sql-dataset
language:
- en
pipeline_tag: text-generation
---
# Gemma 270M-IT Text-to-SQL Model (Version 1)
This model is a fine-tuned version of `google/gemma-3-270m-it` specifically adapted for **Text-to-SQL conversion**. It takes natural language questions, optionally along with a database schema, and generates the corresponding SQL query.
## Model Description
- **Base Model:** `google/gemma-3-270m-it`
- **Task:** Text-to-SQL Generation (Instruction-tuned language model)
- **Fine-tuning Framework:** `trl` library with `SFTTrainer`
- **Optimization:** LoRA (Low-Rank Adaptation) for Parameter-Efficient Fine-Tuning (PEFT) and 4-bit quantization using `bitsandbytes` to reduce memory footprint and enable training on resource-constrained environments (e.g., T4 GPU).
## Training Data
The model was fine-tuned on the `sirunchained/text-to-sql-dataset` from Hugging Face. This dataset consists of natural language queries, corresponding database schemas (optional), and the target SQL queries.
## Model Performance
The model's performance was evaluated on a test set, yielding the following results:
- **Evaluation Loss:** `0.681`
- **Mean Token Accuracy:** `0.836`
During training, the model showed a consistent decrease in both training and validation loss, indicating successful learning and generalization. The validation mean token accuracy steadily improved, reaching approximately `84%` by the end of the training process, suggesting that the model is capable of generating accurate SQL queries given natural language input and schema information.
## Version Information
This is **Version 1** of the `sirunchained/text-to-sql-model`. The model will be continuously updated and improved over time with more data and potentially different architectures.
## Gradio Demo
A Gradio web interface was developed to demonstrate the model's capabilities, allowing users to input natural language queries and provide database schema details table by table, then view the generated SQL output. You can find the Gradio app code within the accompanying notebook.
You can test it [here](https://huggingface.co/spaces/sirunchained/text-to-sql-gradio). |