Instructions to use A-Kishore/llama-3.2-3b-text2sql with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use A-Kishore/llama-3.2-3b-text2sql with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="A-Kishore/llama-3.2-3b-text2sql")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("A-Kishore/llama-3.2-3b-text2sql")
model = AutoModelForCausalLM.from_pretrained("A-Kishore/llama-3.2-3b-text2sql")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use A-Kishore/llama-3.2-3b-text2sql with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use A-Kishore/llama-3.2-3b-text2sql with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "A-Kishore/llama-3.2-3b-text2sql"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "A-Kishore/llama-3.2-3b-text2sql",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/A-Kishore/llama-3.2-3b-text2sql

SGLang

How to use A-Kishore/llama-3.2-3b-text2sql with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "A-Kishore/llama-3.2-3b-text2sql" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "A-Kishore/llama-3.2-3b-text2sql",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "A-Kishore/llama-3.2-3b-text2sql" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "A-Kishore/llama-3.2-3b-text2sql",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use A-Kishore/llama-3.2-3b-text2sql with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for A-Kishore/llama-3.2-3b-text2sql to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for A-Kishore/llama-3.2-3b-text2sql to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for A-Kishore/llama-3.2-3b-text2sql to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="A-Kishore/llama-3.2-3b-text2sql",
    max_seq_length=2048,
)

Docker Model Runner
How to use A-Kishore/llama-3.2-3b-text2sql with Docker Model Runner:
```
docker model run hf.co/A-Kishore/llama-3.2-3b-text2sql
```

llama-3.2-3b-text2sql / README.md

A-Kishore

Update README.md

1f81940 verified 21 days ago

preview code

Raw

History Blame Contribute Delete

7.56 kB

	---
	base_model: unsloth/Llama-3.2-3B-Instruct-bnb-4bit
	tags:
	- text-generation-inference
	- transformers
	- unsloth
	- llama
	- text-to-sql
	- peft
	- lora
	license: apache-2.0
	language:
	- en
	datasets:
	- gretelai/synthetic_text_to_sql
	---

	# 🔮 Llama-3.2-3B-Instruct Text-to-SQL

	A fine-tuned version of [`unsloth/Llama-3.2-3B-Instruct-bnb-4bit`](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-bnb-4bit) for generating SQL queries from natural language questions and database DDL schemas.

	[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
	[![Finetuned with Unsloth](https://img.shields.io/badge/Finetuned%20with-Unsloth-blue.svg)](https://github.com/unslothai/unsloth)

	---

	## Model Summary

	\| Attribute \| Value \|
	\|:---\|:---\|
	\| Base Model \| `unsloth/Llama-3.2-3B-Instruct-bnb-4bit` \|
	\| Task \| Text-to-SQL (natural language → SQL) \|
	\| Fine-Tuning Method \| LoRA (PEFT) via Unsloth + TRL \|
	\| Training Dataset \| `gretelai/synthetic_text_to_sql` (50k samples) \|
	\| Trainable Parameters \| ~0.75% of base model \|
	\| Export Format \| Merged FP16 (`merged_16bit`) \|
	\| License \| Apache-2.0 + Meta Llama 3 Community License \|
	\| Developer \| A-Kishore \|

	---

	## Evaluation Results

	Evaluated on the first 200 samples of the `gretelai/synthetic_text_to_sql` test split using greedy decoding. ROUGE F-measures reported.

	\| Model \| ROUGE-1 \| ROUGE-2 \| ROUGE-L \|
	\|:---\|:---:\|:---:\|:---:\|
	\| Base Model (`unsloth/Llama-3.2-3B-Instruct-bnb-4bit`) \| 0.2908 \| 0.2016 \| 0.2651 \|
	\| Fine-Tuned (`A-Kishore/llama-3.2-3b-text2sql`) \| 0.8486 \| 0.7232 \| 0.8151 \|
	\| Improvement \| +191.82% \| +258.73% \| +207.47% \|

	Metric interpretation:
	- ROUGE-1 (unigram overlap) reflects accurate retrieval of schema identifiers and SQL keywords.
	- ROUGE-2 (bigram overlap) captures structural alignment of consecutive SQL constructs (e.g. `GROUP BY`, `ORDER BY`).
	- ROUGE-L (longest common subsequence) tracks overall query flow including nested clauses and join ordering.

	> Note: ROUGE measures lexical overlap, not SQL executability. A query may score slightly lower due to stylistic differences (alias names, join ordering) while still being functionally equivalent. See [Limitations](#limitations).

	---

	## How to Use

	The model weights are fully merged in 16-bit precision and load with standard `transformers` or `unsloth`.

	### Prompt Format

	Always use this exact template — the model was trained on it:

	```
	###TASK
	Generate the SQL query to answer the following question

	### Database Schema
	{sql_context}

	### Question
	{sql_prompt}

	### SQL Query
	```

	### (a) Standard `transformers`

	```python
	import torch
	from transformers import AutoTokenizer, AutoModelForCausalLM

	model_name = "A-Kishore/llama-3.2-3b-text2sql"

	tokenizer = AutoTokenizer.from_pretrained(model_name)
	model = AutoModelForCausalLM.from_pretrained(
	model_name,
	torch_dtype=torch.float16,
	device_map="auto"
	)

	prompt = """###TASK
	Generate the SQL query to answer the following question

	### Database Schema
	{sql_context}

	### Question
	{sql_prompt}

	### SQL Query
	"""

	sql_context = "CREATE TABLE employees (id INT, name TEXT, department TEXT, salary REAL);"
	sql_prompt = "What is the average salary per department?"

	inputs = tokenizer(
	prompt.format(sql_context=sql_context, sql_prompt=sql_prompt),
	return_tensors="pt"
	).to("cuda")

	outputs = model.generate(
	**inputs,
	max_new_tokens=150,
	use_cache=True,
	pad_token_id=tokenizer.eos_token_id
	)

	result = tokenizer.decode(outputs[0], skip_special_tokens=True)
	sql = result.split("### SQL Query")[-1].strip()
	print(sql)
	# SELECT department, AVG(salary) FROM employees GROUP BY department;
	```

	### (b) Unsloth Fast Inference

	```python
	import torch
	from unsloth import FastLanguageModel

	model, tokenizer = FastLanguageModel.from_pretrained(
	model_name="A-Kishore/llama-3.2-3b-text2sql",
	max_seq_length=768,
	dtype=torch.float16,
	load_in_4bit=False,
	)
	FastLanguageModel.for_inference(model)

	prompt = """###TASK
	Generate the SQL query to answer the following question

	### Database Schema
	{sql_context}

	### Question
	{sql_prompt}

	### SQL Query
	"""

	sql_context = "CREATE TABLE employees (id INT, name TEXT, department TEXT, salary REAL);"
	sql_prompt = "What is the average salary per department?"

	inputs = tokenizer(
	prompt.format(sql_context=sql_context, sql_prompt=sql_prompt),
	return_tensors="pt"
	).to("cuda")

	outputs = model.generate(
	**inputs,
	max_new_tokens=150,
	temperature=None,
	do_sample=False,
	pad_token_id=tokenizer.eos_token_id
	)

	result = tokenizer.decode(outputs[0], skip_special_tokens=True)
	sql = result.split("### SQL Query")[-1].strip()
	print(sql)
	```

	---

	## Training Details

	### Dataset

	- Source: [`gretelai/synthetic_text_to_sql`](https://huggingface.co/datasets/gretelai/synthetic_text_to_sql)
	- Training split: 50,000 shuffled samples from `train`
	- Evaluation split: First 200 samples from `test`

	### LoRA Configuration

	\| Parameter \| Value \|
	\|:---\|:---\|
	\| Rank (`r`) \| `16` \|
	\| Alpha (`lora_alpha`) \| `16` \|
	\| Target Modules \| `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj` \|
	\| Bias \| `none` \|

	LoRA freezes the base model weights and injects trainable rank-decomposition matrices into all attention and MLP projections. Only ~0.75% of parameters are updated, dramatically reducing VRAM usage and preventing catastrophic forgetting.

	### Hyperparameters

	\| Parameter \| Value \|
	\|:---\|:---\|
	\| Optimizer \| `paged_adamw_8bit` \|
	\| Learning Rate \| `2e-4` \|
	\| LR Scheduler \| `linear` \|
	\| Warmup Steps \| `5` \|
	\| Epochs \| `1` \|
	\| Per-Device Batch Size \| `8` \|
	\| Gradient Accumulation \| `1` \|
	\| Max Sequence Length \| `768` \|
	\| Sequence Packing \| `True` \|
	\| Mixed Precision \| `fp16` \|
	\| Experiment Tracking \| Weights & Biases \|

	Training was accelerated using the `unsloth` library, which provides optimized GPU kernels for 4-bit quantized training (~2× faster than standard configurations).

	---

	## Repository

	Training and evaluation code: [a-kishore-dev/llama-text2sql-finetune](https://github.com/a-kishore-dev)

	Notebooks included:
	- `Text_to_SQL_Finetuning.ipynb` — dataset prep, LoRA config, training, export
	- `evaluate_model.ipynb` — ROUGE evaluation comparing base vs fine-tuned

	---

	## Limitations

	- SQL executability: ROUGE is a lexical proxy. High ROUGE does not guarantee a query will execute or return logically correct results. A query with different aliases or reordered joins may score lower despite being equivalent.
	- Out-of-distribution schemas: Performance degrades on high-cardinality databases, deeply nested subqueries, or DDL patterns that diverge significantly from the training distribution.
	- Single epoch: The model was trained for one epoch on 50k samples. Further training may improve generalization.

	---

	## License

	The model adapter is released under Apache 2.0. The underlying base model is governed by the [Meta Llama 3 Community License Agreement](https://llama.meta.com/llama3/license/). Users must comply with both.

	---

	## Acknowledgements

	- Unsloth — optimized kernels for 4-bit training and sequence packing
	- Hugging Face — `trl` (`SFTTrainer`) and `transformers`
	- Meta AI — Llama 3.2 open weights
	- Gretel AI — synthetic Text-to-SQL dataset

	---

	## Author

	A-Kishore · [GitHub](https://github.com/a-kishore-dev) · [HuggingFace](https://huggingface.co/A-Kishore)