Instructions to use A-Kishore/llama-3.2-3b-text2sql with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use A-Kishore/llama-3.2-3b-text2sql with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="A-Kishore/llama-3.2-3b-text2sql")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("A-Kishore/llama-3.2-3b-text2sql")
model = AutoModelForCausalLM.from_pretrained("A-Kishore/llama-3.2-3b-text2sql")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use A-Kishore/llama-3.2-3b-text2sql with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use A-Kishore/llama-3.2-3b-text2sql with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "A-Kishore/llama-3.2-3b-text2sql"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "A-Kishore/llama-3.2-3b-text2sql",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/A-Kishore/llama-3.2-3b-text2sql

SGLang

How to use A-Kishore/llama-3.2-3b-text2sql with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "A-Kishore/llama-3.2-3b-text2sql" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "A-Kishore/llama-3.2-3b-text2sql",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "A-Kishore/llama-3.2-3b-text2sql" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "A-Kishore/llama-3.2-3b-text2sql",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use A-Kishore/llama-3.2-3b-text2sql with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for A-Kishore/llama-3.2-3b-text2sql to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for A-Kishore/llama-3.2-3b-text2sql to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for A-Kishore/llama-3.2-3b-text2sql to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="A-Kishore/llama-3.2-3b-text2sql",
    max_seq_length=2048,
)

Docker Model Runner
How to use A-Kishore/llama-3.2-3b-text2sql with Docker Model Runner:
```
docker model run hf.co/A-Kishore/llama-3.2-3b-text2sql
```

🔮 Llama-3.2-3B-Instruct Text-to-SQL

A fine-tuned version of unsloth/Llama-3.2-3B-Instruct-bnb-4bit for generating SQL queries from natural language questions and database DDL schemas.

Model Summary

Attribute	Value
Base Model	`unsloth/Llama-3.2-3B-Instruct-bnb-4bit`
Task	Text-to-SQL (natural language → SQL)
Fine-Tuning Method	LoRA (PEFT) via Unsloth + TRL
Training Dataset	`gretelai/synthetic_text_to_sql` (50k samples)
Trainable Parameters	~0.75% of base model
Export Format	Merged FP16 (`merged_16bit`)
License	Apache-2.0 + Meta Llama 3 Community License
Developer	A-Kishore

Evaluation Results

Evaluated on the first 200 samples of the gretelai/synthetic_text_to_sql test split using greedy decoding. ROUGE F-measures reported.

Model	ROUGE-1	ROUGE-2	ROUGE-L
Base Model (`unsloth/Llama-3.2-3B-Instruct-bnb-4bit`)	0.2908	0.2016	0.2651
Fine-Tuned (`A-Kishore/llama-3.2-3b-text2sql`)	0.8486	0.7232	0.8151
Improvement	+191.82%	+258.73%	+207.47%

Metric interpretation:

ROUGE-1 (unigram overlap) reflects accurate retrieval of schema identifiers and SQL keywords.
ROUGE-2 (bigram overlap) captures structural alignment of consecutive SQL constructs (e.g. GROUP BY, ORDER BY).
ROUGE-L (longest common subsequence) tracks overall query flow including nested clauses and join ordering.

Note: ROUGE measures lexical overlap, not SQL executability. A query may score slightly lower due to stylistic differences (alias names, join ordering) while still being functionally equivalent. See Limitations.

How to Use

The model weights are fully merged in 16-bit precision and load with standard transformers or unsloth.

Prompt Format

Always use this exact template — the model was trained on it:

###TASK
Generate the SQL query to answer the following question

### Database Schema
{sql_context}

### Question
{sql_prompt}

### SQL Query

(a) Standard `transformers`

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "A-Kishore/llama-3.2-3b-text2sql"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

prompt = """###TASK
Generate the SQL query to answer the following question

### Database Schema
{sql_context}

### Question
{sql_prompt}

### SQL Query
"""

sql_context = "CREATE TABLE employees (id INT, name TEXT, department TEXT, salary REAL);"
sql_prompt = "What is the average salary per department?"

inputs = tokenizer(
    prompt.format(sql_context=sql_context, sql_prompt=sql_prompt),
    return_tensors="pt"
).to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=150,
    use_cache=True,
    pad_token_id=tokenizer.eos_token_id
)

result = tokenizer.decode(outputs[0], skip_special_tokens=True)
sql = result.split("### SQL Query")[-1].strip()
print(sql)
# SELECT department, AVG(salary) FROM employees GROUP BY department;

(b) Unsloth Fast Inference

import torch
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="A-Kishore/llama-3.2-3b-text2sql",
    max_seq_length=768,
    dtype=torch.float16,
    load_in_4bit=False,
)
FastLanguageModel.for_inference(model)

prompt = """###TASK
Generate the SQL query to answer the following question

### Database Schema
{sql_context}

### Question
{sql_prompt}

### SQL Query
"""

sql_context = "CREATE TABLE employees (id INT, name TEXT, department TEXT, salary REAL);"
sql_prompt = "What is the average salary per department?"

inputs = tokenizer(
    prompt.format(sql_context=sql_context, sql_prompt=sql_prompt),
    return_tensors="pt"
).to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=150,
    temperature=None,
    do_sample=False,
    pad_token_id=tokenizer.eos_token_id
)

result = tokenizer.decode(outputs[0], skip_special_tokens=True)
sql = result.split("### SQL Query")[-1].strip()
print(sql)

Training Details

Dataset

Source: gretelai/synthetic_text_to_sql
Training split: 50,000 shuffled samples from train
Evaluation split: First 200 samples from test

LoRA Configuration

Parameter	Value
Rank (`r`)	`16`
Alpha (`lora_alpha`)	`16`
Target Modules	`q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`
Bias	`none`

LoRA freezes the base model weights and injects trainable rank-decomposition matrices into all attention and MLP projections. Only ~0.75% of parameters are updated, dramatically reducing VRAM usage and preventing catastrophic forgetting.

Hyperparameters

Parameter	Value
Optimizer	`paged_adamw_8bit`
Learning Rate	`2e-4`
LR Scheduler	`linear`
Warmup Steps	`5`
Epochs	`1`
Per-Device Batch Size	`8`
Gradient Accumulation	`1`
Max Sequence Length	`768`
Sequence Packing	`True`
Mixed Precision	`fp16`
Experiment Tracking	Weights & Biases

Training was accelerated using the unsloth library, which provides optimized GPU kernels for 4-bit quantized training (~2× faster than standard configurations).

Repository

Training and evaluation code: a-kishore-dev/llama-text2sql-finetune

Notebooks included:

Text_to_SQL_Finetuning.ipynb — dataset prep, LoRA config, training, export
evaluate_model.ipynb — ROUGE evaluation comparing base vs fine-tuned

Limitations

SQL executability: ROUGE is a lexical proxy. High ROUGE does not guarantee a query will execute or return logically correct results. A query with different aliases or reordered joins may score lower despite being equivalent.
Out-of-distribution schemas: Performance degrades on high-cardinality databases, deeply nested subqueries, or DDL patterns that diverge significantly from the training distribution.
Single epoch: The model was trained for one epoch on 50k samples. Further training may improve generalization.

License

The model adapter is released under Apache 2.0. The underlying base model is governed by the Meta Llama 3 Community License Agreement. Users must comply with both.

Acknowledgements

Unsloth — optimized kernels for 4-bit training and sequence packing
Hugging Face — trl (SFTTrainer) and transformers
Meta AI — Llama 3.2 open weights
Gretel AI — synthetic Text-to-SQL dataset

Author

A-Kishore · GitHub · HuggingFace

Downloads last month: 161

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for A-Kishore/llama-3.2-3b-text2sql

Base model

meta-llama/Llama-3.2-3B-Instruct

Quantized

unsloth/Llama-3.2-3B-Instruct-bnb-4bit

Adapter

(55)

this model

A-Kishore
/

llama-3.2-3b-text2sql

🔮 Llama-3.2-3B-Instruct Text-to-SQL

Model Summary

Evaluation Results

How to Use

Prompt Format

(a) Standard `transformers`

(b) Unsloth Fast Inference

Training Details

Dataset

LoRA Configuration

Hyperparameters

Repository

Limitations

License

Acknowledgements

Author

Model tree for A-Kishore/llama-3.2-3b-text2sql

Dataset used to train A-Kishore/llama-3.2-3b-text2sql

🔮 Llama-3.2-3B-Instruct Text-to-SQL

Model Summary

Evaluation Results

How to Use

Prompt Format

(a) Standard transformers

(b) Unsloth Fast Inference

Training Details

Dataset

LoRA Configuration

Hyperparameters

Repository

Limitations

License

Acknowledgements

Author

Model tree for A-Kishore/llama-3.2-3b-text2sql

Dataset used to train A-Kishore/llama-3.2-3b-text2sql

(a) Standard `transformers`