Instructions to use ihebaker10/text2sql-7b-v3-16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ihebaker10/text2sql-7b-v3-16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ihebaker10/text2sql-7b-v3-16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ihebaker10/text2sql-7b-v3-16")
model = AutoModelForCausalLM.from_pretrained("ihebaker10/text2sql-7b-v3-16", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ihebaker10/text2sql-7b-v3-16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ihebaker10/text2sql-7b-v3-16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ihebaker10/text2sql-7b-v3-16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ihebaker10/text2sql-7b-v3-16

SGLang

How to use ihebaker10/text2sql-7b-v3-16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ihebaker10/text2sql-7b-v3-16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ihebaker10/text2sql-7b-v3-16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ihebaker10/text2sql-7b-v3-16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ihebaker10/text2sql-7b-v3-16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ihebaker10/text2sql-7b-v3-16 with Docker Model Runner:
```
docker model run hf.co/ihebaker10/text2sql-7b-v3-16
```

text2sql-7b-v3-16 / README.md

ihebaker10

Update technical model card

e590305 verified about 23 hours ago

preview code

Raw

History Blame Contribute Delete

7.46 kB

metadata

library_name: transformers
pipeline_tag: text-generation
language:
  - en
tags:
  - text2sql
  - sql
  - pgvector
  - grpo
  - lora
  - safetensors
  - bf16

text2sql-7b-v3-16

Model Details

text2sql-7b-v3-16 is an 8B-parameter causal language model fine-tuned for text-to-SQL generation. It accepts a natural-language question plus schema context and returns SQL inside the notebook's required answer format.

Model type: causal language model, text generation
Primary task: text-to-SQL
Base model family: Arctic-Text2SQL-R1-7B
Output format: <think>...</think><answer>SQL</answer>
Tensor format: Safetensors
Precision: BF16
Fine-tuning method: PEFT LoRA adapters merged into the final model artifact
Final artifact path used locally: outputs/text2sql-7b-v3-16/merged-text2sql-7b-v3-16

Intended Technical Use

Use this model to generate PostgreSQL-style SQL from a natural-language question and schema description. The fine-tuning set contains both plain SQL examples and pgvector retrieval examples using embed_query() with vector-distance operators.

This model is not a SQL execution engine. Generated SQL should be parsed, reviewed, and run in a controlled environment before use.

How to Use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "ihebaker10/text2sql-7b-v3-16"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "system",
        "content": "Generate SQL. Return the final SQL inside <answer>...</answer>.",
    },
    {
        "role": "user",
        "content": "Schema:\n...\n\nQuestion:\nList the latest 10 records.",
    },
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=768,
        temperature=0.2,
        top_p=0.95,
        do_sample=False,
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Data

The training data contains 10,000 text-to-SQL examples loaded from a MongoDB collection and split into 9,001 training examples and 999 validation examples.

Split / Type	Examples	Share
Train	9,001	90.0%
Validation	999	10.0%
SQL-only	3,931	39.3%
pgvector	6,069	60.7%

Each record contains an identifier, is_sql_only, natural-language question, reference SQL query, reasoning hint, schema text, table list, and version.

Training Procedure

Training used a three-stage PEFT workflow:

Stage	Objective	Data	Schedule
Stage 1a	SQL-only SFT warm-up	SQL-only examples	1 epoch
Stage 1b	Mixed SFT	Natural mixed SQL + pgvector split	2 epochs
Stage 2	GRPO refinement	Natural mixed SQL + pgvector split	about 0.5 epoch, capped by max steps

The LoRA adapter was merged into the base weights before publishing the final Safetensors artifact.

Hyperparameters

Parameter	Stage 1a SFT	Stage 1b SFT	Stage 2 GRPO
Learning rate	`1e-4`	`7e-5`	`1e-05`
Scheduler	`cosine`	`cosine`	`SchedulerType.COSINE`
Warmup ratio	`0.05`	`0.03`	`0.05`
Per-device train batch size	`1`	`1`	`2`
Gradient accumulation	`16`	`16`	`4`
Effective batch size	`16`	`16`	`8`
Weight decay	`0.01`	`0.01`	`0.01`
Max grad norm	`1.0`	`1.0`	`1.0`
Precision	`bf16`	`bf16`	`bf16`
Optimizer	`adamw_torch`	`adamw_torch`	`adamw_torch`
Max sequence length	`1536`	`1536`	n/a
Max prompt length	n/a	n/a	`1024`
Max completion length	n/a	n/a	`192`
Rollouts per prompt	n/a	n/a	`2`
Temperature	n/a	n/a	`0.8`
Top-p	n/a	n/a	`0.95`
KL beta	n/a	n/a	`0.002`
Early stopping patience	n/a	n/a	`2`

LoRA configuration:

Parameter	Value
Rank	`64`
Alpha	`64`
Dropout	`0.05`
Bias	`none`
Target modules	`down_proj`, `gate_proj`, `k_proj`, `o_proj`, `q_proj`, `up_proj`, `v_proj`

Reward Functions

Stage 2 used the following reward checks:

format_reward: validates the <think> and <answer> response structure.
sql_syntax_reward: rewards SQL that parses successfully.
pgvector_usage_reward: checks whether embed_query() is used only when expected.
execution_reward: optionally compares generated SQL results with reference SQL when DB_URL is configured.

Evaluation

Positive class for precision, recall, and F1 is pgvector usage, meaning the generated SQL contains embed_query().

Split	N	Format	Syntax	Use accuracy	Precision	Recall	F1	Exact match
SQL-only	393	100.0%	99.5%	89.8%	0.0%	0.0%	0.0%	33.1%
pgvector	606	100.0%	99.0%	73.8%	100.0%	73.8%	84.9%	11.7%
combined	999	100.0%	99.2%	80.1%	91.8%	73.8%	81.8%	20.1%

Stage eval losses:

Stage	Eval loss
Stage 1a	`0.013554`
Stage 1b	`0.002467`
Stage 2	`0.000403`

Detailed pgvector decision metrics from the saved final merged-model evaluation:

Split	TP	FP	FN	TN	SQL-only specificity	Embed false positive rate
SQL-only	0	40	0	353	89.8%	10.2%
pgvector	447	0	159	0	0.0%	0.0%
combined	447	40	159	353	89.8%	10.2%

Detailed saved stage metrics:

Stage	Train loss	Eval loss	Train runtime	Eval runtime	Train samples/s	Eval samples/s	Train steps/s	Eval steps/s	Total FLOPs
Stage 1a	`2.460437`	`0.013554`	2478s	167.1s	1.428	5.979	0.089	5.979	`2.222e+17`
Stage 1b	`0.049128`	`0.002467`	12935s	164.9s	1.392	6.057	0.087	6.057	`1.165e+18`
Stage 2	`0.000392`	`0.000403`	36192s	2772.4s	0.124	0.360	0.016	0.045	`0.000e+00`

Limitations

SQL should be validated before execution.
Exact-match SQL is strict and may undercount semantically equivalent queries.
pgvector recall on the validation split is lower than precision, so some retrieval-style prompts may be answered as plain SQL.
Execution-based reward depends on DB_URL; without it, training optimizes formatting, SQL syntax, and pgvector usage only.

Technical Environment

Frameworks: transformers, peft, trl, datasets, torch
Attention implementation: sdpa
Gradient checkpointing: enabled
CUDA memory fraction in notebook: 0.98
TF32 matmul: enabled
TensorBoard logging: enabled for all training stages

Environmental Impact

Carbon emissions were not measured. Recorded training runtimes were approximately:

Stage	Runtime
Stage 1a	2478 seconds
Stage 1b	12935 seconds
Stage 2	36192 seconds

Total recorded training runtime was approximately 14.3 hours on a local CUDA GPU.