Instructions to use shivenkk/sqlpup-394m-sft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use shivenkk/sqlpup-394m-sft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="shivenkk/sqlpup-394m-sft")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("shivenkk/sqlpup-394m-sft")
model = AutoModelForCausalLM.from_pretrained("shivenkk/sqlpup-394m-sft", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use shivenkk/sqlpup-394m-sft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "shivenkk/sqlpup-394m-sft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shivenkk/sqlpup-394m-sft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/shivenkk/sqlpup-394m-sft

SGLang

How to use shivenkk/sqlpup-394m-sft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "shivenkk/sqlpup-394m-sft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shivenkk/sqlpup-394m-sft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "shivenkk/sqlpup-394m-sft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shivenkk/sqlpup-394m-sft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use shivenkk/sqlpup-394m-sft with Docker Model Runner:
```
docker model run hf.co/shivenkk/sqlpup-394m-sft
```

sqlpup-394m-sft / README.md

shivenkk

link the source repository

d99a6ad verified 4 days ago

preview code

Raw

History Blame Contribute Delete

2.82 kB

metadata

license: mit
language:
  - en
library_name: transformers
pipeline_tag: text-generation
tags:
  - text-to-sql
  - sql
  - bird-bench
datasets:
  - HuggingFaceFW/fineweb-edu
  - bigcode/starcoderdata

sqlpup-394m-sft

Supervised fine-tuning of a from-scratch 394M text-to-SQL decoder. 22.84% execution accuracy on BIRD dev.

Pretrained from scratch on 9.67B tokens, then supervised fine-tuned on filtered natural-language-to-SQL pairs.

Code, configs, and the per-run artifacts behind every number here: github.com/shivenkk/sqlpup.

What it is

A 394M-parameter decoder trained from scratch for text-to-SQL: no pretrained initialisation, no distillation from a larger model. The corpus (9.67B tokens) and the 32,768-entry byte-level BPE tokenizer were built for this task.

Architecture: Llama-style, 1024 wide by 32 layers, grouped-query attention with 16 query heads and 4 key/value heads, RoPE, SwiGLU, RMSNorm, 2048-token context.

Measured accuracy

Execution accuracy on the BIRD development split, all 1534 questions:

decoding	this model	`sqlpup-394m-sft-grpo`
greedy	17.41%	18.12%
+ schema compaction	17.86%	19.23%
+ compaction, 7-sample voting	22.84%	23.51%

The voting rows are means over sampling seeds 0/101/202. Absolute accuracy is far from frontier systems; the point of the project was a controlled measurement of what from-scratch pretraining at this scale buys, not a leaderboard score.

Prompt format

The model expects the schema's CREATE TABLE statements verbatim, then the question, then BIRD's evidence string. It emits a short comment block naming the tables and columns it intends to use, then the SQL. Use the accompanying code so the format and the SQL extraction match what the numbers were measured with.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("shivenkk/sqlpup-394m-sft")
tok = AutoTokenizer.from_pretrained("shivenkk/sqlpup-394m-sft")

For evaluation, prefer the project's own harness, which handles the prompt spec, over-context schema compaction, and execution-guided voting:

python -m sqlpup.cli eval predict --model-dir <dir> --examples dev.json \
  --db-root dev_databases --out-dir out --compact-overflow --self-consistency 7

Limitations

2048-token context. Large schemas need the compaction pass to fit.
Trained and measured on BIRD only. No test-split number, no second dataset.
Roughly 1% of dev questions produce no parseable SQL in the voting configuration, recorded as errors rather than dropped.

Training data

FineWeb-Edu, StarCoder, the Python subset of The Stack, SynSQL-2.5M, and SchemaPile, deduplicated and decontaminated against the evaluation sets.