Instructions to use AlioLeuchtmann/flash-sqlite with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AlioLeuchtmann/flash-sqlite with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AlioLeuchtmann/flash-sqlite")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("AlioLeuchtmann/flash-sqlite")
model = AutoModelForMultimodalLM.from_pretrained("AlioLeuchtmann/flash-sqlite", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AlioLeuchtmann/flash-sqlite with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AlioLeuchtmann/flash-sqlite"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AlioLeuchtmann/flash-sqlite",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AlioLeuchtmann/flash-sqlite

SGLang

How to use AlioLeuchtmann/flash-sqlite with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AlioLeuchtmann/flash-sqlite" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AlioLeuchtmann/flash-sqlite",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AlioLeuchtmann/flash-sqlite" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AlioLeuchtmann/flash-sqlite",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use AlioLeuchtmann/flash-sqlite with Docker Model Runner:
```
docker model run hf.co/AlioLeuchtmann/flash-sqlite
```

AlioLeuchtmann commited on 5 days ago

Commit

c6a9bef

verified ·

1 Parent(s): 7b5df48

Update README.md

Browse files

Files changed (1) hide show

README.md +37 -3

README.md CHANGED Viewed

@@ -83,6 +83,8 @@ All scores are **BIRD test-set execution accuracy**. Markers show how many candi
 **Frontier and large general-purpose LLMs** (single-model leaderboard):
 | model | provider | size | test EX |
 |---|---|---|---|
 | **flash-sqlite (this model)** | — | **2B** | **73.73\*\*** |
@@ -97,7 +99,11 @@ All scores are **BIRD test-set execution accuracy**. Markers show how many candi
 | Mistral (2024 baseline) | Mistral AI | 123B | 55.84 |
 | GPT-4 | OpenAI | proprietary | 54.89 |
-**Specialised text-to-SQL models** (single-model leaderboard):
 | model | size | test EX |
 |---|---|---|
@@ -120,7 +126,7 @@ flash-sqlite samples 32 candidates and selects one with its own judge (see [How
   - The 32 samples per question come from one request (`n=32`), and vLLM batches them together.
   - With **prefix caching**, the long schema prompt is prefilled once and shared by all 32 samples.
   - The judge prompts of the same question also share their schema prefix.
-- **Throughput:** about **6,400–7,600 output tokens/s** on one 96 GB RTX 6000 MaxQ GPU, which is 8–11 finished candidates per second.
 - **The judge is almost free:**
   - It reads one token (P(Yes)) per result group instead of generating text.
   - On BIRD dev, **38% of questions needed no judge call at all**, because all executable candidates agreed.
@@ -131,7 +137,7 @@ flash-sqlite samples 32 candidates and selects one with its own judge (see [How
 ## BIRD dev results
-These come from the dev revision `bird-sql-dev-1106`: 1,532 questions with executable gold SQL, scored by execution accuracy with set comparison. All three rows come from the same generation run.
 | inference | EX | simple | moderate | challenging |
 |---|---|---|---|---|
@@ -156,6 +162,34 @@ For each question:
    - P(Yes) is read from the first generated token, restricted to *Yes* / *No* and renormalised.
 4. **Select** the group with the highest **votes × P(Yes)**.
 ## Usage
 The model expects the OmniSQL-style prompt shown below and answers with its reasoning followed by a fenced SQL block. Always take the **last** ```sql block. Use **non-thinking mode** (`enable_thinking=False`).

 **Frontier and large general-purpose LLMs** (single-model leaderboard):
+![flash-sqlite vs frontier LLM baselines on the BIRD test set](bird_test_frontier.png)
 | model | provider | size | test EX |
 |---|---|---|---|
 | **flash-sqlite (this model)** | — | **2B** | **73.73\*\*** |
 | Mistral (2024 baseline) | Mistral AI | 123B | 55.84 |
 | GPT-4 | OpenAI | proprietary | 54.89 |
+**Accuracy per parameter.** The chart shows every single-model entry with a published size. flash-sqlite lies on the Pareto frontier, so no smaller entry scores higher. It is the highest-scoring model at 2B parameters or fewer.
+![BIRD test EX by model size, single-model leaderboard](bird_size_vs_score.png)
+**Specialised text-to-SQL models** (single-model leaderboard, selection):
 | model | size | test EX |
 |---|---|---|
   - The 32 samples per question come from one request (`n=32`), and vLLM batches them together.
   - With **prefix caching**, the long schema prompt is prefilled once and shared by all 32 samples.
   - The judge prompts of the same question also share their schema prefix.
+- **Throughput:** about **6,400–7,600 output tokens/s** on one 96 GB workstation GPU, which is 8–11 finished candidates per second.
 - **The judge is almost free:**
   - It reads one token (P(Yes)) per result group instead of generating text.
   - On BIRD dev, **38% of questions needed no judge call at all**, because all executable candidates agreed.
 ## BIRD dev results
+These come from the dev revision `bird-sql-dev-1106`: 1,532 questions with executable gold SQL, scored by execution accuracy with set comparison. The three candidate-based rows come from the same generation run; the greedy row comes from a separate single-answer run.
 | inference | EX | simple | moderate | challenging |
 |---|---|---|---|---|
    - P(Yes) is read from the first generated token, restricted to *Yes* / *No* and renormalised.
 4. **Select** the group with the highest **votes × P(Yes)**.
+## Training
+Training is **supervised fine-tuning only**: no reinforcement learning, no preference optimisation and no reward model. Every SQL target is **execution-verified**. A chain-of-thought trace was only kept if its final SQL, run on the real database, returned the gold result set.
+**Stage 1: synthetic pre-training.**
+- Data: 57,178 execution-verified synthetic examples, over generated databases and questions designed to match BIRD's style.
+- Training: 2 epochs.
+**Stage 2: joint generator + judge SFT.** 27,179 rows in total:
+- **14,327 SQL generation traces** (BIRD train and synthetic databases):
+  - The traces follow a structured chain of thought: question and hint mapping, schema selection, join path, filters, aggregation, edge-case checks, then the final SQL.
+  - They were rejection-sampled from a Qwen 27B teacher.
+- **8,314 pointwise critiques:**
+  - Each one judges a single candidate query and its execution result, and ends in a Yes/No verdict.
+  - The teacher wrote them without seeing the label. Only critiques whose verdict matched the execution-based label were kept.
+- **4,538 listwise judge rows:**
+  - Several candidates (grouped by execution result) are compared, then the model chooses one or answers "none".
+Hyperparameters for stage 2:
+- Full fine-tune in bf16, 3 epochs.
+- Learning rate 2e-5 (cosine schedule), 50 warmup steps.
+- Effective batch size 32.
+- NEFTune α = 5.
+- Maximum sequence length 16,384.
+The prompts contain per-column descriptions, read from each BIRD database's `database_description/*.csv` files.
 ## Usage
 The model expects the OmniSQL-style prompt shown below and answers with its reasoning followed by a fenced SQL block. Always take the **last** ```sql block. Use **non-thinking mode** (`enable_thinking=False`).