Instructions to use gaussalgo/T5-LM-Large-text2sql-spider with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use gaussalgo/T5-LM-Large-text2sql-spider with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="gaussalgo/T5-LM-Large-text2sql-spider")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("gaussalgo/T5-LM-Large-text2sql-spider")
model = AutoModelForSeq2SeqLM.from_pretrained("gaussalgo/T5-LM-Large-text2sql-spider")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use gaussalgo/T5-LM-Large-text2sql-spider with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "gaussalgo/T5-LM-Large-text2sql-spider"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gaussalgo/T5-LM-Large-text2sql-spider",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/gaussalgo/T5-LM-Large-text2sql-spider

SGLang

How to use gaussalgo/T5-LM-Large-text2sql-spider with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "gaussalgo/T5-LM-Large-text2sql-spider" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gaussalgo/T5-LM-Large-text2sql-spider",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "gaussalgo/T5-LM-Large-text2sql-spider" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gaussalgo/T5-LM-Large-text2sql-spider",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use gaussalgo/T5-LM-Large-text2sql-spider with Docker Model Runner:
```
docker model run hf.co/gaussalgo/T5-LM-Large-text2sql-spider
```

Add chat_template to tokenizer_config.json

by rafaelwh - opened Jun 12, 2025

base: refs/heads/main

←

from: refs/pr/4

Discussion Files changed

-0

rafaelwh

Jun 12, 2025

No description provided.

Add chat_template to tokenizer_config.jsonf22d13ab

michal-stefanik

Gauss Algorithmic org Jun 12, 2025

Hi @rafaelwh , thanks for contributing. Did you test the updated config yourself? Also, do I get it right that this is something that should also work with earlier versions of Transformers?

rafaelwh

Jun 12, 2025

Hi @michal-stefanik

I haven't tried with earlier versions of Transformers.

Here are better discussions on the issue I was having because there was not a chat_template on the tokenizer_confing.json file:
https://discuss.huggingface.co/t/chat-template-is-not-set-throwing-error/104095/5
https://github.com/huggingface/open-r1/issues/85

As they point out, you could also use this in the JSON instead of my pull request, if you prefer:
"chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n' + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",

I tested both my pull request, and the snippet above, and I was able to get a response from the model.

michal-stefanik changed pull request status to merged Jun 12, 2025

michal-stefanik

Gauss Algorithmic org Jun 12, 2025

Hi @rafaelwh , thank you for all the info. It's likely that this will also avoid problems for other people. I'm not sure about older versions, but I doubt that it would cause a non-recoverable error in loading the model anyway.

Thank you for contributing!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment