Instructions to use TianshengHuang/s1k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TianshengHuang/s1k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TianshengHuang/s1k")
model = AutoModelForCausalLM.from_pretrained("TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TianshengHuang/s1k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TianshengHuang/s1k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TianshengHuang/s1k

SGLang

How to use TianshengHuang/s1k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TianshengHuang/s1k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TianshengHuang/s1k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use TianshengHuang/s1k with Docker Model Runner:
```
docker model run hf.co/TianshengHuang/s1k
```

TianshengHuang commited on Feb 26, 2025

Commit

cd7d990

verified ·

1 Parent(s): 232d2e1

Upload Qwen2ForCausalLM

Browse files

Files changed (14) hide show

model-00001-of-00014.safetensors +1 -1
model-00002-of-00014.safetensors +1 -1
model-00003-of-00014.safetensors +1 -1
model-00004-of-00014.safetensors +1 -1
model-00005-of-00014.safetensors +1 -1
model-00006-of-00014.safetensors +1 -1
model-00007-of-00014.safetensors +1 -1
model-00008-of-00014.safetensors +1 -1
model-00009-of-00014.safetensors +1 -1
model-00010-of-00014.safetensors +1 -1
model-00011-of-00014.safetensors +1 -1
model-00012-of-00014.safetensors +1 -1
model-00013-of-00014.safetensors +1 -1
model-00014-of-00014.safetensors +1 -1

model-00001-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f42f2faead93998265f957f49f3a8e8f5a8fd61010f3fecc72b41939e1c9757b
 size 4891730992

 version https://git-lfs.github.com/spec/v1
+oid sha256:9ac6b35094b88884cbba8bfbe52220152dc6c6497eb9be52ae5a0cf04bbf16cc
 size 4891730992

model-00002-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2a400f12567329fc38ae029c8d74a426890702249befe05d16d20f24f57c7455
 size 4876059352

 version https://git-lfs.github.com/spec/v1
+oid sha256:8070f46afa683059521e5addd569341c052e853ed486d3392fedba0d70a7ceeb
 size 4876059352

model-00003-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:85c0ff85d8218ce6e616193f058781959bd5ee83948e9dd78c132c364b04c9da
 size 4876059384

 version https://git-lfs.github.com/spec/v1
+oid sha256:b8565188676c592dc7705209599eb74db1ad32ca00b0777bf74f279104b384e0
 size 4876059384

model-00004-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a6ea8240172b8f8c39fac1739a14d569731123f7c431afc0409cf286b78480ae
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:e397bf8a65073fc7e67c737164f896fa8b996612d5eb77df2db970a21f5c2105
 size 4876059416

model-00005-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f45ba07cf7995f4280d9f0f86f34a68564ca23f6bc95d7b43c7252bb030cbe4d
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:2b494fa73ade77861b316bd0a33659459dbd52e135156dcea4fd743a84cc4b43
 size 4876059416

model-00006-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c44c9c19ef703ce40dd9acc92d3982c5375cba4c340ef2ccbc9426af38a0ef66
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:52b6640ab8df69c283c067a4e91a5a7ab5b9b2304ddcafd572f78bf6696e70af
 size 4876059416

model-00007-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3d8d7d65067a1046ee12389da837521221ec8645abf01816d06e6e6b7bb53b37
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:58f137c8e82e4de4e385175d73682eb174435125eaf0180f013919c68c5b09ba
 size 4876059416

model-00008-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ca776a013917a6b4f9fa36de32f2c137085998a46b9a3eaebdddf85d1f4affea
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:8b8a982c57e20f794ccb4c9ed947da62ac5d0923358d688e97df5606c45e51ad
 size 4876059416

model-00009-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7359c6e8cf26121a4b3c56781e934018718f49d9dae834539a8fcd090cf32af4
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:21c2db8a2553a4521897b920e8acb11edbd89ff8cd7c2195d980be6a5cdaa42f
 size 4876059416

model-00010-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8198f4bfe5b8aa6af32f9fe5040e00865917963d90a4fbde03a9f7d1c4601a43
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:25d1eb3c960c828fc2197acb51d940c83f0e0d695e39784c38ab350bd1615814
 size 4876059416

model-00011-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:54d12a2077949a0084ee2c8c83f83be05a6440249f1099c1adca8c60597de79d
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:8f05c6565b7041878cf2c3fbc58abf60a05c6b1928387716f4f9aa1316ed12db
 size 4876059416

model-00012-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:10f58e53ec99b5ac0742422dd3e3ca65cf581ba2fe44990dd52ce796220dc4a5
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:3fdbc908f99ccb61c304dddc3057b9febb27040b0af69f93cfd8bcbb9764e007
 size 4876059416

model-00013-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d2683af30b003bd2d6495c3537894fb1160396b35efa49dfad6ffbd621e64ac8
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:1bcde3f01b2ed639541dd71034baad2930f06dc24a73bfa4c31678c0ed83459e
 size 4876059416

model-00014-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:551ff128ca2423f0fbb13d831169f7c31eaee205936d50350eeb828772460530
 size 2123397800

 version https://git-lfs.github.com/spec/v1
+oid sha256:976b74f85a60c3f70e280ebbe682d82ce435a5606ef26f27cd1e8d755aad52c4
 size 2123397800