Instructions to use TianshengHuang/s1k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TianshengHuang/s1k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TianshengHuang/s1k")
model = AutoModelForCausalLM.from_pretrained("TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TianshengHuang/s1k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TianshengHuang/s1k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TianshengHuang/s1k

SGLang

How to use TianshengHuang/s1k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TianshengHuang/s1k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TianshengHuang/s1k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use TianshengHuang/s1k with Docker Model Runner:
```
docker model run hf.co/TianshengHuang/s1k
```

TianshengHuang commited on Feb 26, 2025

Commit

5fb8759

verified ·

1 Parent(s): cd7d990

Upload Qwen2ForCausalLM

Browse files

Files changed (14) hide show

model-00001-of-00014.safetensors +1 -1
model-00002-of-00014.safetensors +1 -1
model-00003-of-00014.safetensors +1 -1
model-00004-of-00014.safetensors +1 -1
model-00005-of-00014.safetensors +1 -1
model-00006-of-00014.safetensors +1 -1
model-00007-of-00014.safetensors +1 -1
model-00008-of-00014.safetensors +1 -1
model-00009-of-00014.safetensors +1 -1
model-00010-of-00014.safetensors +1 -1
model-00011-of-00014.safetensors +1 -1
model-00012-of-00014.safetensors +1 -1
model-00013-of-00014.safetensors +1 -1
model-00014-of-00014.safetensors +1 -1

model-00001-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9ac6b35094b88884cbba8bfbe52220152dc6c6497eb9be52ae5a0cf04bbf16cc
 size 4891730992

 version https://git-lfs.github.com/spec/v1
+oid sha256:29d318e9d6e9da447dc4b1ed41bf981cb066d861b85c740f5aaa916cbcabbaea
 size 4891730992

model-00002-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8070f46afa683059521e5addd569341c052e853ed486d3392fedba0d70a7ceeb
 size 4876059352

 version https://git-lfs.github.com/spec/v1
+oid sha256:1be578c963fe2bcf56f5b9e1e1cd8cf03ce37e12f6aff7710620b150205821c0
 size 4876059352

model-00003-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b8565188676c592dc7705209599eb74db1ad32ca00b0777bf74f279104b384e0
 size 4876059384

 version https://git-lfs.github.com/spec/v1
+oid sha256:c501677280ab3c7d74d12a78822ed1d05028e926e065e96226f6f0567a9ad299
 size 4876059384

model-00004-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e397bf8a65073fc7e67c737164f896fa8b996612d5eb77df2db970a21f5c2105
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:950a1bdea4691de5a332fd58d013649de20cf8060d0728c3203b66466a934f17
 size 4876059416

model-00005-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2b494fa73ade77861b316bd0a33659459dbd52e135156dcea4fd743a84cc4b43
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:693c3599114f2c224378d43f5234b51e059078ea7f881464caf2110586e524ca
 size 4876059416

model-00006-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:52b6640ab8df69c283c067a4e91a5a7ab5b9b2304ddcafd572f78bf6696e70af
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:8ac5ddcfd3f6066e5b4c808f235b89762c7f3a9a93e555703bfe0d3338b5a590
 size 4876059416

model-00007-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:58f137c8e82e4de4e385175d73682eb174435125eaf0180f013919c68c5b09ba
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:42229c2a781d46f66cdafbcf5693d25cfde19ccbfecbff7cbcf8e8d7638bcf5e
 size 4876059416

model-00008-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8b8a982c57e20f794ccb4c9ed947da62ac5d0923358d688e97df5606c45e51ad
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:c52e3e742e751ddb576ba9f2bdc7a21f6b248d1e1124397c2115e6fb6549c694
 size 4876059416

model-00009-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:21c2db8a2553a4521897b920e8acb11edbd89ff8cd7c2195d980be6a5cdaa42f
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:31dc2af9f1e0911dfa6548d9b61aab3fc733aa906d4a4f9af671db9f367c40d3
 size 4876059416

model-00010-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:25d1eb3c960c828fc2197acb51d940c83f0e0d695e39784c38ab350bd1615814
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:0cf5303d34e357ef16e3ef57561257964c13326d69fae429f722c3cc5747d470
 size 4876059416

model-00011-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8f05c6565b7041878cf2c3fbc58abf60a05c6b1928387716f4f9aa1316ed12db
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:6d7be9ac97d2d970b89457810ca2e9bb6c900370e7302bcf8f32a642e0627e8f
 size 4876059416

model-00012-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3fdbc908f99ccb61c304dddc3057b9febb27040b0af69f93cfd8bcbb9764e007
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:2e081f26fa19ff1227d05295f693d9abe198bfe3bab2ecf2fb36a68d6ce30c2c
 size 4876059416

model-00013-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1bcde3f01b2ed639541dd71034baad2930f06dc24a73bfa4c31678c0ed83459e
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:fc4d2970197d4199694522f4bf556de547caae9b3dec08923b11eee6b5e7e062
 size 4876059416

model-00014-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:976b74f85a60c3f70e280ebbe682d82ce435a5606ef26f27cd1e8d755aad52c4
 size 2123397800

 version https://git-lfs.github.com/spec/v1
+oid sha256:6eab0526af0dff2616c99a5a8de358cf119c390f850156a9022c4a0460cbd3bc
 size 2123397800