Instructions to use TianshengHuang/s1k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TianshengHuang/s1k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TianshengHuang/s1k")
model = AutoModelForCausalLM.from_pretrained("TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TianshengHuang/s1k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TianshengHuang/s1k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TianshengHuang/s1k

SGLang

How to use TianshengHuang/s1k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TianshengHuang/s1k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TianshengHuang/s1k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use TianshengHuang/s1k with Docker Model Runner:
```
docker model run hf.co/TianshengHuang/s1k
```

TianshengHuang commited on Feb 26, 2025

Commit

0edc458

verified ·

1 Parent(s): 9de244d

Upload Qwen2ForCausalLM

Browse files

Files changed (14) hide show

model-00001-of-00014.safetensors +1 -1
model-00002-of-00014.safetensors +1 -1
model-00003-of-00014.safetensors +1 -1
model-00004-of-00014.safetensors +1 -1
model-00005-of-00014.safetensors +1 -1
model-00006-of-00014.safetensors +1 -1
model-00007-of-00014.safetensors +1 -1
model-00008-of-00014.safetensors +1 -1
model-00009-of-00014.safetensors +1 -1
model-00010-of-00014.safetensors +1 -1
model-00011-of-00014.safetensors +1 -1
model-00012-of-00014.safetensors +1 -1
model-00013-of-00014.safetensors +1 -1
model-00014-of-00014.safetensors +1 -1

model-00001-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:941080803af1f10aa8b46d808061b079a4bb4cb33b095d8ac288e11dc1b7570b
 size 4891730992

 version https://git-lfs.github.com/spec/v1
+oid sha256:b2d1f7307b8b06a40e5950a6f318d44eb5d76f3341ae1369c20d7bcdec48bb55
 size 4891730992

model-00002-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bba0d934f11650f18fb9ecb83af89d894de1a983080136168255dcee5eaffdf2
 size 4876059352

 version https://git-lfs.github.com/spec/v1
+oid sha256:0118947f39eaa73e6d38722252a601c36877f8c34a4efebc2a334747bc719b2c
 size 4876059352

model-00003-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e75104789af91a302dbefd35d257f62767b53de4da2bc910f0b2100b422e0f81
 size 4876059384

 version https://git-lfs.github.com/spec/v1
+oid sha256:50d679ac38d3905b11e8b49e5b4f87dffd36df7d7deb1259c4d63b8817bddf1d
 size 4876059384

model-00004-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:78c3f28f9210d9f3e9996b3973d3c7289d802fa89d7da30028ca8ef4bf04ab49
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:a6777b0a7c84ab2253d24031c821b495d4156a5252c0c21f81970478c71de3ef
 size 4876059416

model-00005-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1f6e5a2f23a8aa52192e06e1914b550acad47e51c324ec9b8c977cee402ce83e
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:c02764ab5de5ebd0e9aa44ae63b227bc96e21e08018ca86c729b051f819ae589
 size 4876059416

model-00006-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3f390393169ca4680e2b108343d2606a27a35dfb2278107bc8985f8fc638682b
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:21f250c8a573230cbd8770aed6f8251ce19e88c86e0896d5b8f4f42aedc04503
 size 4876059416

model-00007-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f38e4ea55733303b37f826eef321cf1dd252af4f7c9143dc05033ae79f984b86
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:8d7cd9718e0d0d4438a5c148b3b9d814525f456d99f3a082ac6f7c271085e6be
 size 4876059416

model-00008-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e1c9b9b1ab76ca89912079b2e50657b6a943275eccebb66b846230c7ca20614d
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:15a14dd43145d1eaace17f44e5d7d008962f63158d2c112811bbc83d259e7782
 size 4876059416

model-00009-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0c8753558acaf028b284fc9ee3d9c54ffe31a86d6e2a3aeeb04112649a6531bb
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:2e1585f1a7eadf787b6ead05a474ec35e7ee5f15d334969581ebb4766291a828
 size 4876059416

model-00010-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5759160c46dd2d06c5fd080b9cc181f5b8c76cdebd528e36fd40f1a36845de44
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:3ddd91e888ff7dfc2ab0328d615dd0b5d39deb1dc0981db92b9b745edab97752
 size 4876059416

model-00011-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3bfa2e5d569e506c40213b849d3341f2f771e5966f963248c840af4c4522a5ea
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:9ee8cfc304aa563704ea0961c053b3cd2a463f4cf2203063d58584fe4b7d345d
 size 4876059416

model-00012-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1be75026843f4b4b5736a0e80b1c6ccb687d3db05869c2115acfdfb8b6bb7d2a
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:72a7cc9b423a24021940bd5691e9aee447ca31749e6f0fc5d0f3f2cee762ca39
 size 4876059416

model-00013-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cbe6a984a50104c15cea08956c94e6b382d92e519a9685815404c454fd9fc4af
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:278e78ac3cab75007b87eaaa611ca2c39325acc550c026c2b1e8cb6e49c6f08c
 size 4876059416

model-00014-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:08717bbc4a8c16807cc8a820a9f6e02a8bed8288e6d9e9370b0dad5ded474a57
 size 2123397800

 version https://git-lfs.github.com/spec/v1
+oid sha256:1cb6a1c39372c15f208b15b8641a3e8ea5f5b7f8ad05a674d1fbb28a39919ec9
 size 2123397800