Instructions to use TianshengHuang/s1k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TianshengHuang/s1k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TianshengHuang/s1k")
model = AutoModelForCausalLM.from_pretrained("TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TianshengHuang/s1k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TianshengHuang/s1k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TianshengHuang/s1k

SGLang

How to use TianshengHuang/s1k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TianshengHuang/s1k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TianshengHuang/s1k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use TianshengHuang/s1k with Docker Model Runner:
```
docker model run hf.co/TianshengHuang/s1k
```

TianshengHuang commited on Feb 26, 2025

Commit

232d2e1

verified ·

1 Parent(s): 2a3c286

Upload Qwen2ForCausalLM

Browse files

Files changed (14) hide show

model-00001-of-00014.safetensors +1 -1
model-00002-of-00014.safetensors +1 -1
model-00003-of-00014.safetensors +1 -1
model-00004-of-00014.safetensors +1 -1
model-00005-of-00014.safetensors +1 -1
model-00006-of-00014.safetensors +1 -1
model-00007-of-00014.safetensors +1 -1
model-00008-of-00014.safetensors +1 -1
model-00009-of-00014.safetensors +1 -1
model-00010-of-00014.safetensors +1 -1
model-00011-of-00014.safetensors +1 -1
model-00012-of-00014.safetensors +1 -1
model-00013-of-00014.safetensors +1 -1
model-00014-of-00014.safetensors +1 -1

model-00001-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:33662456af28e2f3cdc0c4b34a6efdf0abe39bcdd535fd808f7c8b497481b72c
 size 4891730992

 version https://git-lfs.github.com/spec/v1
+oid sha256:f42f2faead93998265f957f49f3a8e8f5a8fd61010f3fecc72b41939e1c9757b
 size 4891730992

model-00002-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1a1a94ad076247e0deb364d6a0607a9a0c6a305a9d2cda738aac9138904b2ef3
 size 4876059352

 version https://git-lfs.github.com/spec/v1
+oid sha256:2a400f12567329fc38ae029c8d74a426890702249befe05d16d20f24f57c7455
 size 4876059352

model-00003-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fc72aecc72e9660bd32ecd57aab23ce7707a3f6bf034b527880de2f7242d3ad8
 size 4876059384

 version https://git-lfs.github.com/spec/v1
+oid sha256:85c0ff85d8218ce6e616193f058781959bd5ee83948e9dd78c132c364b04c9da
 size 4876059384

model-00004-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:99acaf16ff7074f24f8aca9fa77a0c6f2767540d6603bf72cc7c94b9f561ff4e
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:a6ea8240172b8f8c39fac1739a14d569731123f7c431afc0409cf286b78480ae
 size 4876059416

model-00005-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:23d95d293f228791ffac2c916166bb23e4915990055ec7fbb270d233ee44d6ac
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:f45ba07cf7995f4280d9f0f86f34a68564ca23f6bc95d7b43c7252bb030cbe4d
 size 4876059416

model-00006-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:28347ff57297c8b848eae8d70423b7f01029f3adfb3a3da3fbebc1d5dd425433
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:c44c9c19ef703ce40dd9acc92d3982c5375cba4c340ef2ccbc9426af38a0ef66
 size 4876059416

model-00007-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ebce35adb633b7da5c6881d54a0703acb33fa6463f3e2c4954b4e0e9c79afa43
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:3d8d7d65067a1046ee12389da837521221ec8645abf01816d06e6e6b7bb53b37
 size 4876059416

model-00008-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:042c7a07f359eda75915fd6fef12ca89ca97238daf163bed7668b26ff7eba100
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:ca776a013917a6b4f9fa36de32f2c137085998a46b9a3eaebdddf85d1f4affea
 size 4876059416

model-00009-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:946c13a45cd31c7679e90c12d7e1bb5cfac63f850a09f72c7ad110962a79722f
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:7359c6e8cf26121a4b3c56781e934018718f49d9dae834539a8fcd090cf32af4
 size 4876059416

model-00010-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:26c96e5960f531342e7110da2fa27dcbcee152384623a7774bd947bd8c86425c
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:8198f4bfe5b8aa6af32f9fe5040e00865917963d90a4fbde03a9f7d1c4601a43
 size 4876059416

model-00011-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f87dfd8aba4eb858d38c46cd67d302f8b1b8af4716bb48de575f14d5b897db72
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:54d12a2077949a0084ee2c8c83f83be05a6440249f1099c1adca8c60597de79d
 size 4876059416

model-00012-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2bda10a240f71514c6ad80fef8d3cc1a8c239bcd07d9e72c21cb9a7706fe5049
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:10f58e53ec99b5ac0742422dd3e3ca65cf581ba2fe44990dd52ce796220dc4a5
 size 4876059416

model-00013-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f5d87851d0658b2f01ff8cfec67da5f343613dd01044d82c030895e9cc7e18bb
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:d2683af30b003bd2d6495c3537894fb1160396b35efa49dfad6ffbd621e64ac8
 size 4876059416

model-00014-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:666f9b3d4ba93db5bae6dc3a654dbe8bb4469132249ca9377fc25fed288a8a47
 size 2123397800

 version https://git-lfs.github.com/spec/v1
+oid sha256:551ff128ca2423f0fbb13d831169f7c31eaee205936d50350eeb828772460530
 size 2123397800