Instructions to use TianshengHuang/s1k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TianshengHuang/s1k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TianshengHuang/s1k")
model = AutoModelForCausalLM.from_pretrained("TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TianshengHuang/s1k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TianshengHuang/s1k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TianshengHuang/s1k

SGLang

How to use TianshengHuang/s1k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TianshengHuang/s1k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TianshengHuang/s1k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use TianshengHuang/s1k with Docker Model Runner:
```
docker model run hf.co/TianshengHuang/s1k
```

TianshengHuang commited on Feb 26, 2025

Commit

73f3737

verified ·

1 Parent(s): 587ab4c

Upload Qwen2ForCausalLM

Browse files

Files changed (14) hide show

model-00001-of-00014.safetensors +1 -1
model-00002-of-00014.safetensors +1 -1
model-00003-of-00014.safetensors +1 -1
model-00004-of-00014.safetensors +1 -1
model-00005-of-00014.safetensors +1 -1
model-00006-of-00014.safetensors +1 -1
model-00007-of-00014.safetensors +1 -1
model-00008-of-00014.safetensors +1 -1
model-00009-of-00014.safetensors +1 -1
model-00010-of-00014.safetensors +1 -1
model-00011-of-00014.safetensors +1 -1
model-00012-of-00014.safetensors +1 -1
model-00013-of-00014.safetensors +1 -1
model-00014-of-00014.safetensors +1 -1

model-00001-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6bb38575162510a4e1d5047cccd3281d4b90e74b0d3f361ddca7bffe88f84a60
 size 4891730992

 version https://git-lfs.github.com/spec/v1
+oid sha256:7ff8099197c9463c2ce67671ded305f9794e1a9800894c19cf8c6cc1824cf65a
 size 4891730992

model-00002-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:81c2e5a8366e29c79f24c326ebe4e5f3761275a75ce1f1782d7d8f8ae4d619a2
 size 4876059352

 version https://git-lfs.github.com/spec/v1
+oid sha256:ae2344787996ee583f1f361ea6d9f2fd0841c01d410a0a1afcd58c891cfa699c
 size 4876059352

model-00003-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dfa8335cc41073550a091a50b338909906c3fe0cd922d367507bda07771ab5ec
 size 4876059384

 version https://git-lfs.github.com/spec/v1
+oid sha256:80d7fa9860bea092c147fc17aa207f4e793425f68e820e0122fce3dca30838fa
 size 4876059384

model-00004-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:62c5a2b2265fca290164fcfe178cbb58482295cfe4b50a533369725b00658f2b
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:a36add0889d6d71e462ca30fd438daea0dfac0bf6e8b35dcac40eee5ede9dd35
 size 4876059416

model-00005-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6e7bfe86953e9196e4cfa17d79057974f8e285bc0328f7736c63e165f559c3e6
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:b61321839527c906f2f22336fc07035f2c2467e204eda64d5acb15204af34e1d
 size 4876059416

model-00006-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:98dbce5c2693ce8bdc4a394cf2656d7db09793539325b689164d383c1b252038
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:e72b13bd9b2932a8b24de9b1d5f43b6fbae50976c1fa55829fff3962e283387e
 size 4876059416

model-00007-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:92cb881555ec2d45f987e883f77fae368cdc77c99ea7bc7d6481e7f021687f73
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:8da61cc14f4b14f49f82caaf878b7d633b8aae41acd99d88ea54f19777e423a1
 size 4876059416

model-00008-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ab2ed94385995c605ace5bd02e427ddd14ff5a5e151fda6e44dcf311c0aebf98
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:427f6007c3b883358b63d223b592259228b4fafcac12d554c60cf91c4c8a4d0f
 size 4876059416

model-00009-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:58051c244163c425f8c279389032d9cf9a4a4ccca6ff0567ab42559598891f74
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:dc8198dfc567e916b8faf7c42cf30c481eab92650caba4a1bfcfe8f212a1c893
 size 4876059416

model-00010-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0cea9345b8a067ba7bdff4b39af9a023c18377628a223ccb58c0cd8f5a233ffd
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:18b0dece8e5233b9b9a6f56285c28a2f0466a40ae3b8ec4976928fd6a75a1d3f
 size 4876059416

model-00011-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7ea1a93df43c76dd0ec61ab58b0c72dfe0322ee97591638a7a4a646df8042cd4
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:60f382210505f0c50d95912c0606ce39965412b091319843bd7b8fd8a036eaec
 size 4876059416

model-00012-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f4e02b4c5d445fcc5671e5c67c177158394671109f4ea53992dff26f5348eb06
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:83462b5d7f34665bca3b00e7961ac7c67efcca2182801ef6d1c1683fea1e4e09
 size 4876059416

model-00013-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:991346ed223475cb42f10c693d4f7ea2dc29467d53e112f4658cbe65fe5249bb
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:fec2defba67ade305e109d0e3acd7883f883958459507b756ce17861a1da09d8
 size 4876059416

model-00014-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8ac5c2a3ba5fa9b03380723b37e184919cc8f1bddee0c6c175398c8cfa3db8d9
 size 2123397800

 version https://git-lfs.github.com/spec/v1
+oid sha256:3ce5ff1b4efd0504c82bdee99473dd6015b339426d9a822d05e7ba13f619c862
 size 2123397800