Instructions to use TianshengHuang/s1k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TianshengHuang/s1k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TianshengHuang/s1k")
model = AutoModelForCausalLM.from_pretrained("TianshengHuang/s1k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TianshengHuang/s1k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TianshengHuang/s1k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TianshengHuang/s1k

SGLang

How to use TianshengHuang/s1k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TianshengHuang/s1k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TianshengHuang/s1k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TianshengHuang/s1k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use TianshengHuang/s1k with Docker Model Runner:
```
docker model run hf.co/TianshengHuang/s1k
```

TianshengHuang commited on Feb 26, 2025

Commit

9de244d

verified ·

1 Parent(s): 73f3737

Upload Qwen2ForCausalLM

Browse files

Files changed (14) hide show

model-00001-of-00014.safetensors +1 -1
model-00002-of-00014.safetensors +1 -1
model-00003-of-00014.safetensors +1 -1
model-00004-of-00014.safetensors +1 -1
model-00005-of-00014.safetensors +1 -1
model-00006-of-00014.safetensors +1 -1
model-00007-of-00014.safetensors +1 -1
model-00008-of-00014.safetensors +1 -1
model-00009-of-00014.safetensors +1 -1
model-00010-of-00014.safetensors +1 -1
model-00011-of-00014.safetensors +1 -1
model-00012-of-00014.safetensors +1 -1
model-00013-of-00014.safetensors +1 -1
model-00014-of-00014.safetensors +1 -1

model-00001-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7ff8099197c9463c2ce67671ded305f9794e1a9800894c19cf8c6cc1824cf65a
 size 4891730992

 version https://git-lfs.github.com/spec/v1
+oid sha256:941080803af1f10aa8b46d808061b079a4bb4cb33b095d8ac288e11dc1b7570b
 size 4891730992

model-00002-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ae2344787996ee583f1f361ea6d9f2fd0841c01d410a0a1afcd58c891cfa699c
 size 4876059352

 version https://git-lfs.github.com/spec/v1
+oid sha256:bba0d934f11650f18fb9ecb83af89d894de1a983080136168255dcee5eaffdf2
 size 4876059352

model-00003-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:80d7fa9860bea092c147fc17aa207f4e793425f68e820e0122fce3dca30838fa
 size 4876059384

 version https://git-lfs.github.com/spec/v1
+oid sha256:e75104789af91a302dbefd35d257f62767b53de4da2bc910f0b2100b422e0f81
 size 4876059384

model-00004-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a36add0889d6d71e462ca30fd438daea0dfac0bf6e8b35dcac40eee5ede9dd35
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:78c3f28f9210d9f3e9996b3973d3c7289d802fa89d7da30028ca8ef4bf04ab49
 size 4876059416

model-00005-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b61321839527c906f2f22336fc07035f2c2467e204eda64d5acb15204af34e1d
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:1f6e5a2f23a8aa52192e06e1914b550acad47e51c324ec9b8c977cee402ce83e
 size 4876059416

model-00006-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e72b13bd9b2932a8b24de9b1d5f43b6fbae50976c1fa55829fff3962e283387e
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:3f390393169ca4680e2b108343d2606a27a35dfb2278107bc8985f8fc638682b
 size 4876059416

model-00007-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8da61cc14f4b14f49f82caaf878b7d633b8aae41acd99d88ea54f19777e423a1
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:f38e4ea55733303b37f826eef321cf1dd252af4f7c9143dc05033ae79f984b86
 size 4876059416

model-00008-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:427f6007c3b883358b63d223b592259228b4fafcac12d554c60cf91c4c8a4d0f
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:e1c9b9b1ab76ca89912079b2e50657b6a943275eccebb66b846230c7ca20614d
 size 4876059416

model-00009-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dc8198dfc567e916b8faf7c42cf30c481eab92650caba4a1bfcfe8f212a1c893
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:0c8753558acaf028b284fc9ee3d9c54ffe31a86d6e2a3aeeb04112649a6531bb
 size 4876059416

model-00010-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:18b0dece8e5233b9b9a6f56285c28a2f0466a40ae3b8ec4976928fd6a75a1d3f
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:5759160c46dd2d06c5fd080b9cc181f5b8c76cdebd528e36fd40f1a36845de44
 size 4876059416

model-00011-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:60f382210505f0c50d95912c0606ce39965412b091319843bd7b8fd8a036eaec
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:3bfa2e5d569e506c40213b849d3341f2f771e5966f963248c840af4c4522a5ea
 size 4876059416

model-00012-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:83462b5d7f34665bca3b00e7961ac7c67efcca2182801ef6d1c1683fea1e4e09
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:1be75026843f4b4b5736a0e80b1c6ccb687d3db05869c2115acfdfb8b6bb7d2a
 size 4876059416

model-00013-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fec2defba67ade305e109d0e3acd7883f883958459507b756ce17861a1da09d8
 size 4876059416

 version https://git-lfs.github.com/spec/v1
+oid sha256:cbe6a984a50104c15cea08956c94e6b382d92e519a9685815404c454fd9fc4af
 size 4876059416

model-00014-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3ce5ff1b4efd0504c82bdee99473dd6015b339426d9a822d05e7ba13f619c862
 size 2123397800

 version https://git-lfs.github.com/spec/v1
+oid sha256:08717bbc4a8c16807cc8a820a9f6e02a8bed8288e6d9e9370b0dad5ded474a57
 size 2123397800