Instructions to use QuantTrio/GLM-5.1-AWQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use QuantTrio/GLM-5.1-AWQ with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="QuantTrio/GLM-5.1-AWQ")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("QuantTrio/GLM-5.1-AWQ")
model = AutoModelForCausalLM.from_pretrained("QuantTrio/GLM-5.1-AWQ")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use QuantTrio/GLM-5.1-AWQ with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "QuantTrio/GLM-5.1-AWQ"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QuantTrio/GLM-5.1-AWQ",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/QuantTrio/GLM-5.1-AWQ

SGLang

How to use QuantTrio/GLM-5.1-AWQ with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "QuantTrio/GLM-5.1-AWQ" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QuantTrio/GLM-5.1-AWQ",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "QuantTrio/GLM-5.1-AWQ" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QuantTrio/GLM-5.1-AWQ",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use QuantTrio/GLM-5.1-AWQ with Docker Model Runner:
```
docker model run hf.co/QuantTrio/GLM-5.1-AWQ
```

JunHowie commited on Apr 21

Commit

a0e0ba7

verified ·

1 Parent(s): 5973981

Add files using upload-large-folder tool

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

.gitattributes +1 -0
model-00001-of-00141.safetensors +3 -0
model-00011-of-00141.safetensors +3 -0
model-00020-of-00141.safetensors +3 -0
model-00021-of-00141.safetensors +3 -0
model-00024-of-00141.safetensors +3 -0
model-00028-of-00141.safetensors +3 -0
model-00039-of-00141.safetensors +3 -0
model-00044-of-00141.safetensors +3 -0
model-00048-of-00141.safetensors +3 -0
model-00049-of-00141.safetensors +3 -0
model-00051-of-00141.safetensors +3 -0
model-00052-of-00141.safetensors +3 -0
model-00053-of-00141.safetensors +3 -0
model-00054-of-00141.safetensors +3 -0
model-00055-of-00141.safetensors +3 -0
model-00056-of-00141.safetensors +3 -0
model-00061-of-00141.safetensors +3 -0
model-00062-of-00141.safetensors +3 -0
model-00063-of-00141.safetensors +3 -0
model-00064-of-00141.safetensors +3 -0
model-00065-of-00141.safetensors +3 -0
model-00066-of-00141.safetensors +3 -0
model-00067-of-00141.safetensors +3 -0
model-00068-of-00141.safetensors +3 -0
model-00072-of-00141.safetensors +3 -0
model-00078-of-00141.safetensors +3 -0
model-00079-of-00141.safetensors +3 -0
model-00084-of-00141.safetensors +3 -0
model-00088-of-00141.safetensors +3 -0
model-00089-of-00141.safetensors +3 -0
model-00090-of-00141.safetensors +3 -0
model-00093-of-00141.safetensors +3 -0
model-00094-of-00141.safetensors +3 -0
model-00095-of-00141.safetensors +3 -0
model-00096-of-00141.safetensors +3 -0
model-00097-of-00141.safetensors +3 -0
model-00101-of-00141.safetensors +3 -0
model-00103-of-00141.safetensors +3 -0
model-00104-of-00141.safetensors +3 -0
model-00105-of-00141.safetensors +3 -0
model-00106-of-00141.safetensors +3 -0
model-00107-of-00141.safetensors +3 -0
model-00118-of-00141.safetensors +3 -0
model-00129-of-00141.safetensors +3 -0
model-00131-of-00141.safetensors +3 -0
model-00132-of-00141.safetensors +3 -0
model-00133-of-00141.safetensors +3 -0
model-00134-of-00141.safetensors +3 -0
model-00141-of-00141.safetensors +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

model-00001-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bcfb0a483c95e00f83719379971ab8f0e00f25e61e075b9894bff0f1c179f0f6
+size 2993793472

model-00011-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1f3d96f8d3678e2ecea76b45d6c48fd190becf4d637d38d0e8558692a9dd11cb
+size 2994212672

model-00020-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e14f3214c7a093911fc1f0110dfaa852688d744ebc6acf5110b0b09d629bb982
+size 2995959528

model-00021-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cbb7812ca7333408109dcb388fee37c688c94f6d35edef74b17f816d83a879ab
+size 2999875224

model-00024-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:624f8521527ff4e549ce7e0857fe19c02f49727b169fe8bd0bd469ccffb8e151
+size 2994213784

model-00028-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9fae58669f172dfed406173625c69c01834d5fcff6a02cc5096dd4412c314292
+size 2999532328

model-00039-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d7a1398575b42115abbe047fbf0bb2d05565bfe2a30ba9354b0631807919daa9
+size 2992576552

model-00044-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9409c3b7edc8e03208cd11557de282504a3b7506a2a1fb302d03e33badb9908b
+size 2994213784

model-00048-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8c99ff8851dd1838db1cff1148c08fa905c5f825dd19fbe0bb334cb5b38ccbe6
+size 2998537240

model-00049-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:98dab9961004819c42f82753b62092a5551317d9b7e57ccefca50637839aa8fa
+size 2995552016

model-00051-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b6f563924397b9b76f46ad5d7959cd3b0def86c2e98595c49e557ba5c4383625
+size 2994214040

model-00052-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e5ff22ee6898432ee2109096925f801ecefb2187702da224d9b26fb2283b0287
+size 2999875216

model-00053-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:08c7983d7199d047da3fe40daaa2774ccff625b26f7a2a4f11550d3e4716ae65
+size 2994213928

model-00054-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:edec6491be250bcd817e04a86228760c72c712911ab0cd593c4a5dd9752e7f64
+size 2999875344

model-00055-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4ed51780ff17d3efc0b0eca092c27e0375801e0f49c1155f4f74d018cfa61466
+size 2994213672

model-00056-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5aee007cbfe093a509daab16ced8ad06c448291b27dab2edd1f6b950ed9fb3c9
+size 2999875592

model-00061-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2a1487ff118adf4cfc3d2823cf84842222ed47b8cb0845f65a3e7a60b596f854
+size 3000071960

model-00062-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c51b164d8939f28f0a1810f53bf7c187d10542df40dbc8469ae629387844224a
+size 2994214048

model-00063-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:547a9b28d1366b9b09098ca9e084ba06e55eb3c1f9627e54fc21b50fc7679e25
+size 2999875216

model-00064-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:250b68c62b12e8c7ab41f4f76160ab14cb10c11c8d00bc01e1e013ded8517466
+size 2994213880

model-00065-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c525664b6e037839af84b7578a0e84abc8e9011a9c10fff209d00c43260921fd
+size 2999875392

model-00066-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b5e7bdac0aee3689dff8b00b647a57e3a82a5a00cce38eb640cb6a144aadccb3
+size 2994213632

model-00067-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:af6f90ee8e22a770f22d24c7526c8b20bec44322a9c81b427a0ee69044d19928
+size 2999875632

model-00068-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1c7c3b23b58100d851b773d6c8c1be48c0b84947c343ec890fba50aca0cee180
+size 2994213384

model-00072-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:56e0f184c5355cb30b24ff846308b0aa72b6d5ae894c88c57e7604ede168256b
+size 2999875224

model-00078-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f25e79a245e5c1585ff5e17082c7228234edad8306c0db8a04900c3eb0c90127
+size 2999875736

model-00079-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cfd54ab4094a7a7a4fd1a9aa2e22ffdcdc60b80605ca1ec38ab9aa8a0c418b23
+size 2999532328

model-00084-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:af3e8f38dd6f1832d70c1760bfdd636df7331703827baf6f9a26a2844aecf69d
+size 2994213912

model-00088-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f6e90f7a63fd54c6ad3fdf6ccca75344d5ce51d632f36f26f568cf2336afbd86
+size 2994213416

model-00089-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d301b8c40b557e443696f6a6bd5b832143569dd83ab90ee8cffed22006aaea4e
+size 2999875840

model-00090-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4fc7f404955d31c15b5a1b0ec6bee046ed3165fc83a406589726220e9c8a6906
+size 2992576552

model-00093-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2e87cd33d8128914562b346eecfbb7f655dc15199cf2421a81927ae031f0094c
+size 2994214040

model-00094-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:539f8f860de8016db44b2112c4a0b9031c0d5e45bbe5a148d54d4fe73e33d16f
+size 2999875224

model-00095-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:57068596c059a0e30980d9e9155f5a0b20e638f4a6aad6a55e67f5418c7eda5f
+size 2994213784

model-00096-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:54071af8436eb728ddf83b42fbdf37bde449bf0c3559415c45cfb206a8d4c751
+size 2999875480

model-00097-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:45f4fc7f6cc6437a166637ac9ff3535d0fff80431272ddec2432a9c430fea1e1
+size 2994213536

model-00101-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:30184c068ea2ba57f55cd7bcc95d02f765b5993079f0dcac499ee09f855f14d7
+size 2999875216

model-00103-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d7ec27da934519ca0c91fcb8abe8a2a47c8d46f36ee6ec2a87b847d99fcb5d10
+size 2999875216

model-00104-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3ba357da1b77751e908efe4c33f0c21b2a1761f51b75efeae90d206d314e9a95
+size 2994213928

model-00105-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:27dfc509a99c76b57eb61b125db6a796f45e4fd55047e32a0c00af0fc626d789
+size 2999875344

model-00106-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1f98d82ff12d12083624bb59dff1a4e5bed640786f6df4c0847e6e574b2ab9ef
+size 2994213672

model-00107-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5b483020cf290ee4c26f61a911e482554eb0cc574cb5500fc1d071a3373f61e9
+size 2999875592

model-00118-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:221c91ead1812cf923868e6a7c0ed51f9019071d56a2b0759db8d234b771af10
+size 2999875632

model-00129-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cffa8933093c137fd3fcf83509570fdc25310aa00ffedcfee4dec69d948b71ce
+size 2999875736

model-00131-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ced75a3f239c9d2cb2616a15e4af3e68299820aa345d5c166841e89882fb88d8
+size 2994556920

model-00132-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e343365f0a42e4e9709095aae1eb18e28ea2fd49e307e4eea3fe7bc01c98236f
+size 2999875216

model-00133-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e804053aa79d4cf1c85730a898bd3d6f0086a35b5f0f61cf65a316f312bd383c
+size 2994214048

model-00134-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cb1bea5eb6baea82ec9edf5ebb35a4dd82b98d7f3a928d933f025b9d51b0562d
+size 2999875216

model-00141-of-00141.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2765668bc6de99c49fa46706cbbf360cd29e56c4dff78718c3e67a8873fbd269
+size 2054210192