Instructions to use cyijun2k/glm-5.3-flash-tiny-random-nvfp4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cyijun2k/glm-5.3-flash-tiny-random-nvfp4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="cyijun2k/glm-5.3-flash-tiny-random-nvfp4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("cyijun2k/glm-5.3-flash-tiny-random-nvfp4")
model = AutoModelForMultimodalLM.from_pretrained("cyijun2k/glm-5.3-flash-tiny-random-nvfp4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cyijun2k/glm-5.3-flash-tiny-random-nvfp4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cyijun2k/glm-5.3-flash-tiny-random-nvfp4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyijun2k/glm-5.3-flash-tiny-random-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/cyijun2k/glm-5.3-flash-tiny-random-nvfp4

SGLang

How to use cyijun2k/glm-5.3-flash-tiny-random-nvfp4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cyijun2k/glm-5.3-flash-tiny-random-nvfp4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyijun2k/glm-5.3-flash-tiny-random-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cyijun2k/glm-5.3-flash-tiny-random-nvfp4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyijun2k/glm-5.3-flash-tiny-random-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use cyijun2k/glm-5.3-flash-tiny-random-nvfp4 with Docker Model Runner:
```
docker model run hf.co/cyijun2k/glm-5.3-flash-tiny-random-nvfp4
```

glm-5.3-flash-tiny-random-nvfp4 / VALIDATION.json

cyijun2k

Record public GB10 adapter image digest

8ff6065 verified 2 days ago

Raw

History Blame Contribute Delete

1.65 kB

	{
	"validated_at": "2026-08-27",
	"hardware": {
	"gpu": "NVIDIA GB10",
	"compute_capability": "SM121",
	"memory_gib": 121.69
	},
	"base_runtime": {
	"image": "vllm/vllm-openai:glm53-flash-arm64-cu130",
	"digest": "sha256:905c02933be6021301db2dc284e24e3727467aa3a0f63b41d609885778a07bce",
	"image_id": "sha256:d42649063a2b05810bce6e1462918475b4ae6e44ad56033735f7118d5c16f136",
	"vllm": "0.1.dev20051+g487ecf187",
	"flashinfer": "0.6.17",
	"transformers": "5.15.1",
	"torch": "2.13.0+cu130"
	},
	"schema": {
	"critical_config_fields_matched": 21,
	"mock_tensor_count": 5342,
	"analogous_target_tensor_names": 5342,
	"missing_target_tensor_names": 0,
	"nvfp4_weight_dtype": "U8",
	"nvfp4_scale_dtype": "F8_E4M3",
	"nvfp4_global_scale_dtype": "F32"
	},
	"runtime": {
	"adapter_image_runtime_tested_local_id": "sha256:7e79066623ddc0485c71cc01a181637cc287457d86c74676043d93bc478ea491",
	"published_image_local_id": "sha256:0d255dd6c0f4c797214a185706fc57cf982e85952945904414eaa83e15fc1e71",
	"published_image": "ghcr.io/cyijun/glm-5.3-flash-nvfp4-gb10@sha256:4251b561d111d817765ed4097512ce36811deac071a4a7411d20242df5c74a47",
	"published_source_revision": "5bb0a598829839a9e0c420c6b737a742e084948c",
	"published_manifest_visibility": "public",
	"health": "pass",
	"openai_models": "pass",
	"short_chat_decode": "pass",
	"prefill_491_tokens_and_decode_16_tokens": "pass",
	"mtp_one_speculative_token": "pass",
	"note": "Generated text is intentionally random; validation concerns loading, cache layout, CUDA dispatch, prefill, decode, and MTP execution."
	}
	}