Instructions to use cyijun2k/deepseek-v4-tiny-random-nvfp4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cyijun2k/deepseek-v4-tiny-random-nvfp4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cyijun2k/deepseek-v4-tiny-random-nvfp4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cyijun2k/deepseek-v4-tiny-random-nvfp4")
model = AutoModelForCausalLM.from_pretrained("cyijun2k/deepseek-v4-tiny-random-nvfp4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cyijun2k/deepseek-v4-tiny-random-nvfp4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cyijun2k/deepseek-v4-tiny-random-nvfp4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyijun2k/deepseek-v4-tiny-random-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cyijun2k/deepseek-v4-tiny-random-nvfp4

SGLang

How to use cyijun2k/deepseek-v4-tiny-random-nvfp4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cyijun2k/deepseek-v4-tiny-random-nvfp4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyijun2k/deepseek-v4-tiny-random-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cyijun2k/deepseek-v4-tiny-random-nvfp4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyijun2k/deepseek-v4-tiny-random-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use cyijun2k/deepseek-v4-tiny-random-nvfp4 with Docker Model Runner:
```
docker model run hf.co/cyijun2k/deepseek-v4-tiny-random-nvfp4
```

DeepSeek-V4 Tiny Random NVFP4

This is a randomly initialized, tiny debugging checkpoint with the DeepSeek-V4 architecture. It is not a trained language model and must not be used to evaluate model quality.

It was converted from yujiepan/deepseek-v4-tiny-random at revision 082d9c80d8e5b3f1b81e4a18e0a22daad84537af to exercise NVFP4 fused-MoE kernels on NVIDIA Blackwell GPUs.

Quantization layout

All routed expert w1, w2, and w3 weights in the 7 transformer layers were converted from MXFP4 group-32 to NVFP4 group-16.
The routed-expert tensors use the ModelOpt-compatible NVFP4 schema: packed uint8 weights, FP8 E4M3 block scales, and scalar global/input scales.
Dense FP8 tensors, shared experts, embeddings, the LM head, and MTP tensors are unchanged from the source mock checkpoint.
The model config uses activation="silu" and swiglu_limit=10.0, requiring the clamped SwiGLU semantics:
```
gate = min(gate, 10.0)
up   = clamp(up, -10.0, 10.0)
out  = silu(gate) * up
```

This is therefore a mixed checkpoint intended specifically for routed-expert NVFP4 testing, not a claim that every linear layer is NVFP4.

Validation

The checkpoint was tested on an NVIDIA GB10 (SM121) with:

vLLM nightly v0.26.1rc1.dev306+gcb8104839
FlashInfer 0.6.18 plus the B12X SiLU clamp fix
forced FLASHINFER_B12X NVFP4 MoE backend
activation="silu", swiglu_limit=10.0

All seven B12X MoE wrappers were constructed with the limit and an end-to-end prefill plus two decode steps completed. The tested vLLM nightly predates the B12X clamp capability declaration and parameter forwarding, so it required an in-process adapter bridge. This is a runtime integration limitation, not part of the checkpoint.

Some non-MoE DeepSeek-V4 tiny-shape paths also required reference fallbacks on GB10. The checkpoint should be treated as an expert-kernel/debugging fixture, not as a drop-in production serving model.

Integrity

model.safetensors SHA256:

9859140eff50a9dbb0454aa5d008418dfbdbccb2dce3a5858d252bb125aa37b3

See CONVERSION_INFO.json for the source revision and conversion summary.

Limitations

Random weights; generated text is meaningless.
Intended for architecture, loader, quantization, and fused-MoE testing only.
Requires software with DeepSeek-V4 and ModelOpt-style mixed-precision checkpoint support.
B12X execution requires an SM120/SM121 GPU and a FlashInfer version that applies swiglu_limit to standard SiLU.

Downloads last month: 55

Safetensors

Model size

0.3B params

Tensor type

I64

F32

BF16

F8_E4M3

Model tree for cyijun2k/deepseek-v4-tiny-random-nvfp4

Base model

deepseek-ai/DeepSeek-V4-Pro

Quantized

yujiepan/deepseek-v4-tiny-random

Quantized

(1)

this model