Instructions to use open-machine/Qwen3-8B-FlashNorm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use open-machine/Qwen3-8B-FlashNorm with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="open-machine/Qwen3-8B-FlashNorm")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("open-machine/Qwen3-8B-FlashNorm")
model = AutoModelForCausalLM.from_pretrained("open-machine/Qwen3-8B-FlashNorm")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use open-machine/Qwen3-8B-FlashNorm with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "open-machine/Qwen3-8B-FlashNorm"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "open-machine/Qwen3-8B-FlashNorm",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/open-machine/Qwen3-8B-FlashNorm

SGLang

How to use open-machine/Qwen3-8B-FlashNorm with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "open-machine/Qwen3-8B-FlashNorm" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "open-machine/Qwen3-8B-FlashNorm",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "open-machine/Qwen3-8B-FlashNorm" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "open-machine/Qwen3-8B-FlashNorm",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use open-machine/Qwen3-8B-FlashNorm with Docker Model Runner:
```
docker model run hf.co/open-machine/Qwen3-8B-FlashNorm
```

nielsr HF Staff commited on Apr 23

Commit

c0e2238

verified ·

1 Parent(s): 90957a6

Add library_name and paper link

Browse files

Hi! I'm Niels from the community science team at Hugging Face. I'm opening this PR to improve the model card for your FlashNorm checkpoint.

Changes include:
- Added `library_name: transformers` to the metadata to enable the "Use in Transformers" snippet and button on the Hub.
- Explicitly linked the research paper [FlashNorm: Fast Normalization for Transformers](https://huggingface.co/papers/2407.09577).
- Added a BibTeX citation section for the paper.
- Preserved existing usage instructions and technical details.

Files changed (1) hide show

README.md +26 -9

README.md CHANGED Viewed

@@ -1,17 +1,20 @@
 ---
-license: apache-2.0
 base_model: Qwen/Qwen3-8B
-tags:
-  - flashnorm
-  - transformer-tricks
-  - efficient-inference
-  - weightless-rmsnorm
 pipeline_tag: text-generation
 ---
 # Qwen3-8B-FlashNorm
-FlashNorm-prepared checkpoint of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B). Mathematically equivalent to the source model. The per-channel RMSNorm weight tensors (`input_layernorm.weight`, `post_attention_layernorm.weight`, `model.norm.weight`) are folded into the following linear layers and then removed from the state dict entirely.
 > **Framework support note.** Stock vLLM currently does not load this checkpoint because the norm weight tensors are absent. The upstream patch to accept missing tensors is tracked at: **TBD (vLLM issue link)**. Until the patch lands, use HuggingFace Transformers; it loads this with a warning that norm weights were not initialized and defaults them to ones, which is the correct behavior for FlashNorm.
@@ -23,7 +26,7 @@ An exact reformulation of `RMSNorm -> Linear`:
 - After folding, the RMSNorm layer has no learnable per-channel scale. At runtime it simply divides by `rms(x)`.
 - The resulting model computes the same output as the original, by Proposition 1 of the FlashNorm paper.
-See the [paper](https://arxiv.org/abs/2407.09577) and the [transformer-tricks](https://github.com/OpenMachine-ai/transformer-tricks) repo for details.
 ## Usage
@@ -53,6 +56,20 @@ A warning about missing norm weights is expected; Transformers defaults those to
 Not yet supported. See the tracking issue linked above.
 ## License
-Inherited from the source model.

 ---
 base_model: Qwen/Qwen3-8B
+license: apache-2.0
 pipeline_tag: text-generation
+library_name: transformers
+tags:
+- flashnorm
+- transformer-tricks
+- efficient-inference
+- weightless-rmsnorm
 ---
 # Qwen3-8B-FlashNorm
+This is a FlashNorm-prepared checkpoint of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B), as presented in the paper [FlashNorm: Fast Normalization for Transformers](https://huggingface.co/papers/2407.09577).
+Mathematically equivalent to the source model. The per-channel RMSNorm weight tensors (`input_layernorm.weight`, `post_attention_layernorm.weight`, `model.norm.weight`) are folded into the following linear layers and then removed from the state dict entirely.
 > **Framework support note.** Stock vLLM currently does not load this checkpoint because the norm weight tensors are absent. The upstream patch to accept missing tensors is tracked at: **TBD (vLLM issue link)**. Until the patch lands, use HuggingFace Transformers; it loads this with a warning that norm weights were not initialized and defaults them to ones, which is the correct behavior for FlashNorm.
 - After folding, the RMSNorm layer has no learnable per-channel scale. At runtime it simply divides by `rms(x)`.
 - The resulting model computes the same output as the original, by Proposition 1 of the FlashNorm paper.
+See the [paper](https://huggingface.co/papers/2407.09577) and the [transformer-tricks](https://github.com/OpenMachine-ai/transformer-tricks) repo for details.
 ## Usage
 Not yet supported. See the tracking issue linked above.
+## Citation
+```bibtex
+@misc{graef2024flashnormfastnormalizationtransformers,
+      title={FlashNorm: Fast Normalization for Transformers},
+      author={Nils Graef and Matthew Clapp and Andrew Wasielewski},
+      year={2024},
+      eprint={2407.09577},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG},
+      url={https://arxiv.org/abs/2407.09577},
+}
+```
 ## License
+Inherited from the source model.