Instructions to use DataSnake/Muse-12B-NVFP4-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DataSnake/Muse-12B-NVFP4-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DataSnake/Muse-12B-NVFP4-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DataSnake/Muse-12B-NVFP4-FP8")
model = AutoModelForCausalLM.from_pretrained("DataSnake/Muse-12B-NVFP4-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DataSnake/Muse-12B-NVFP4-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DataSnake/Muse-12B-NVFP4-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Muse-12B-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DataSnake/Muse-12B-NVFP4-FP8

SGLang

How to use DataSnake/Muse-12B-NVFP4-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DataSnake/Muse-12B-NVFP4-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Muse-12B-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DataSnake/Muse-12B-NVFP4-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Muse-12B-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DataSnake/Muse-12B-NVFP4-FP8 with Docker Model Runner:
```
docker model run hf.co/DataSnake/Muse-12B-NVFP4-FP8
```

Muse-12B-NVFP4-FP8

Commit History

Update README.md

1276e97
verified

DataSnake commited on May 23

Add paper link, library metadata, and code link (#1)

0d73821

DataSnake

nielsr HF Staff commited on May 23

Update README.md

22ec27a
verified

DataSnake commited on Apr 27

Update README.md

793f8c0
verified

DataSnake commited on Apr 8

Update README.md

4e5c90c
verified

DataSnake commited on Apr 8

Update README.md

5395954
verified

DataSnake commited on Apr 8

Update README.md

9ac2ac9
verified

DataSnake commited on Apr 5

Update README.md

fe4d954
verified

DataSnake commited on Apr 5

Update README.md

dd665f9
verified

DataSnake commited on Apr 5

Upload 11 files

353cba2
verified

DataSnake commited on Apr 4

initial commit

d3c9d60
verified

DataSnake commited on Apr 4

Commit History

Update README.md 1276e97 verified

Add paper link, library metadata, and code link (#1) 0d73821

Update README.md 22ec27a verified

Update README.md 793f8c0 verified

Update README.md 4e5c90c verified

Update README.md 5395954 verified

Update README.md 9ac2ac9 verified

Update README.md fe4d954 verified

Update README.md dd665f9 verified

Upload 11 files 353cba2 verified

initial commit d3c9d60 verified

Update README.md

1276e97
verified

Add paper link, library metadata, and code link (#1)

0d73821

Update README.md

22ec27a
verified

Update README.md

793f8c0
verified

Update README.md

4e5c90c
verified

Update README.md

5395954
verified

Update README.md

9ac2ac9
verified

Update README.md

fe4d954
verified

Update README.md

dd665f9
verified

Upload 11 files

353cba2
verified

initial commit

d3c9d60
verified