Instructions to use DataSnake/Wayfarer-12B-NVFP4-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DataSnake/Wayfarer-12B-NVFP4-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DataSnake/Wayfarer-12B-NVFP4-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DataSnake/Wayfarer-12B-NVFP4-FP8")
model = AutoModelForCausalLM.from_pretrained("DataSnake/Wayfarer-12B-NVFP4-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DataSnake/Wayfarer-12B-NVFP4-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DataSnake/Wayfarer-12B-NVFP4-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Wayfarer-12B-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DataSnake/Wayfarer-12B-NVFP4-FP8

SGLang

How to use DataSnake/Wayfarer-12B-NVFP4-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DataSnake/Wayfarer-12B-NVFP4-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Wayfarer-12B-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DataSnake/Wayfarer-12B-NVFP4-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Wayfarer-12B-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DataSnake/Wayfarer-12B-NVFP4-FP8 with Docker Model Runner:
```
docker model run hf.co/DataSnake/Wayfarer-12B-NVFP4-FP8
```

Wayfarer-12B-NVFP4-FP8

Commit History

Update README.md

35d2f46
verified

DataSnake commited on May 23

Add library name and link to paper/code repository (#1)

946da17

DataSnake

nielsr HF Staff commited on May 23

Update README.md

ff89221
verified

DataSnake commited on Apr 27

Update README.md

7168e43
verified

DataSnake commited on Apr 8

Update README.md

f744bb9
verified

DataSnake commited on Apr 8

Update README.md

00b0866
verified

DataSnake commited on Apr 8

Update README.md

948bfd4
verified

DataSnake commited on Apr 5

Update README.md

7b6497d
verified

DataSnake commited on Apr 5

Update README.md

9909631
verified

DataSnake commited on Apr 5

Update README.md

c85aed9
verified

DataSnake commited on Apr 5

Upload 11 files

ab083b9
verified

DataSnake commited on Apr 4

initial commit

1d1c10c
verified

DataSnake commited on Apr 4

Commit History

Update README.md 35d2f46 verified

Add library name and link to paper/code repository (#1) 946da17

Update README.md ff89221 verified

Update README.md 7168e43 verified

Update README.md f744bb9 verified

Update README.md 00b0866 verified

Update README.md 948bfd4 verified

Update README.md 7b6497d verified

Update README.md 9909631 verified

Update README.md c85aed9 verified

Upload 11 files ab083b9 verified

initial commit 1d1c10c verified

Update README.md

35d2f46
verified

Add library name and link to paper/code repository (#1)

946da17

Update README.md

ff89221
verified

Update README.md

7168e43
verified

Update README.md

f744bb9
verified

Update README.md

00b0866
verified

Update README.md

948bfd4
verified

Update README.md

7b6497d
verified

Update README.md

9909631
verified

Update README.md

c85aed9
verified

Upload 11 files

ab083b9
verified

initial commit

1d1c10c
verified