Instructions to use DataSnake/Wayfarer-2-12B-NVFP4-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DataSnake/Wayfarer-2-12B-NVFP4-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DataSnake/Wayfarer-2-12B-NVFP4-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DataSnake/Wayfarer-2-12B-NVFP4-FP8")
model = AutoModelForCausalLM.from_pretrained("DataSnake/Wayfarer-2-12B-NVFP4-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use DataSnake/Wayfarer-2-12B-NVFP4-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DataSnake/Wayfarer-2-12B-NVFP4-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Wayfarer-2-12B-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DataSnake/Wayfarer-2-12B-NVFP4-FP8

SGLang

How to use DataSnake/Wayfarer-2-12B-NVFP4-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DataSnake/Wayfarer-2-12B-NVFP4-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Wayfarer-2-12B-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DataSnake/Wayfarer-2-12B-NVFP4-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Wayfarer-2-12B-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DataSnake/Wayfarer-2-12B-NVFP4-FP8 with Docker Model Runner:
```
docker model run hf.co/DataSnake/Wayfarer-2-12B-NVFP4-FP8
```

Wayfarer-2-12B-NVFP4-FP8

Commit History

Add library name, code link and citation (#1)

9490e54

DataSnake

nielsr HF Staff commited on 9 days ago

Update README.md

0b21ab2
verified

DataSnake commited on Apr 27

Update README.md

ec95a21
verified

DataSnake commited on Apr 8

Update README.md

6a5b04c
verified

DataSnake commited on Apr 8

Update README.md

38aeba1
verified

DataSnake commited on Apr 8

Update README.md

b95379f
verified

DataSnake commited on Apr 5

Update README.md

c6263b9
verified

DataSnake commited on Apr 5

Update README.md

3e3791c
verified

DataSnake commited on Apr 5

Update README.md

42f10ff
verified

DataSnake commited on Apr 5

Upload 11 files

68e8b94
verified

DataSnake commited on Apr 4

initial commit

024682d
verified

DataSnake commited on Apr 4

Commit History

Add library name, code link and citation (#1) 9490e54

Update README.md 0b21ab2 verified

Update README.md ec95a21 verified

Update README.md 6a5b04c verified

Update README.md 38aeba1 verified

Update README.md b95379f verified

Update README.md c6263b9 verified

Update README.md 3e3791c verified

Update README.md 42f10ff verified

Upload 11 files 68e8b94 verified

initial commit 024682d verified

Add library name, code link and citation (#1)

9490e54

Update README.md

0b21ab2
verified

Update README.md

ec95a21
verified

Update README.md

6a5b04c
verified

Update README.md

38aeba1
verified

Update README.md

b95379f
verified

Update README.md

c6263b9
verified

Update README.md

3e3791c
verified

Update README.md

42f10ff
verified

Upload 11 files

68e8b94
verified

initial commit

024682d
verified