Instructions to use DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8")
model = AutoModelForCausalLM.from_pretrained("DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8

SGLang

How to use DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8 with Docker Model Runner:
```
docker model run hf.co/DataSnake/Mistral-Nemo-Instruct-2407-NVFP4-FP8
```

Mistral-Nemo-Instruct-2407-NVFP4-FP8 / README.md

Commit History

Update README.md

9c6a437
verified

DataSnake commited on 7 days ago

Improve model card metadata and add paper/code links (#1)

84657dd

DataSnake

nielsr HF Staff commited on 7 days ago

Update README.md

b7b5666
verified

DataSnake commited on Apr 16

Update README.md

b4460c7
verified

DataSnake commited on Apr 10

Update README.md

f6a8de0
verified

DataSnake commited on Apr 10

Update README.md

061d252
verified

DataSnake commited on Apr 9

Update README.md

7bf2acd
verified

DataSnake commited on Apr 9

Update README.md

0f3477e
verified

DataSnake commited on Apr 9

Update README.md

bd8f6cc
verified

DataSnake commited on Apr 9

Update README.md

e69a9f9
verified

DataSnake commited on Apr 8

Update README.md

a024b1e
verified

DataSnake commited on Apr 8

Update README.md

b9628bb
verified

DataSnake commited on Apr 8

Update README.md

a8be161
verified

DataSnake commited on Apr 8

Update README.md

7c38d03
verified

DataSnake commited on Apr 8

Update README.md

a62ba4b
verified

DataSnake commited on Apr 8

Update README.md

eae8fbc
verified

DataSnake commited on Apr 8

Update README.md

dca2797
verified

DataSnake commited on Apr 8

Update README.md

6f2aaf8
verified

DataSnake commited on Apr 8

Update README.md

b17b8fb
verified

DataSnake commited on Apr 8

Update README.md

fa2b71b
verified

DataSnake commited on Apr 8

Update README.md

7f5387b
verified

DataSnake commited on Apr 8

Update README.md

bd80fb6
verified

DataSnake commited on Apr 8

Update README.md

6c5d5c6
verified

DataSnake commited on Apr 7

Update README.md

42be636
verified

DataSnake commited on Apr 6

Update README.md

7d9d4f3
verified

DataSnake commited on Apr 6

Update README.md

30d930d
verified

DataSnake commited on Apr 6

Update README.md

864ce67
verified

DataSnake commited on Apr 6

Update README.md

15df550
verified

DataSnake commited on Apr 5

Update README.md

2696cfe
verified

DataSnake commited on Apr 4

Update README.md

fc32d06
verified

DataSnake commited on Apr 4

Update README.md

2488a69
verified

DataSnake commited on Apr 4

Update README.md

751d79a
verified

DataSnake commited on Apr 4

Update README.md

6f07f11
verified

DataSnake commited on Apr 4

Update README.md

0fe8131
verified

DataSnake commited on Apr 2

Update README.md

d3becf8
verified

DataSnake commited on Apr 2

Update README.md

698901c
verified

DataSnake commited on Apr 1

Update README.md

6692468
verified

DataSnake commited on Mar 31

Update README.md

6e07260
verified

DataSnake commited on Mar 31

Update README.md

69ee3f4
verified

DataSnake commited on Mar 30

Update README.md

d1f08a0
verified

DataSnake commited on Mar 30

Update README.md

e7042e9
verified

DataSnake commited on Mar 29

Update README.md

3ce6504
verified

DataSnake commited on Mar 28

Update README.md

57de853
verified

DataSnake commited on Mar 28

Update README.md

257192a
verified

DataSnake commited on Mar 28

Update README.md

a6744d1
verified

DataSnake commited on Mar 28

Update README.md

0cc2704
verified

DataSnake commited on Mar 28

Update README.md

e1db118
verified

DataSnake commited on Mar 27

Update README.md

e1cfc63
verified

DataSnake commited on Mar 27

Update README.md

a8c15e7
verified

DataSnake commited on Mar 27

Update README.md

7b23ab3
verified

DataSnake commited on Mar 27

Commit History

Update README.md 9c6a437 verified

Improve model card metadata and add paper/code links (#1) 84657dd

Update README.md b7b5666 verified

Update README.md b4460c7 verified

Update README.md f6a8de0 verified

Update README.md 061d252 verified

Update README.md 7bf2acd verified

Update README.md 0f3477e verified

Update README.md bd8f6cc verified

Update README.md e69a9f9 verified

Update README.md a024b1e verified

Update README.md b9628bb verified

Update README.md a8be161 verified

Update README.md 7c38d03 verified

Update README.md a62ba4b verified

Update README.md eae8fbc verified

Update README.md dca2797 verified

Update README.md 6f2aaf8 verified

Update README.md b17b8fb verified

Update README.md fa2b71b verified

Update README.md 7f5387b verified

Update README.md bd80fb6 verified

Update README.md 6c5d5c6 verified

Update README.md 42be636 verified

Update README.md 7d9d4f3 verified

Update README.md 30d930d verified

Update README.md 864ce67 verified

Update README.md 15df550 verified

Update README.md 2696cfe verified

Update README.md fc32d06 verified

Update README.md 2488a69 verified

Update README.md 751d79a verified

Update README.md 6f07f11 verified

Update README.md 0fe8131 verified

Update README.md d3becf8 verified

Update README.md 698901c verified

Update README.md 6692468 verified

Update README.md 6e07260 verified

Update README.md 69ee3f4 verified

Update README.md d1f08a0 verified

Update README.md e7042e9 verified

Update README.md 3ce6504 verified

Update README.md 57de853 verified

Update README.md 257192a verified

Update README.md a6744d1 verified

Update README.md 0cc2704 verified

Update README.md e1db118 verified

Update README.md e1cfc63 verified

Update README.md a8c15e7 verified

Update README.md 7b23ab3 verified

Update README.md

9c6a437
verified

Improve model card metadata and add paper/code links (#1)

84657dd

Update README.md

b7b5666
verified

Update README.md

b4460c7
verified

Update README.md

f6a8de0
verified

Update README.md

061d252
verified

Update README.md

7bf2acd
verified

Update README.md

0f3477e
verified

Update README.md

bd8f6cc
verified

Update README.md

e69a9f9
verified

Update README.md

a024b1e
verified

Update README.md

b9628bb
verified

Update README.md

a8be161
verified

Update README.md

7c38d03
verified

Update README.md

a62ba4b
verified

Update README.md

eae8fbc
verified

Update README.md

dca2797
verified

Update README.md

6f2aaf8
verified

Update README.md

b17b8fb
verified

Update README.md

fa2b71b
verified

Update README.md

7f5387b
verified

Update README.md

bd80fb6
verified

Update README.md

6c5d5c6
verified

Update README.md

42be636
verified

Update README.md

7d9d4f3
verified

Update README.md

30d930d
verified

Update README.md

864ce67
verified

Update README.md

15df550
verified

Update README.md

2696cfe
verified

Update README.md

fc32d06
verified

Update README.md

2488a69
verified

Update README.md

751d79a
verified

Update README.md

6f07f11
verified

Update README.md

0fe8131
verified

Update README.md

d3becf8
verified

Update README.md

698901c
verified

Update README.md

6692468
verified

Update README.md

6e07260
verified

Update README.md

69ee3f4
verified

Update README.md

d1f08a0
verified

Update README.md

e7042e9
verified

Update README.md

3ce6504
verified

Update README.md

57de853
verified

Update README.md

257192a
verified

Update README.md

a6744d1
verified

Update README.md

0cc2704
verified

Update README.md

e1db118
verified

Update README.md

e1cfc63
verified

Update README.md

a8c15e7
verified

Update README.md

7b23ab3
verified