Hugging Face's logo

llmat
/
Mistral-Small-24B-Instruct-2501-NVFP4

Text Generation
Safetensors
mistral
quantization
nvfp4
vllm
conversational
8-bit precision
compressed-tensors
Model card Files
xet
 Community
llmat commited on
Commit
d2b08cb
·
verified ·
1 Parent(s): 99dd71f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -38,7 +38,7 @@ from vllm import LLM, SamplingParams
38
  from transformers import AutoTokenizer
39
 
40
  model_id = "llmat/Mistral-Small-24B-Instruct-2501-NVFP4"
41
- number_gpus = 2
42
 
43
  sampling_params = SamplingParams(temperature=0.6, top_p=0.9, max_tokens=256)
44
 
 
38
  from transformers import AutoTokenizer
39
 
40
  model_id = "llmat/Mistral-Small-24B-Instruct-2501-NVFP4"
41
+ number_gpus = 1
42
 
43
  sampling_params = SamplingParams(temperature=0.6, top_p=0.9, max_tokens=256)
44