cvoegele-nv commited on
Commit
197d946
·
verified ·
1 Parent(s): 266e9c6

Add benchmarking parameters (T=1.0, top_p=0.95, max num tokens 65536) and MXFP8 baseline note to Evaluation section

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -136,6 +136,8 @@ vllm serve nvidia/MiniMax-M3-NVFP4 \
136
  | FP8 | **92.53** | **76.62** | **92.22** | **71.97** | **49.90** |
137
  | NVFP4 | **91.92** | **75.60** | **91.89** | **71.01** | **49.70** |
138
 
 
 
139
  ## Model Limitations:
140
  The base model was trained on data that contains toxic language and societal biases originally crawled from the internet. Therefore, the model may amplify those biases and return toxic responses especially when prompted with toxic prompts. The model may generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive.
141
 
 
136
  | FP8 | **92.53** | **76.62** | **92.22** | **71.97** | **49.90** |
137
  | NVFP4 | **91.92** | **75.60** | **91.89** | **71.01** | **49.70** |
138
 
139
+ Baseline: MiniMax-M3 in its native MXFP8 format. Benchmarked with temperature=1.0, top_p=0.95, max num tokens 65536.
140
+
141
  ## Model Limitations:
142
  The base model was trained on data that contains toxic language and societal biases originally crawled from the internet. Therefore, the model may amplify those biases and return toxic responses especially when prompted with toxic prompts. The model may generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive.
143