Add benchmarking parameters (T=1.0, top_p=0.95, max num tokens 65536) and MXFP8 baseline note to Evaluation section
Browse files
README.md
CHANGED
|
@@ -136,6 +136,8 @@ vllm serve nvidia/MiniMax-M3-NVFP4 \
|
|
| 136 |
| FP8 | **92.53** | **76.62** | **92.22** | **71.97** | **49.90** |
|
| 137 |
| NVFP4 | **91.92** | **75.60** | **91.89** | **71.01** | **49.70** |
|
| 138 |
|
|
|
|
|
|
|
| 139 |
## Model Limitations:
|
| 140 |
The base model was trained on data that contains toxic language and societal biases originally crawled from the internet. Therefore, the model may amplify those biases and return toxic responses especially when prompted with toxic prompts. The model may generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive.
|
| 141 |
|
|
|
|
| 136 |
| FP8 | **92.53** | **76.62** | **92.22** | **71.97** | **49.90** |
|
| 137 |
| NVFP4 | **91.92** | **75.60** | **91.89** | **71.01** | **49.70** |
|
| 138 |
|
| 139 |
+
Baseline: MiniMax-M3 in its native MXFP8 format. Benchmarked with temperature=1.0, top_p=0.95, max num tokens 65536.
|
| 140 |
+
|
| 141 |
## Model Limitations:
|
| 142 |
The base model was trained on data that contains toxic language and societal biases originally crawled from the internet. Therefore, the model may amplify those biases and return toxic responses especially when prompted with toxic prompts. The model may generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive.
|
| 143 |
|