Add vLLM commands, remove calibration datasets
#3
by frida-a - opened
README.md
CHANGED
|
@@ -75,12 +75,6 @@ The model version is NVFP4 1.0 version and is quantized with nvidia-modelopt **v
|
|
| 75 |
We calibrated the model using the dataset noted below, and performed evaluation using the benchmarks noted under Evaluation Datasets.
|
| 76 |
We did not perform training or testing for this Model Optimizer release. The methods noted under Training and Testing Datasets below represent the data collection and labeling methods used by the third-party to train and test the underlying model.<br>
|
| 77 |
|
| 78 |
-
## Calibration Dataset:
|
| 79 |
-
**Link:** [Nemotron-SFT-Instruction-Following-Chat-v2](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Instruction-Following-Chat-v2), [Nemotron-Science-v1](https://huggingface.co/datasets/nvidia/Nemotron-Science-v1), [Nemotron-Competitive-Programming-v1](https://huggingface.co/datasets/nvidia/Nemotron-Competitive-Programming-v1), [Nemotron-SFT-Agentic-v2](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Agentic-v2), [Nemotron-Math-v2](https://huggingface.co/datasets/nvidia/Nemotron-Math-v2), [Nemotron-SFT-SWE-v2](https://huggingface.co/datasets/nvidia/Nemotron-SFT-SWE-v2), [Nemotron-SFT-Multilingual-v1](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Multilingual-v1) <br>
|
| 80 |
-
**Data Collection Method by dataset:** Hybrid: Human, Synthetic, Automated. <br>
|
| 81 |
-
**Labeling method:** Hybrid: Human, Automated. <br>
|
| 82 |
-
**Properties:** Nemotron-SFT-Instruction-Following-Chat-v2 contains ~2M synthetic chat samples designed to strengthen open-ended chat and precise instruction following capabilities. Nemotron-Science-v1 is a synthetic science reasoning dataset with ~226K samples covering GPQA-style science questions and chemistry problems to enhance LLM reasoning in scientific domains. Nemotron-Competitive-Programming-v1 is a large-scale synthetic coding dataset with 2M+ Python and 1M+ C++ samples spanning 34K+ competitive programming questions for code completion and critique. Nemotron-SFT-Agentic-v2 contains ~992K samples of tool-calling trajectories, customer service conversations, and web-search trajectories to train interactive, tool-using agents. Nemotron-Math-v2 is a large-scale mathematical reasoning dataset with ~347K problems and 7M model-generated reasoning trajectories across multiple reasoning modes and tool-use configurations. Nemotron-SFT-SWE-v2 contains ~256K software engineering samples including agentic SWE trajectories and agentless code localization, repair, and test generation samples for SWE-Bench style tasks. Nemotron-SFT-Multilingual-v1 contains ~3M multilingual reasoning samples translated from math, code, and STEM data into German, French, Japanese, Italian, Chinese, and Spanish. <br>
|
| 83 |
-
|
| 84 |
## Training Dataset:
|
| 85 |
**Data Modality:** Undisclosed <br>
|
| 86 |
**Data Collection Method by dataset:** Undisclosed <br>
|
|
@@ -125,6 +119,22 @@ python3 -m sglang.launch_server \
|
|
| 125 |
--mem-fraction-static 0.80
|
| 126 |
```
|
| 127 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 128 |
## Evaluation
|
| 129 |
The accuracy benchmark results are presented in the table below. AA-LCR was measured with SGLang; all other benchmarks were measured with vLLM.
|
| 130 |
<table>
|
|
|
|
| 75 |
We calibrated the model using the dataset noted below, and performed evaluation using the benchmarks noted under Evaluation Datasets.
|
| 76 |
We did not perform training or testing for this Model Optimizer release. The methods noted under Training and Testing Datasets below represent the data collection and labeling methods used by the third-party to train and test the underlying model.<br>
|
| 77 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
## Training Dataset:
|
| 79 |
**Data Modality:** Undisclosed <br>
|
| 80 |
**Data Collection Method by dataset:** Undisclosed <br>
|
|
|
|
| 119 |
--mem-fraction-static 0.80
|
| 120 |
```
|
| 121 |
|
| 122 |
+
### vLLM
|
| 123 |
+
|
| 124 |
+
To serve this checkpoint with [vLLM](https://github.com/vllm-project/vllm), use the `vllm/vllm-openai:v0.23.0` image and run:
|
| 125 |
+
|
| 126 |
+
```sh
|
| 127 |
+
vllm serve nvidia/GLM-5.2-NVFP4 \
|
| 128 |
+
--tensor-parallel-size 8 \
|
| 129 |
+
--enable-expert-parallel \
|
| 130 |
+
--trust-remote-code \
|
| 131 |
+
--reasoning-parser glm45 \
|
| 132 |
+
--tool-call-parser glm47 \
|
| 133 |
+
--enable-auto-tool-choice \
|
| 134 |
+
--kv-cache-dtype fp8_e4m3 \
|
| 135 |
+
--host 0.0.0.0 --port 8000
|
| 136 |
+
```
|
| 137 |
+
|
| 138 |
## Evaluation
|
| 139 |
The accuracy benchmark results are presented in the table below. AA-LCR was measured with SGLang; all other benchmarks were measured with vLLM.
|
| 140 |
<table>
|