linzhao-amd commited on
Commit
f5f3e62
·
verified ·
1 Parent(s): 73a8ee8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -3
README.md CHANGED
@@ -51,7 +51,7 @@ This model can be deployed efficiently using the [SGLang](https://docs.sglang.ai
51
 
52
  ## Evaluation
53
 
54
- The model was evaluated on AIME24, GPQA Diamond, MATH-500, and GSM8K benchmarks. The tasks of AIME24, GPQA Diamond, and MATH-500 were conducted using the [lighteval](https://github.com/huggingface/lighteval/tree/v0.10.0) framework while GSM8K using [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness).
55
 
56
  ### Accuracy
57
 
@@ -111,8 +111,7 @@ The model was evaluated on AIME24, GPQA Diamond, MATH-500, and GSM8K benchmarks.
111
 
112
  ### Reproduction
113
 
114
- The results can be reproduced using vLLM emulation docker: `rocmshared/pytorch:vllm-gfx950-mxfp4-mxfp6-v3`.
115
- The results of AIME24, MATH-500, and GPQA Diamond, were obtained using [vLLM](https://docs.vllm.ai/en/latest/) while GSM8K was obtained using [SGLang](https://docs.sglang.ai/). For AIME24, MATH-500, and GPQA Diamond, we took 10 rounds with different random seeds for reliable performance.
116
 
117
  ```
118
  MODEL_ARGS="model_name=amd/DeepSeek-R1-0528-MXFP4-ASQ,dtype=bfloat16,tensor_parallel_size=8,max_model_length=71536,max_num_batched_tokens=32768,gpu_memory_utilization=0.85,generation_parameters={max_new_tokens:65536,temperature:0.6,top_p:0.95,seed:$SEED}"
 
51
 
52
  ## Evaluation
53
 
54
+ The model was evaluated on AIME24, GPQA Diamond, MATH-500, and GSM8K benchmarks. The tasks of AIME24, GPQA Diamond, and MATH-500 were conducted using [lighteval](https://github.com/huggingface/lighteval/tree/v0.10.0) with 10 rounds for different generation seeds. GSM8K was conducted using [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness).
55
 
56
  ### Accuracy
57
 
 
111
 
112
  ### Reproduction
113
 
114
+ The results of AIME24, MATH-500, and GPQA Diamond, were obtained using [lighteval](https://github.com/huggingface/lighteval/tree/v0.10.0) and [vLLM](https://docs.vllm.ai/en/latest/), running with docker `rocm/vllm-private:pytorch-vllm-gfx950-mxfp4-mxfp6-v3`.
 
115
 
116
  ```
117
  MODEL_ARGS="model_name=amd/DeepSeek-R1-0528-MXFP4-ASQ,dtype=bfloat16,tensor_parallel_size=8,max_model_length=71536,max_num_batched_tokens=32768,gpu_memory_utilization=0.85,generation_parameters={max_new_tokens:65536,temperature:0.6,top_p:0.95,seed:$SEED}"