linzhao-amd commited on
Commit
518dcc8
·
verified ·
1 Parent(s): 9f4609e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -112,7 +112,7 @@ The model was evaluated on AIME24, GPQA Diamond, and MATH-500 benchmarks using t
112
 
113
  ### Reproduction
114
 
115
- The results were obtained using the following commands, with custom tasks and 10 rounds using different random seeds.
116
 
117
  ```
118
  MODEL_ARGS="model_name=amd/DeepSeek-R1-0528-MXFP4-ASQ,dtype=bfloat16,tensor_parallel_size=8,max_model_length=71536,max_num_batched_tokens=32768,gpu_memory_utilization=0.85,generation_parameters={max_new_tokens:65536,temperature:0.6,top_p:0.95,seed:$SEED}"
@@ -125,7 +125,7 @@ lighteval vllm $MODEL_ARGS "custom|aime24_single|0|0,custom|math_500_single|0|0,
125
  2>&1 | tee -a "$LOG"
126
  ```
127
 
128
- gsm8k results were obtained using lm-eval-harness, with 10 rounds using different random seeds.
129
 
130
  ```
131
  MODEL_ARGS="model=amd/DeepSeek-R1-0528-MXFP4-ASQ,base_url=http://localhost:8000/v1/completions,num_concurrent=999999,timeout=999999,tokenized_requests=False,max_length=38768,temperature=0.6,top_p=0.95,add_bos_token=True,seed=$SEED"
 
112
 
113
  ### Reproduction
114
 
115
+ The results of AIME24, MATH-500, and GPQA Diamond, were obtained using the following commands, with custom tasks and 10 rounds using different random seeds.
116
 
117
  ```
118
  MODEL_ARGS="model_name=amd/DeepSeek-R1-0528-MXFP4-ASQ,dtype=bfloat16,tensor_parallel_size=8,max_model_length=71536,max_num_batched_tokens=32768,gpu_memory_utilization=0.85,generation_parameters={max_new_tokens:65536,temperature:0.6,top_p:0.95,seed:$SEED}"
 
125
  2>&1 | tee -a "$LOG"
126
  ```
127
 
128
+ The result of gsm8k was obtained using lm-eval-harness.
129
 
130
  ```
131
  MODEL_ARGS="model=amd/DeepSeek-R1-0528-MXFP4-ASQ,base_url=http://localhost:8000/v1/completions,num_concurrent=999999,timeout=999999,tokenized_requests=False,max_length=38768,temperature=0.6,top_p=0.95,add_bos_token=True,seed=$SEED"