linzhao-amd commited on
Commit
07dfa3d
·
verified ·
1 Parent(s): 8726bd7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -1
README.md CHANGED
@@ -100,13 +100,18 @@ The model was evaluated on AIME24, GPQA Diamond, and MATH-500 benchmarks using t
100
 
101
  ### Reproduction
102
 
103
- The results of AIME24, MATH-500, and GPQA Diamond, were obtained using [lighteval](https://github.com/huggingface/lighteval/tree/v0.10.0) and [vLLM](https://docs.vllm.ai/en/latest/), running with docker `rocm/vllm-private:pytorch-vllm-gfx950-mxfp4-mxfp6-v3`.
104
 
105
  ```
 
 
 
 
106
  MODEL_ARGS="model_name=amd/DeepSeek-R1-0528-MXFP4-Preview,dtype=bfloat16,tensor_parallel_size=8,max_model_length=71536,max_num_batched_tokens=32768,gpu_memory_utilization=0.85,generation_parameters={max_new_tokens:65536,temperature:0.6,top_p:0.95,seed:$SEED}"
107
  OUTPUT_DIR="results/DeepSeek-R1-0528-MXFP4-Preview-Seed"
108
  LOG="logs/deepseek_0528_maxfp4.log"
109
 
 
110
  for i in $(seq 1 10); do
111
  # seed in [0, 2**30 - 1]
112
  SEED=$(shuf -i 0-1073741823 -n 1)
 
100
 
101
  ### Reproduction
102
 
103
+ The results of AIME24, MATH-500, and GPQA Diamond, were obtained using forked [lighteval](https://github.com/zhaolin-amd/lighteval/tree/v0.10-release-custom) and vLLM docker `rocm/vllm-private:pytorch-vllm-gfx950-mxfp4-mxfp6-v3`.
104
 
105
  ```
106
+ # Set docker env
107
+ export VLLM_QUARK_F4F6_OFFLINE_DEQUANT_TMPENVVAR=1
108
+
109
+ # Set model args
110
  MODEL_ARGS="model_name=amd/DeepSeek-R1-0528-MXFP4-Preview,dtype=bfloat16,tensor_parallel_size=8,max_model_length=71536,max_num_batched_tokens=32768,gpu_memory_utilization=0.85,generation_parameters={max_new_tokens:65536,temperature:0.6,top_p:0.95,seed:$SEED}"
111
  OUTPUT_DIR="results/DeepSeek-R1-0528-MXFP4-Preview-Seed"
112
  LOG="logs/deepseek_0528_maxfp4.log"
113
 
114
+ # Evaluating 10 rounds
115
  for i in $(seq 1 10); do
116
  # seed in [0, 2**30 - 1]
117
  SEED=$(shuf -i 0-1073741823 -n 1)