Update README.md

README.md

@@ -50,5 +50,71 @@ python3 quantize_quark.py --model_dir $MODEL_DIR \
 
 This model can be deployed efficiently using the [SGLang](https://docs.sglang.ai/) backend.
 
+## Evaluation
+
+The model was evaluated on AIME24, GPQA Diamond, and MATH-500 benchmarks using the [lighteval](https://github.com/huggingface/lighteval/tree/v0.10.0) framework. Each benchmark was run 10 times with different random seeds for reliable performance estimation.
+
+### Accuracy
+
+<table>
+  <tr>
+   <td><strong>Benchmark</strong>
+   </td>
+   <td><strong>DeepSeek-R1-0528 </strong>
+   </td>
+   <td><strong>DeepSeek-R1-0528-MXFP4(this model)</strong>
+   </td>
+   <td><strong>Recovery</strong>
+   </td>
+  </tr>
+  <tr>
+   <td>AIME24 pass@1
+   </td>
+   <td>88.00
+   </td>
+   <td>87.67
+   </td>
+   <td>99.62%
+   </td>
+  </tr>
+  <tr>
+   <td>GPQA Diamond pass@1
+   </td>
+   <td>79.90
+   </td>
+   <td>79.65
+   </td>
+   <td>99.69%
+   </td>
+  </tr>
+  <tr>
+   <td>MATH-500 pass@1
+   </td>
+   <td>97.06
+   </td>
+   <td>96.90
+   </td>
+   <td>99.84%
+   </td>
+  </tr>
+</table>
+
+
+### Reproduction
+
+The results were obtained using the following commands, with custom tasks and 10 rounds using different random seeds.
+
+```
+MODEL_ARGS="model_name=amd/DeepSeek-R1-0528-MXFP4-ASQ,dtype=bfloat16,tensor_parallel_size=8,max_model_length=71536,max_num_batched_tokens=32768,gpu_memory_utilization=0.85,generation_parameters={max_new_tokens:65536,temperature:0.6,top_p:0.95,seed:$SEED}"
+OUTPUT_DIR="results/DeepSeek-R1-0528-MXFP4-ASQ-Seed"
+LOG="logs/deepseek_0528_maxfp4.log"
+
+lighteval vllm $MODEL_ARGS "custom|aime24_single|0|0,custom|math_500_single|0|0,custom|gpqa:diamond_single|0|0" \
+    --use-chat-template \
+    --output-dir "results/DeepSeek-R1-0528-MXFP4-ASQ-Seed/seed_$SEED" \
+    2>&1 | tee -a "$LOG"
+```
+
+
 # License
 Modifications Copyright(c) 2025 Advanced Micro Devices, Inc. All rights reserved.