Update README.md

README.md

@@ -62,7 +62,20 @@ We have conducted a comprehensive evaluation of Ling-3.0-flash across multiple a
 >     - <font style="color:rgb(16, 18, 20);">Draco: Scored based on official rubrics per question, with the final score calculated as the average across all questions using </font>Claude Opus 4.6<font style="color:rgb(16, 18, 20);"> as the scoring model.</font>
 >     - BrowseComp (Single-Agent): Evaluated using a resume strategy for context management: once the context reaches a 64K-token threshold, the trajectory is summarized, the original history is discarded, and execution is resumed from the summary.
 >     - BrowseComp (Multi-Agent):<font style="color:rgb(16, 18, 20);"> Evaluated on English and ZH Revised datasets using an internal multi-agent search harness based on SearchSwarm/Tongyi DeepResearch, configured with </font>`temperature=0.85, top_p=0.95, max_tokens=8K`<font style="color:rgb(16, 18, 20);">, and main/sub-agent context windows of 128K and 64K, respectively.</font>
->
+
+
+## Quantized Models
+We evaluate the quantized models using several datasets. The FP8 quantized model is applied via the blockwise quantization, and INT4 and FP4 models are applied via groupwise quantization with routed experts weights.
+
+|dataset	| BF16	| **FP8**	| INT4	| FP4 |
+| -- | -- | -- | -- | -- |
+|GPQA-diamond	| 84.97	| 84.00	| 83.65	| 82.42 |
+|IFBench |	73.40	| 73.40	| 72.20	| 72.33 |
+|SciCode |	41.24	| 40.37	| 39.35	| 39.79 |
+|ArcPrize |	68.75	| 67.18	| 67.56	| 64.16 |
+
+
+
 ## Quickstart
 #### Install our SGLang
 ```bash
@@ -70,17 +83,17 @@ pip install uv
 uv venv ~/my_ling_env
 source ~/my_ling_env/bin/activate
 git clone -b ling_v3_support https://github.com/inclusionAI/sglang_ling_v3.git
-cd sglang
+cd sglang_ling_v3
 pip install --upgrade pip
 pip install -e "python"
 ```
 
 #### Run Inference
-Here is the example to run Ling-3.0-flash with 4 GPUs, where the master node IP is `${MASTER_IP}` and server port is `${PORT}`:
+Here is the example to run Ling-3.0-flash fp8 quantized model with 2 GPUs, where the master node IP is `${MASTER_IP}` and server port is `${PORT}`:
 
 **Server**
 
-As our model is trained with MTP, we recommend to turn on MTP (i.e., --speculative-algorithm NEXTN) in inference for lower latency.
+As our model is trained with MTP, we recommend to turn on MTP (i.e., --speculative-algorithm NEXTN) in inference for lower latency. Currently, the implementation only supports `tp<=2` for blockwise fp8 inference.
 
 ```bash
 export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
@@ -91,7 +104,7 @@ python -m sglang.launch_server \
     --nnodes 1 \
     --mem-fraction-static 0.8 \
     --max-running-requests 64 \
-    --tp-size 4 \
+    --tp-size 2 \
     --chunked-prefill-size 8192 \
     --allow-auto-output-truncate \
     --tool-call-parser ling3 \