Update README.md

README.md

@@ -35,12 +35,40 @@ extra_gated_description: >-
 pipeline_tag: image-text-to-text
 ---
 
-Checkpoint of Mistral-Small-3.1-24B-Instruct-2503 with FP8 per-tensor quantization in the Mistral-format. Please run with vLLM like so:
+Checkpoint of Mistral-Small-3.1-24B-Instruct-2503 with FP8 per-tensor quantization in the Mistral-format. 
+
+Please run with vLLM like so:
 ```
 vllm serve nm-testing/Mistral-Small-3.1-24B-Instruct-2503-FP8 --tokenizer_mode mistral --config_format mistral --load_format mistral --tool-call-parser mistral --enable-auto-tool-choice --limit_mm_per_prompt 'image=10'
 ```
 
-# Model Card for Mistral-Small-3.1-24B-Instruct-2503
+Evaluations against the unquantized baseline on ChartQA:
+```
+vllm serve mistralai/Mistral-Small-3.1-24B-Instruct-2503 --tokenizer_mode mistral --config_format mistral --load_format mistral
+python -m eval.run eval_vllm --model_name mistralai/Mistral-Small-3.1-24B-Instruct-2503 --url http://0.0.0.0:8000 --output_dir output/ --eval_name "chartqa"
+Querying model: 100%|██████████████████████████████████████████████████████████████████████| 2500/2500 [07:37<00:00,  5.47it/s]
+================================================================================
+Metrics:
+{
+    "explicit_prompt_relaxed_correctness": 0.8604,
+    "anywhere_in_answer_relaxed_correctness": 0.8604
+}
+================================================================================
+
+vllm serve nm-testing/Mistral-Small-3.1-24B-Instruct-2503-FP8 --tokenizer_mode mistral --config_format mistral --load_format mistral
+python -m eval.run eval_vllm --model_name nm-testing/Mistral-Small-3.1-24B-Instruct-2503-FP8 --url http://0.0.0.0:8000 --output_dir output/ --eval_name "chartqa"
+Querying model: 100%|██████████████████████████████████████████████████████████████████████| 2500/2500 [06:37<00:00,  6.28it/s]
+================================================================================
+Metrics:
+{
+    "explicit_prompt_relaxed_correctness": 0.8596,
+    "anywhere_in_answer_relaxed_correctness": 0.86
+}
+================================================================================
+```
+
+
+# Original Model Card for Mistral-Small-3.1-24B-Instruct-2503
 
 Building upon Mistral Small 3 (2501), Mistral Small 3.1 (2503) **adds state-of-the-art vision understanding** and enhances **long context capabilities up to 128k tokens** without compromising text performance. 
 With 24 billion parameters, this model achieves top-tier capabilities in both text and vision tasks.