Add model card

README.md

@@ -2,10 +2,116 @@
 license: other
 license_name: modified-mit
 license_link: LICENSE
+base_model:
+- moonshotai/Kimi-K2.5
 ---
 
-**Disclaimer**
+# Model Overview
 
-This model is provided for research and evaluation purposes only.
-Quantization may introduce accuracy or behavioral differences compared to the original model.
-Users are responsible for validating the model in their own environments and complying with the original model license.
+- **Model Architecture:** Kimi-K2.5
+  - **Input:** Text, Image, Video
+  - **Output:** Text
+- **Supported Hardware Microarchitecture:** AMD MI300/MI350/MI355 (emulation)
+- **ROCm:** 7.2.2
+- **Operating System(s):** Linux
+- **Inference Engine:** [vLLM](https://docs.vllm.ai/en/latest/)
+- **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (V0.12)
+  - **Quantized layers:** `experts` and `shared_experts`
+  - **Weight quantization:** NVFP4, Static 
+  - **Activation quantization:** NVFP4, Dynamic 
+- **Calibration Dataset:** [Pile](https://huggingface.co/datasets/mit-han-lab/pile-val-backup)
+
+This model was built with Kimi-K2.5 model by applying [AMD-Quark](https://quark.docs.amd.com/latest/index.html) for NVFP4 quantization.
+
+# Model Quantization
+
+The model was quantized from [moonshotai/Kimi-K2.5](https://huggingface.co/moonshotai/Kimi-K2.5) using [AMD-Quark](https://quark.docs.amd.com/latest/index.html). The weights and activations are quantized to NVFP4.
+
+**Quantization scripts:**
+```
+cd Quark/examples/torch/language_modeling/llm_ptq/
+MODEL_DIR=moonshotai/Kimi-K2.5
+export output_dir=amd/Kimi-K2.5-NVFP4
+exclude_layers="*self_attn* *mlp.gate *lm_head *mm_projector* *vision_tower*"
+
+python3 quantize_quark.py --model_dir $MODEL_DIR \
+                          --quant_scheme nvfp4 \
+                          --num_calib_data 128 \
+                          --exclude_layers $exclude_layers \
+                          --model_export hf_format \
+                          --output_dir $output_dir \
+                          --trust_remote_code \
+                          --multi_gpu balanced 
+```
+
+# Deployment
+### Use with vLLM
+
+This model can be deployed efficiently using the [vLLM](https://docs.vllm.ai/en/latest/) backend.
+
+## Evaluation
+The model was evaluated on GSM8K benchmarks. 
+
+### Accuracy
+
+<table>
+  <tr>
+   <td><strong>Benchmark</strong>
+   </td>
+   <td><strong>Kimi-K2.5 </strong>
+   </td>
+   <td><strong>Kimi-K2.5-NVFP4(this model) </strong>
+   </td>
+   <td><strong>Recovery</strong>
+   </td>
+  </tr>
+  <tr>
+   <td>GSM8K (flexible-extract)
+   </td>
+   <td>93.56
+   </td>
+   <td>92.87
+   </td>
+   <td>99.26%
+   </td>
+  </tr>
+
+  </tr>
+</table>
+
+### Reproduction
+
+The GSM8K result was obtained using the `lm-evaluation-harness` framework, based on the Docker image `rocm/vllm-dev:nightly_main_20260603`.
+
+Install the lm-eval `(Version: 0.4.12)` in container first.
+```
+pip install lm-eval
+pip install lm-eval[api]
+```
+
+#### Launching server
+```
+export VLLM_ROCM_USE_AITER=1
+vllm serve amd/Kimi-K2.5-NVFP4 -tp 8 \
+  --mm-encoder-tp-mode data \
+  --tool-call-parser kimi_k2 \
+  --reasoning-parser kimi_k2 \
+  --enforce-eager \
+  --trust-remote-code
+```
+
+#### Evaluating model in a new terminal
+```
+lm_eval \
+  --model local-completions \
+  --model_args "model=amd/Kimi-K2.5-NVFP4,kv_cache_dtype=fp8,base_url=http://0.0.0.0:8000/v1/completions,tokenized_requests=False,tokenizer_backend=None,num_concurrent=32" \
+  --tasks gsm8k \
+  --num_fewshot 5 \
+  --batch_size 1 
+```
+```
+
+
+# License
+Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
+ 

tokenizer_config.json

@@ -199,19 +199,18 @@
     "<|media_end|>",
     "<|media_pad|>"
   ],
-  "auto_map": {
-    "AutoTokenizer": [
-      "tokenization_kimi.TikTokenTokenizer",
-      null
-    ]
-  },
   "bos_token": "[BOS]",
   "clean_up_tokenization_spaces": false,
   "eos_token": "[EOS]",
   "extra_special_tokens": {},
   "model_max_length": 1000000000000000019884624838656,
-  "pad_token": "[EOS]",
-  "padding_side": "left",
+  "pad_token": "[PAD]",
   "tokenizer_class": "TikTokenTokenizer",
-  "unk_token": "[UNK]"
-}
+  "unk_token": "[UNK]",
+  "auto_map": {
+    "AutoTokenizer": [
+      "tokenization_kimi.TikTokenTokenizer",
+      null
+    ]
+  }
+}