Update README_convert_gguf.md

README_convert_gguf.md

@@ -25,21 +25,10 @@ Dedicated to building a more intuitive, comprehensive, and efficient LLMs compre
 ## 📣Latest News
 - [26/02/09] We have released HY-1.8B-2Bit, 2bit on-device large language model.
 - [26/01/13] We have released v0.3. We support the training and deployment of Eagle3 for all-scale LLMs/VLMs/Audio models, as detailed in the [guidance documentation](https://angelslim.readthedocs.io/zh-cn/latest/features/speculative_decoding/eagle/index.html). And We released **Sherry**, the hardware-efficient 1.25 bit quantization algorithm [Paper Comming soon] | [[Code]](https://github.com/Tencent/AngelSlim/tree/sherry/Sherry)🔥🔥🔥
-- [25/11/05] We have released v0.2. Quantization support for new models, such as `GLM-4.6`, `Qwen3-VL` and `Qwen3-Omni`, open-sources the Eagle3 speculative decoding training framework, and updates the Diffusion model quantization tools.
-- [25/09/30] We have released **SpecExit**, the reasoning early-exit algorithm: [[Paper]](http://arxiv.org/abs/2509.24248) | [[Docs]](https://angelslim.readthedocs.io/zh-cn/latest/features/speculative_decoding/spec_exit.html) | [[vLLM Code]](https://github.com/vllm-project/vllm/pull/27192)
-- [25/09/26] We have released **TEQUILA**, the ternary quantization algorithm [[Paper]](https://arxiv.org/abs/2509.23809) | [[Code]](https://github.com/Tencent/AngelSlim/tree/tequila/TernaryQuant)
-- [25/09/24] We now support the PTQ quantization of NVFP4 for the Qwen3 series models. We also opensource [Qwen3-32B-NVFP4](https://huggingface.co/AngelSlim/Qwen3-32B_nvfp4) and [Qwen3-235B-A22B-NVFP4](https://huggingface.co/AngelSlim/Qwen3-235B-A22B_nvfp4) weights.
 
-<details>
-<summary>Previous News</summary>
+For more detailed information, please refer to[[AngelSlim]](https://github.com/Tencent/AngelSlim)
 
-- [25/09/01] We now support ​FP8 quantization​ of the [Hunyuan-MT-7B](https://huggingface.co/tencent/Hunyuan-MT-7B-fp8) translation model. And enabled ​Torch inference and Benchmark evaluation​ for Eagle3. And implemented support for ​quantization and Cache​ for [FLUX](https://github.com/Tencent/AngelSlim/tree/main/configs/flux). And support ​quantization​ for the [Seed-OSS](https://github.com/Tencent/AngelSlim/tree/main/configs/seed_oss).
-- [25/08/06] We now support quantization for `Hunyuan 0.5B/1.8B/4B/7B` and multimodal model `Qwen2.5VL 3B/7B/32B/72B`, including `FP8/INT4` algorithms, and quantization for `DeepSeek-R1/V3` and `Kimi-K2`, including `FP8-Static` and `W4A8-FP8` algorithms. We also opensource `Hunyuan 1.8B/4B/7B` series Eagle3 model weight.
-- [25/07/04] We now support quantization for `Hunyuan/Qwen2.5/Qwen3/DeepSeek-R1-Distill-Qwen` and other models, including `INT8/FP8/INT4` algorithms. We also opensource `Qwen3` series Eagle3 model weight.
-
-</details>
-
-## 🌟Convert hf to gguf-q2_0
+## 🌟Convert hf to gguf-fp16
 
 **Step1**: Clone llama.cpp
 
@@ -54,23 +43,70 @@ cd llama.cpp
 python convert_hf_to_gguf.py ../qdq_model_path/ --outfile ./model-fp16.gguf --outtype f16
 ```
 
-**Step3**: Build llama.cpp with `0001-Add-support-for-int2-per-channel-quantization.patch`
+## 💻Deployment 
+This setup ONLY works on SME2-capable devices (for example, Apple M4, vivo x300 and Arm CPUs with SME2 support). Neon kernel will follow up.
+
+### Running Hunyuan model on MacBook M4
+
+ Clone llama.cpp
 
 ```bash
 git clone https://github.com/ggml-org/llama.cpp.git
+```
+
+Enter the llama.cpp folder
+
+```bash
 cd llama.cpp
-git checkout 960e5e3b46f211b3c4446ce8380e88404274dbce
-git apply 0001-Add-support-for-int2-per-channel-quantization.patch
+```
+
+Fetch and check out the PR branch
+```bash
+git fetch origin pull/19357/head:pr-19357-sme2-int2
+git checkout pr-19357-sme2-int2
+```
+
+Build llama.cpp with KleidiAI enabled
+
+```bash
 mkdir build && cd build
+
 cmake -DGGML_CPU_KLEIDIAI=ON -DGGML_METAL=OFF -DGGML_BLAS=OFF ..
+
 make -j8
 ```
 
-**Step4**: Quantize the Hunyuan fp16 model to int2 per-channel (q2_0c)
+Quantize the Hunyuan fp16 model to int2 per-channel (q2_0c)
+
 ```bash
 ./bin/llama-quantize hunyuan-fp16-qdq.gguf hunyuan-q2_0.gguf q2_0c
 ```
 
+#### Run the CLI llama.cpp example
+
+
+```bash
+export GGML_KLEIDIAI_SME=1
+
+# thinking 
+./bin/llama-cli -m hunyuan-q2_0.gguf -p "写一副春联" -t 1 --seed 4568 -n 32
+# no thinking 
+./bin/llama-cli -m hunyuan-q2_0.gguf -p "/no_think写一副春联" -t 1 --seed 4568 -n 32
+```
+
+
+
+#### Run the llama.cpp benchmark
+
+The general command is:
+
+```bash
+./bin/llama-bench -m hunyuan-q2_0.gguf -p <prompt-length> -t <number-of-threads> -n <gen-length>
+```
+
+![image/jpeg](m4_performance.png)
+
+
 ## 📝 License
 
 The code for this project is open-sourced under the [License for AngelSlim](LICENSE).