Update README.md
Browse files**lightvl Developed by Myself is a lightweight Vision-Language Model (VLM) quantization toolkit supporting FP8, INT8, FP8-Block. It integrates with vLLM for high-throughput inference and supports Qwen3-VL, Qwen3.5, InternVL-Chat, and Gemma-4 models.**
**fast quant your model step by step:**
1、 pip3 install lightvl
2、 lightvl YOUR_HF_MODEL_PATH
3、 the output quant model path is YOUR_HF_MODEL_PATH-fp8
README.md
CHANGED
|
@@ -5,7 +5,8 @@ license_link: https://ai.google.dev/gemma/docs/gemma_4_license
|
|
| 5 |
pipeline_tag: any-to-any
|
| 6 |
tags:
|
| 7 |
- fp8
|
| 8 |
-
-
|
|
|
|
| 9 |
---
|
| 10 |
|
| 11 |
## gemma-4-E4B-it-fp8（最新大模型推理加速技术见微信公众号: HyperAI）
|
|
@@ -21,6 +22,16 @@ tags:
|
|
| 21 |
**vllm serve Hyper-AI/gemma-4-E4B-it-fp8 --max-model-len 32768**
|
| 22 |
|
| 23 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
<div align="center">
|
| 26 |
<img src=https://ai.google.dev/gemma/images/gemma4_banner.png>
|
|
|
|
| 5 |
pipeline_tag: any-to-any
|
| 6 |
tags:
|
| 7 |
- fp8
|
| 8 |
+
- gemma-4
|
| 9 |
+
- lightvl
|
| 10 |
---
|
| 11 |
|
| 12 |
## gemma-4-E4B-it-fp8（最新大模型推理加速技术见微信公众号: HyperAI）
|
|
|
|
| 22 |
**vllm serve Hyper-AI/gemma-4-E4B-it-fp8 --max-model-len 32768**
|
| 23 |
|
| 24 |
|
| 25 |
+
**lightvl Developed by Myself is a lightweight Vision-Language Model (VLM) quantization toolkit supporting FP8, INT8, FP8-Block. It integrates with vLLM for high-throughput inference and supports Qwen3-VL, Qwen3.5, InternVL-Chat, and Gemma-4 models.**
|
| 26 |
+
|
| 27 |
+
**fast quant your model step by step:**
|
| 28 |
+
|
| 29 |
+
1、 pip3 install lightvl
|
| 30 |
+
|
| 31 |
+
2、 lightvl YOUR_HF_MODEL_PATH
|
| 32 |
+
|
| 33 |
+
3、 the output quant model path is YOUR_HF_MODEL_PATH-fp8
|
| 34 |
+
|
| 35 |
|
| 36 |
<div align="center">
|
| 37 |
<img src=https://ai.google.dev/gemma/images/gemma4_banner.png>
|