Video-Text-to-Text
Transformers
Safetensors
English
Chinese
moss_vl
feature-extraction
MOSS-VL
realtime
streaming
video-understanding
FP8
compressed-tensors
HQQ
quantized
custom_code
Instructions to use OpenMOSS-Team/MOSS-VL-Realtime-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenMOSS-Team/MOSS-VL-Realtime-FP8 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("OpenMOSS-Team/MOSS-VL-Realtime-FP8", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add files using upload-large-folder tool
Browse files
.gitattributes
CHANGED
|
@@ -39,3 +39,5 @@ assets/architecture.png filter=lfs diff=lfs merge=lfs -text
|
|
| 39 |
assets/logo.png filter=lfs diff=lfs merge=lfs -text
|
| 40 |
assets/mossvl_quantization_benchmark_comparison_zh_4k.png filter=lfs diff=lfs merge=lfs -text
|
| 41 |
assets/mossvl_quantization_benchmark_comparison_en_4k.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 39 |
assets/logo.png filter=lfs diff=lfs merge=lfs -text
|
| 40 |
assets/mossvl_quantization_benchmark_comparison_zh_4k.png filter=lfs diff=lfs merge=lfs -text
|
| 41 |
assets/mossvl_quantization_benchmark_comparison_en_4k.png filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
assets/mossvl_quantization_benchmark_comparison_final_v3_en_4k.png filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
assets/mossvl_quantization_benchmark_comparison_final_v3_zh_4k.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -54,14 +54,15 @@ inference. This checkpoint is not an SGLang release.
|
|
| 54 |
directly and do not pass a second quantization configuration or replace its
|
| 55 |
generation config with the BF16 source file.
|
| 56 |
|
| 57 |
-
##
|
| 58 |
|
| 59 |
-
The
|
| 60 |
-
|
| 61 |
-
|
|
|
|
| 62 |
|
| 63 |
<p align="center">
|
| 64 |
-
<img src="assets/
|
| 65 |
</p>
|
| 66 |
|
| 67 |
## Hardware requirements
|
|
|
|
| 54 |
directly and do not pass a second quantization configuration or replace its
|
| 55 |
generation config with the BF16 source file.
|
| 56 |
|
| 57 |
+
## Quantization benchmark
|
| 58 |
|
| 59 |
+
The final evaluation compares the original BF16 model with all four release
|
| 60 |
+
profiles on their corresponding benchmark suites. For this streaming FP8
|
| 61 |
+
checkpoint, the scores are 70.66 on OVOBench Avg, 62.93 on StreamingBench Avg,
|
| 62 |
+
and 65.50 on OmniMMI PA, compared with 70.86, 62.42, and 66.00 for BF16.
|
| 63 |
|
| 64 |
<p align="center">
|
| 65 |
+
<img src="assets/mossvl_quantization_benchmark_comparison_final_v3_en_4k.png" alt="MOSS-VL quantization benchmark comparison" width="100%"/>
|
| 66 |
</p>
|
| 67 |
|
| 68 |
## Hardware requirements
|
README_zh.md
CHANGED
|
@@ -52,13 +52,14 @@ tags:
|
|
| 52 |
`generation_config.json` 会自动启用 HQQ KV8。请直接加载 checkpoint,
|
| 53 |
不要再次传入量化配置,也不要用 BF16 原模型的 generation config 覆盖它。
|
| 54 |
|
| 55 |
-
##
|
| 56 |
|
| 57 |
-
|
| 58 |
-
|
|
|
|
| 59 |
|
| 60 |
<p align="center">
|
| 61 |
-
<img src="assets/
|
| 62 |
</p>
|
| 63 |
|
| 64 |
## 硬件要求
|
|
|
|
| 52 |
`generation_config.json` 会自动启用 HQQ KV8。请直接加载 checkpoint,
|
| 53 |
不要再次传入量化配置,也不要用 BF16 原模型的 generation config 覆盖它。
|
| 54 |
|
| 55 |
+
## 量化 Benchmark
|
| 56 |
|
| 57 |
+
最终测评在各自对应的 benchmark 上对比原始 BF16 模型与四个量化发布配置。
|
| 58 |
+
该流式 FP8 checkpoint 的 OVOBench Avg、StreamingBench Avg 和 OmniMMI PA
|
| 59 |
+
分别为 70.66、62.93 和 65.50;对应 BF16 分数为 70.86、62.42 和 66.00。
|
| 60 |
|
| 61 |
<p align="center">
|
| 62 |
+
<img src="assets/mossvl_quantization_benchmark_comparison_final_v3_zh_4k.png" alt="MOSS-VL 量化配置 benchmark 对比" width="100%"/>
|
| 63 |
</p>
|
| 64 |
|
| 65 |
## 硬件要求
|
assets/mossvl_quantization_benchmark_comparison_final_v3_en_4k.png
ADDED
|
Git LFS Details
|
assets/mossvl_quantization_benchmark_comparison_final_v3_zh_4k.png
ADDED
|
Git LFS Details
|