Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -60,7 +60,6 @@ This model is designed as a high-performance offline engine for multimodal tasks
|
|
| 60 |
**MOSS-VL-Instruct-0408** adopts a decoupled multimodal design, utilizing a cross-attention mechanism to bridge high-resolution visual encoding with advanced language reasoning.
|
| 61 |
<p align="center">
|
| 62 |
<img src="assets/structure.png" alt="MOSS-VL Architecture" width="90%"/>
|
| 63 |
-
<br>
|
| 64 |
<em>Figure 1: MOSS-VL Core Architecture.</em>
|
| 65 |
</p>
|
| 66 |
|
|
@@ -70,7 +69,6 @@ This model is designed as a high-performance offline engine for multimodal tasks
|
|
| 70 |
At the model-family level, MOSS-VL uses timestamp-aware multimodal prompting for video understanding. This design gives sampled frames explicit temporal anchors, which helps the model reason about order, duration, and event localization more robustly.
|
| 71 |
<p align="center">
|
| 72 |
<img src="assets/timestamp_input.svg" alt="Timestamped Sequence Input Illustration" width="90%"/>
|
| 73 |
-
<br>
|
| 74 |
<em>Figure 2: Illustration of the timestamped sequence input pipeline.</em>
|
| 75 |
</p>
|
| 76 |
|
|
@@ -79,7 +77,6 @@ At the model-family level, MOSS-VL uses timestamp-aware multimodal prompting for
|
|
| 79 |
MOSS-VL uses multimodal rotary position encoding to align text tokens and visual features in a shared spatial-temporal coordinate system. At a high level, this improves video-text grounding and helps preserve temporal structure during multimodal reasoning.
|
| 80 |
<p align="center">
|
| 81 |
<img src="assets/3d-rope.png" alt="MOSS-VL mRoPE Architecture Illustration" width="80%"/>
|
| 82 |
-
<br>
|
| 83 |
<em>Figure 3: 3D-RoPE spatial-temporal alignment.</em>
|
| 84 |
</p>
|
| 85 |
|
|
@@ -89,7 +86,6 @@ MOSS-VL uses multimodal rotary position encoding to align text tokens and visual
|
|
| 89 |
We evaluate **MOSS-VL-Instruct-0408** across several key multimodal benchmarks, focusing on both video and image understanding.
|
| 90 |
<p align="center">
|
| 91 |
<img src="assets/MOSS-VL-Benchmark.png" alt="MOSS-VL Benchmark Results" width="100%"/>
|
| 92 |
-
<br>
|
| 93 |
<em>Figure 4: Performance comparison on mainstream multimodal benchmarks.</em>
|
| 94 |
</p>
|
| 95 |
|
|
|
|
| 60 |
**MOSS-VL-Instruct-0408** adopts a decoupled multimodal design, utilizing a cross-attention mechanism to bridge high-resolution visual encoding with advanced language reasoning.
|
| 61 |
<p align="center">
|
| 62 |
<img src="assets/structure.png" alt="MOSS-VL Architecture" width="90%"/>
|
|
|
|
| 63 |
<em>Figure 1: MOSS-VL Core Architecture.</em>
|
| 64 |
</p>
|
| 65 |
|
|
|
|
| 69 |
At the model-family level, MOSS-VL uses timestamp-aware multimodal prompting for video understanding. This design gives sampled frames explicit temporal anchors, which helps the model reason about order, duration, and event localization more robustly.
|
| 70 |
<p align="center">
|
| 71 |
<img src="assets/timestamp_input.svg" alt="Timestamped Sequence Input Illustration" width="90%"/>
|
|
|
|
| 72 |
<em>Figure 2: Illustration of the timestamped sequence input pipeline.</em>
|
| 73 |
</p>
|
| 74 |
|
|
|
|
| 77 |
MOSS-VL uses multimodal rotary position encoding to align text tokens and visual features in a shared spatial-temporal coordinate system. At a high level, this improves video-text grounding and helps preserve temporal structure during multimodal reasoning.
|
| 78 |
<p align="center">
|
| 79 |
<img src="assets/3d-rope.png" alt="MOSS-VL mRoPE Architecture Illustration" width="80%"/>
|
|
|
|
| 80 |
<em>Figure 3: 3D-RoPE spatial-temporal alignment.</em>
|
| 81 |
</p>
|
| 82 |
|
|
|
|
| 86 |
We evaluate **MOSS-VL-Instruct-0408** across several key multimodal benchmarks, focusing on both video and image understanding.
|
| 87 |
<p align="center">
|
| 88 |
<img src="assets/MOSS-VL-Benchmark.png" alt="MOSS-VL Benchmark Results" width="100%"/>
|
|
|
|
| 89 |
<em>Figure 4: Performance comparison on mainstream multimodal benchmarks.</em>
|
| 90 |
</p>
|
| 91 |
|