Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -58,7 +58,6 @@ This model is designed as a high-performance offline engine for multimodal tasks
|
|
| 58 |
## 🏗 Model Architecture
|
| 59 |
|
| 60 |
**MOSS-VL-Instruct-0408** adopts a decoupled multimodal design, utilizing a cross-attention mechanism to bridge high-resolution visual encoding with advanced language reasoning.
|
| 61 |
-
|
| 62 |
<p align="center">
|
| 63 |
<img src="assets/structure.png" alt="MOSS-VL Architecture" width="90%"/>
|
| 64 |
<br>
|
|
@@ -69,7 +68,6 @@ This model is designed as a high-performance offline engine for multimodal tasks
|
|
| 69 |
## Temporal-Aware Prompting
|
| 70 |
|
| 71 |
At the model-family level, MOSS-VL uses timestamp-aware multimodal prompting for video understanding. This design gives sampled frames explicit temporal anchors, which helps the model reason about order, duration, and event localization more robustly.
|
| 72 |
-
|
| 73 |
<p align="center">
|
| 74 |
<img src="assets/timestamp_input.svg" alt="Timestamped Sequence Input Illustration" width="90%"/>
|
| 75 |
<br>
|
|
@@ -79,7 +77,6 @@ At the model-family level, MOSS-VL uses timestamp-aware multimodal prompting for
|
|
| 79 |
## Multimodal RoPE
|
| 80 |
|
| 81 |
MOSS-VL uses multimodal rotary position encoding to align text tokens and visual features in a shared spatial-temporal coordinate system. At a high level, this improves video-text grounding and helps preserve temporal structure during multimodal reasoning.
|
| 82 |
-
|
| 83 |
<p align="center">
|
| 84 |
<img src="assets/3d-rope.png" alt="MOSS-VL mRoPE Architecture Illustration" width="80%"/>
|
| 85 |
<br>
|
|
@@ -90,7 +87,6 @@ MOSS-VL uses multimodal rotary position encoding to align text tokens and visual
|
|
| 90 |
## 📊 Model Performance
|
| 91 |
|
| 92 |
We evaluate **MOSS-VL-Instruct-0408** across several key multimodal benchmarks, focusing on both video and image understanding.
|
| 93 |
-
|
| 94 |
<p align="center">
|
| 95 |
<img src="assets/MOSS-VL-Benchmark.png" alt="MOSS-VL Benchmark Results" width="100%"/>
|
| 96 |
<br>
|
|
|
|
| 58 |
## 🏗 Model Architecture
|
| 59 |
|
| 60 |
**MOSS-VL-Instruct-0408** adopts a decoupled multimodal design, utilizing a cross-attention mechanism to bridge high-resolution visual encoding with advanced language reasoning.
|
|
|
|
| 61 |
<p align="center">
|
| 62 |
<img src="assets/structure.png" alt="MOSS-VL Architecture" width="90%"/>
|
| 63 |
<br>
|
|
|
|
| 68 |
## Temporal-Aware Prompting
|
| 69 |
|
| 70 |
At the model-family level, MOSS-VL uses timestamp-aware multimodal prompting for video understanding. This design gives sampled frames explicit temporal anchors, which helps the model reason about order, duration, and event localization more robustly.
|
|
|
|
| 71 |
<p align="center">
|
| 72 |
<img src="assets/timestamp_input.svg" alt="Timestamped Sequence Input Illustration" width="90%"/>
|
| 73 |
<br>
|
|
|
|
| 77 |
## Multimodal RoPE
|
| 78 |
|
| 79 |
MOSS-VL uses multimodal rotary position encoding to align text tokens and visual features in a shared spatial-temporal coordinate system. At a high level, this improves video-text grounding and helps preserve temporal structure during multimodal reasoning.
|
|
|
|
| 80 |
<p align="center">
|
| 81 |
<img src="assets/3d-rope.png" alt="MOSS-VL mRoPE Architecture Illustration" width="80%"/>
|
| 82 |
<br>
|
|
|
|
| 87 |
## 📊 Model Performance
|
| 88 |
|
| 89 |
We evaluate **MOSS-VL-Instruct-0408** across several key multimodal benchmarks, focusing on both video and image understanding.
|
|
|
|
| 90 |
<p align="center">
|
| 91 |
<img src="assets/MOSS-VL-Benchmark.png" alt="MOSS-VL Benchmark Results" width="100%"/>
|
| 92 |
<br>
|