yiyexy commited on
Commit
fbb031a
·
verified ·
1 Parent(s): 21d0914

Model card: TOMATO results under the official setting

Browse files
Files changed (2) hide show
  1. README.md +2 -2
  2. README_CN.md +2 -2
README.md CHANGED
@@ -46,7 +46,7 @@ LLaVA-OneVision-2.5-4B-Instruct is a 4B vision-language model for single-image,
46
  <tr><td align="left">VideoMME (no subtitles)</td><td align="center"><b>71.9</b></td><td align="center">69.7</td><td align="center">71.7</td></tr>
47
  <tr><td align="left">VideoMME (subtitles)</td><td align="center">74.7</td><td align="center">75.2</td><td align="center"><b>75.9</b></td></tr>
48
  <tr><td align="left">MotionBench</td><td align="center"><b>64.2</b> <sup>TC128×313,600</sup></td><td align="center">62.4 <sup>128f×153,664</sup></td><td align="center">60.1 <sup>128f×153,664</sup></td></tr>
49
- <tr><td align="left">TOMATO *</td><td align="center"><b>38.6</b> <sup>TC128×313,600</sup></td><td align="center">33.6 <sup>model default</sup></td><td align="center">34.9 <sup>16f×153,664, fps=2</sup></td></tr>
50
  <tr><td align="left">MVBench</td><td align="center"><b>75.5</b> <sup>TC64×313,600</sup></td><td align="center">72.0 <sup>64f×153,664</sup></td><td align="center">64.0 <sup>64f×153,664</sup></td></tr>
51
  <tr><td align="left">NextQA MC test</td><td align="center"><b>86.4</b> <sup>TC64×313,600</sup></td><td align="center">86.1 <sup>64f×153,664</sup></td><td align="center">80.7 <sup>64f×153,664</sup></td></tr>
52
  <tr><td align="left">TempCompass MC</td><td align="center"><b>79.7</b> <sup>TC64×313,600</sup></td><td align="center">75.8 <sup>64f×153,664</sup></td><td align="center">73.4 <sup>64f×153,664</sup></td></tr>
@@ -88,7 +88,7 @@ LLaVA-OneVision-2.5-4B-Instruct is a 4B vision-language model for single-image,
88
  </tbody>
89
  </table>
90
 
91
- <sub>Compared with Qwen3.5-4B, thinking disabled. **Bold** is the best score in each row; `--` means not available. The superscript on each video score is the setting that produced it: frames × pixels per frame, or for codec, canvases (TC) × pixels per canvas. VideoMME is the mean over the short, medium and long splits. Each side reports its best score over the settings tried, from single runs. `*` On TOMATO the two sides use different frame settings. The image table shows selected benchmarks; full results, settings and protocol will be given in the technical report.</sub>
92
 
93
  ## Model Overview
94
 
 
46
  <tr><td align="left">VideoMME (no subtitles)</td><td align="center"><b>71.9</b></td><td align="center">69.7</td><td align="center">71.7</td></tr>
47
  <tr><td align="left">VideoMME (subtitles)</td><td align="center">74.7</td><td align="center">75.2</td><td align="center"><b>75.9</b></td></tr>
48
  <tr><td align="left">MotionBench</td><td align="center"><b>64.2</b> <sup>TC128×313,600</sup></td><td align="center">62.4 <sup>128f×153,664</sup></td><td align="center">60.1 <sup>128f×153,664</sup></td></tr>
49
+ <tr><td align="left">TOMATO</td><td align="center">38.6 <sup>TC128×313,600</sup></td><td align="center"><b>39.2</b> <sup>official</sup></td><td align="center">32.4 <sup>official</sup></td></tr>
50
  <tr><td align="left">MVBench</td><td align="center"><b>75.5</b> <sup>TC64×313,600</sup></td><td align="center">72.0 <sup>64f×153,664</sup></td><td align="center">64.0 <sup>64f×153,664</sup></td></tr>
51
  <tr><td align="left">NextQA MC test</td><td align="center"><b>86.4</b> <sup>TC64×313,600</sup></td><td align="center">86.1 <sup>64f×153,664</sup></td><td align="center">80.7 <sup>64f×153,664</sup></td></tr>
52
  <tr><td align="left">TempCompass MC</td><td align="center"><b>79.7</b> <sup>TC64×313,600</sup></td><td align="center">75.8 <sup>64f×153,664</sup></td><td align="center">73.4 <sup>64f×153,664</sup></td></tr>
 
88
  </tbody>
89
  </table>
90
 
91
+ <sub>Compared with Qwen3.5-4B, thinking disabled. **Bold** is the best score in each row; `--` means not available. The superscript on each video score is the setting that produced it: frames × pixels per frame, or for codec, canvases (TC) × pixels per canvas. VideoMME is the mean over the short, medium and long splits. Each side reports its best score over the settings tried, from single runs. The image table shows selected benchmarks; full results, settings and protocol will be given in the technical report.</sub>
92
 
93
  ## Model Overview
94
 
README_CN.md CHANGED
@@ -46,7 +46,7 @@ LLaVA-OneVision-2.5-4B-Instruct 是一个 4B 视觉语言模型,支持单图
46
  <tr><td align="left">VideoMME(无字幕)</td><td align="center"><b>71.9</b></td><td align="center">69.7</td><td align="center">71.7</td></tr>
47
  <tr><td align="left">VideoMME(有字幕)</td><td align="center">74.7</td><td align="center">75.2</td><td align="center"><b>75.9</b></td></tr>
48
  <tr><td align="left">MotionBench</td><td align="center"><b>64.2</b> <sup>TC128×313,600</sup></td><td align="center">62.4 <sup>128f×153,664</sup></td><td align="center">60.1 <sup>128f×153,664</sup></td></tr>
49
- <tr><td align="left">TOMATO *</td><td align="center"><b>38.6</b> <sup>TC128×313,600</sup></td><td align="center">33.6 <sup>model default</sup></td><td align="center">34.9 <sup>16f×153,664, fps=2</sup></td></tr>
50
  <tr><td align="left">MVBench</td><td align="center"><b>75.5</b> <sup>TC64×313,600</sup></td><td align="center">72.0 <sup>64f×153,664</sup></td><td align="center">64.0 <sup>64f×153,664</sup></td></tr>
51
  <tr><td align="left">NextQA MC test</td><td align="center"><b>86.4</b> <sup>TC64×313,600</sup></td><td align="center">86.1 <sup>64f×153,664</sup></td><td align="center">80.7 <sup>64f×153,664</sup></td></tr>
52
  <tr><td align="left">TempCompass MC</td><td align="center"><b>79.7</b> <sup>TC64×313,600</sup></td><td align="center">75.8 <sup>64f×153,664</sup></td><td align="center">73.4 <sup>64f×153,664</sup></td></tr>
@@ -88,7 +88,7 @@ LLaVA-OneVision-2.5-4B-Instruct 是一个 4B 视觉语言模型,支持单图
88
  </tbody>
89
  </table>
90
 
91
- <sub>对比对象为关闭 thinking 的 Qwen3.5-4B。**加粗**为该行最高分,`--` 表示暂无结果。视频分数的上标是取得该分数时的设置:帧数 × 每帧像素;codec 为画布数(TC)× 每画布像素。VideoMME 为 short、medium、long 三个子集的平均。两边各取已测设置中的最高分,均为单次运行。`*` TOMATO 两边帧数设置不同。图像表为代表性 bench 的节选,完整结果、设置与评测口径见后续技术报告。</sub>
92
 
93
  ## 模型概览
94
 
 
46
  <tr><td align="left">VideoMME(无字幕)</td><td align="center"><b>71.9</b></td><td align="center">69.7</td><td align="center">71.7</td></tr>
47
  <tr><td align="left">VideoMME(有字幕)</td><td align="center">74.7</td><td align="center">75.2</td><td align="center"><b>75.9</b></td></tr>
48
  <tr><td align="left">MotionBench</td><td align="center"><b>64.2</b> <sup>TC128×313,600</sup></td><td align="center">62.4 <sup>128f×153,664</sup></td><td align="center">60.1 <sup>128f×153,664</sup></td></tr>
49
+ <tr><td align="left">TOMATO</td><td align="center">38.6 <sup>TC128×313,600</sup></td><td align="center"><b>39.2</b> <sup>official</sup></td><td align="center">32.4 <sup>official</sup></td></tr>
50
  <tr><td align="left">MVBench</td><td align="center"><b>75.5</b> <sup>TC64×313,600</sup></td><td align="center">72.0 <sup>64f×153,664</sup></td><td align="center">64.0 <sup>64f×153,664</sup></td></tr>
51
  <tr><td align="left">NextQA MC test</td><td align="center"><b>86.4</b> <sup>TC64×313,600</sup></td><td align="center">86.1 <sup>64f×153,664</sup></td><td align="center">80.7 <sup>64f×153,664</sup></td></tr>
52
  <tr><td align="left">TempCompass MC</td><td align="center"><b>79.7</b> <sup>TC64×313,600</sup></td><td align="center">75.8 <sup>64f×153,664</sup></td><td align="center">73.4 <sup>64f×153,664</sup></td></tr>
 
88
  </tbody>
89
  </table>
90
 
91
+ <sub>对比对象为关闭 thinking 的 Qwen3.5-4B。**加粗**为该行最高分,`--` 表示暂无结果。视频分数的上标是取得该分数时的设置:帧数 × 每帧像素;codec 为画布数(TC)× 每画布像素。VideoMME 为 short、medium、long 三个子集的平均。两边各取已测设置中的最高分,均为单次运行。图像表为代表性 bench 的节选,完整结果、设置与评测口径见后续技术报告。</sub>
92
 
93
  ## 模型概览
94