taewhan commited on
Commit
b4d78cc
·
verified ·
1 Parent(s): 8bc3bc2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -11
README.md CHANGED
@@ -13,10 +13,10 @@ tags:
13
 
14
  # Motif Vision Encoder
15
 
16
- Motif Vision Encoder is a **unified image + video** self-supervised vision encoder on a ViT
17
- backbone. A single **3D-convolutional tokenizer** ingests both modalities — an image is a
18
- 1-frame clip (`T=1`), a video is `T>1` — so the same weights produce **dense patch-level
19
- features** and a **language-aligned global (CLS) representation**.
20
 
21
  Trained on **~1/3 the data of DINOv3** (0.5B vs 1.7B samples), it still reaches competitive
22
  performance across image and video benchmarks — and leads on DAVIS video tracking.
@@ -94,8 +94,8 @@ Outputs (`BaseModelOutputWithPooling`): `last_hidden_state` `(B, 1+4+N, 4096)`,
94
  ## Evaluation
95
 
96
  Compared against the strongest publicly reported self-supervised / vision backbones. Higher is
97
- better for every column **except KITTI depth** (lower is better). Best **comparable** value per
98
- column in **bold**, second best <u>underlined</u>; the footnoted figure (‡) is measured under a
99
  different protocol and is excluded from the ranking.
100
 
101
  DAVIS S/M/L follow the DINOv3 protocol (J&F-mean at video short side 420/480, 840/960,
@@ -114,17 +114,18 @@ single-resolution (S) figure is available.
114
 
115
  <sub>‡ VideoMAEv2 ViT-g: K400 top-1 after **full fine-tuning**, not the frozen attentive-probe protocol used for the other rows.</sub>
116
 
117
- Motif sets a new **state of the art on DAVIS video segmentation**, leading at every resolution
118
- (74.0 / 80.5 / 83.5 J&F), and stays **competitive across the remaining image and video
119
- benchmarks** — despite training on only **~0.5B samples**, roughly **1/3 of DINOv3's data**.
120
- KITTI depth is still being evaluated and will be added as the run completes.
121
-
122
  Protocol: DINOv3-style linear/attentive probes for image tasks; V-JEPA 2-style protocol for
123
  video. Baseline DAVIS / ADE20K / K400 / KITTI figures are taken from the DINOv3 technical
124
  report's unified evaluation (Tab. 3, 5, 6) and ImageNet from Tab. 7; OpenCLIP and VideoMAEv2
125
  are not in that report; VideoMAEv2's K400 figure is cited from its own paper (see ‡), and
126
  OpenCLIP's remaining benchmarks are not reported under a comparable protocol.
127
 
 
 
 
 
 
 
128
  ## License
129
 
130
  Released under the **MIT License** (see `LICENSE`). The model was trained on data governed by the
 
13
 
14
  # Motif Vision Encoder
15
 
16
+ Motif Vision Encoder is a unified image + video self-supervised vision encoder on a ViT
17
+ backbone. A single 3D-convolutional tokenizer ingests both modalities — an image is a
18
+ 1-frame clip (`T=1`), a video is `T>1` — so the same weights produce dense patch-level
19
+ features and a language-aligned global (CLS) representation.
20
 
21
  Trained on **~1/3 the data of DINOv3** (0.5B vs 1.7B samples), it still reaches competitive
22
  performance across image and video benchmarks — and leads on DAVIS video tracking.
 
94
  ## Evaluation
95
 
96
  Compared against the strongest publicly reported self-supervised / vision backbones. Higher is
97
+ better for every column except KITTI depth (lower is better). Best comparable value per
98
+ column in bold, second best <u>underlined</u>; the footnoted figure (‡) is measured under a
99
  different protocol and is excluded from the ranking.
100
 
101
  DAVIS S/M/L follow the DINOv3 protocol (J&F-mean at video short side 420/480, 840/960,
 
114
 
115
  <sub>‡ VideoMAEv2 ViT-g: K400 top-1 after **full fine-tuning**, not the frozen attentive-probe protocol used for the other rows.</sub>
116
 
 
 
 
 
 
117
  Protocol: DINOv3-style linear/attentive probes for image tasks; V-JEPA 2-style protocol for
118
  video. Baseline DAVIS / ADE20K / K400 / KITTI figures are taken from the DINOv3 technical
119
  report's unified evaluation (Tab. 3, 5, 6) and ImageNet from Tab. 7; OpenCLIP and VideoMAEv2
120
  are not in that report; VideoMAEv2's K400 figure is cited from its own paper (see ‡), and
121
  OpenCLIP's remaining benchmarks are not reported under a comparable protocol.
122
 
123
+ Motif sets a new state of the art on DAVIS video segmentation, leading at every resolution
124
+ (74.0 / 80.5 / 83.5 J&F), and stays competitive across the remaining image and video
125
+ benchmarks — despite training on only ~0.5B sample*, roughly 1/3 of DINOv3's data.
126
+ KITTI depth is still being evaluated and will be added as the run completes.
127
+
128
+
129
  ## License
130
 
131
  Released under the **MIT License** (see `LICENSE`). The model was trained on data governed by the