Image Feature Extraction
Transformers
Safetensors
motif_vision
feature-extraction
motif
vision-transformer
self-supervised
video
custom_code
Instructions to use Motif-Technologies/Motif-Vision-Encoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Motif-Technologies/Motif-Vision-Encoder with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-feature-extraction", model="Motif-Technologies/Motif-Vision-Encoder", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Motif-Technologies/Motif-Vision-Encoder", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
docs(README): blank OpenCLIP zero-shot cell; VideoMAEv2 data as 1.35M
Browse filesOpenCLIP ImageNet -> β (drop zero-shot value and β footnote); VideoMAEv2 training data 0.0014B -> 1.35M; update caption + protocol note references.
README.md
CHANGED
|
@@ -96,7 +96,7 @@ Outputs (`BaseModelOutputWithPooling`): `last_hidden_state` `(B, 1+4+N, 4096)`,
|
|
| 96 |
|
| 97 |
Compared against the strongest publicly reported self-supervised / vision backbones. Higher is
|
| 98 |
better for every column **except KITTI depth** (lower is better). Best **comparable** value per
|
| 99 |
-
column in **bold**; the footnoted
|
| 100 |
excluded from the bold ranking.
|
| 101 |
|
| 102 |
DAVIS S/M/L follow the DINOv3 protocol (J&F-mean at video short side 420/480, 840/960,
|
|
@@ -109,11 +109,11 @@ single-resolution (S) figure is available.
|
|
| 109 |
| DINOv3 | 1.7B | 71.1 | 79.7 | 83.3 | 88.4 | **55.9** | 87.8 | **2.3** |
|
| 110 |
| PEcore | 5.4B | 48.2 | 53.1 | 49.8 | **89.3** | 38.9 | **87.9** | 4.1 |
|
| 111 |
| SigLIP2 | 10B | 56.1 | 62.3 | 62.9 | 89.1 | 45.4 | 86.9 | β |
|
| 112 |
-
| OpenCLIP | 2B | β | β | β |
|
| 113 |
| V-JEPA 2.1 | 0.022B | 69.0 | β | β | 85.5 | 47.9 | 87.7 | 3.x |
|
| 114 |
-
| VideoMAEv2 |
|
| 115 |
|
| 116 |
-
<sub>
|
| 117 |
|
| 118 |
- **DAVIS video segmentation (J&F)** β leads at every resolution: **74.0 (S) / 80.5 (M) /
|
| 119 |
83.5 (L)**, surpassing DINOv3 7B (71.1 / 79.7 / 83.3) across the board, reflecting the
|
|
@@ -133,7 +133,8 @@ single-resolution (S) figure is available.
|
|
| 133 |
Protocol: DINOv3-style linear/attentive probes for image tasks; V-JEPA 2-style protocol for
|
| 134 |
video. Baseline DAVIS / ADE20K / K400 / KITTI figures are taken from the DINOv3 technical
|
| 135 |
report's unified evaluation (Tab. 3, 5, 6) and ImageNet from Tab. 7; OpenCLIP and VideoMAEv2
|
| 136 |
-
are not in that report
|
|
|
|
| 137 |
|
| 138 |
## License
|
| 139 |
|
|
|
|
| 96 |
|
| 97 |
Compared against the strongest publicly reported self-supervised / vision backbones. Higher is
|
| 98 |
better for every column **except KITTI depth** (lower is better). Best **comparable** value per
|
| 99 |
+
column in **bold**; the footnoted figure (β‘) is measured under a different protocol and is
|
| 100 |
excluded from the bold ranking.
|
| 101 |
|
| 102 |
DAVIS S/M/L follow the DINOv3 protocol (J&F-mean at video short side 420/480, 840/960,
|
|
|
|
| 109 |
| DINOv3 | 1.7B | 71.1 | 79.7 | 83.3 | 88.4 | **55.9** | 87.8 | **2.3** |
|
| 110 |
| PEcore | 5.4B | 48.2 | 53.1 | 49.8 | **89.3** | 38.9 | **87.9** | 4.1 |
|
| 111 |
| SigLIP2 | 10B | 56.1 | 62.3 | 62.9 | 89.1 | 45.4 | 86.9 | β |
|
| 112 |
+
| OpenCLIP | 2B | β | β | β | β | β | β | β |
|
| 113 |
| V-JEPA 2.1 | 0.022B | 69.0 | β | β | 85.5 | 47.9 | 87.7 | 3.x |
|
| 114 |
+
| VideoMAEv2 | 1.35M | β | β | β | β | β | 90.0<sup>β‘</sup> | β |
|
| 115 |
|
| 116 |
+
<sub>β‘ VideoMAEv2 ViT-g: K400 top-1 after **full fine-tuning**, not the frozen attentive-probe protocol used for the other rows.</sub>
|
| 117 |
|
| 118 |
- **DAVIS video segmentation (J&F)** β leads at every resolution: **74.0 (S) / 80.5 (M) /
|
| 119 |
83.5 (L)**, surpassing DINOv3 7B (71.1 / 79.7 / 83.3) across the board, reflecting the
|
|
|
|
| 133 |
Protocol: DINOv3-style linear/attentive probes for image tasks; V-JEPA 2-style protocol for
|
| 134 |
video. Baseline DAVIS / ADE20K / K400 / KITTI figures are taken from the DINOv3 technical
|
| 135 |
report's unified evaluation (Tab. 3, 5, 6) and ImageNet from Tab. 7; OpenCLIP and VideoMAEv2
|
| 136 |
+
are not in that report; VideoMAEv2's K400 figure is cited from its own paper (see β‘), and
|
| 137 |
+
OpenCLIP's remaining benchmarks are not reported under a comparable protocol.
|
| 138 |
|
| 139 |
## License
|
| 140 |
|