Image Feature Extraction
Transformers
Safetensors
motif_vision
feature-extraction
motif
vision-transformer
self-supervised
video
custom_code
Instructions to use Motif-Technologies/Motif-Vision-Encoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Motif-Technologies/Motif-Vision-Encoder with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-feature-extraction", model="Motif-Technologies/Motif-Vision-Encoder", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Motif-Technologies/Motif-Vision-Encoder", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
docs(README): drop VideoMAEv2 row; update repo id after rename to Motif-Vision-Encoder
Browse filesRemove VideoMAEv2 row and its β‘ footnote/protocol references; update code repo id to Motif-Technologies/Motif-Vision-Encoder (repo renamed, -7B dropped). ViT-7B architecture spec kept.
README.md
CHANGED
|
@@ -46,7 +46,7 @@ from transformers.image_utils import load_image
|
|
| 46 |
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
|
| 47 |
image = load_image(url)
|
| 48 |
|
| 49 |
-
repo = "Motif-Technologies/Motif-Vision-Encoder
|
| 50 |
processor = AutoImageProcessor.from_pretrained(repo)
|
| 51 |
model = AutoModel.from_pretrained(repo, trust_remote_code=True, dtype=torch.bfloat16).to("cuda").eval()
|
| 52 |
|
|
@@ -94,8 +94,7 @@ Outputs (`BaseModelOutputWithPooling`): `last_hidden_state` `(B, 1+4+N, 4096)`,
|
|
| 94 |
## Evaluation
|
| 95 |
|
| 96 |
Compared against the strongest publicly reported self-supervised / vision backbones. Higher is
|
| 97 |
-
better for every column. Best comparable value per column in bold, second best <u>underlined</u>
|
| 98 |
-
the footnoted figure (β‘) is measured under a different protocol and is excluded from the ranking.
|
| 99 |
|
| 100 |
DAVIS S/M/L follow the DINOv3 protocol (J&F-mean at video short side 420/480, 840/960,
|
| 101 |
1260/1440 px). V-JEPA 2.1 is not part of the DINOv3 Table 5 tracking benchmark, so only its
|
|
@@ -110,15 +109,11 @@ single-resolution (S) figure is available.
|
|
| 110 |
| SigLIP2 | 10B | 56.1 | 62.3 | 62.9 | <u>89.1</u> | 45.4 | 86.9 |
|
| 111 |
| OpenCLIP | 2B | β | β | β | β | β | β |
|
| 112 |
| V-JEPA 2.1 | 0.022B | 69.0 | β | β | 85.5 | 47.9 | 87.7 |
|
| 113 |
-
| VideoMAEv2 | 0.0014B | β | β | β | β | β | 90.0<sup>β‘</sup> |
|
| 114 |
-
|
| 115 |
-
<sub>β‘ VideoMAEv2 ViT-g: K400 top-1 after **full fine-tuning**, not the frozen attentive-probe protocol used for the other rows.</sub>
|
| 116 |
|
| 117 |
Protocol: DINOv3-style linear/attentive probes for image tasks; V-JEPA 2-style protocol for
|
| 118 |
-
video. Baseline DAVIS / ADE20K / K400 figures are taken from the DINOv3 technical
|
| 119 |
-
|
| 120 |
-
are not
|
| 121 |
-
OpenCLIP's remaining benchmarks are not reported under a comparable protocol.
|
| 122 |
|
| 123 |
Motif is state of the art on DAVIS video tracking at every resolution (74.0 / 80.5 / 83.5 J&F)
|
| 124 |
and stays competitive on the other image and video benchmarks, using roughly 1/3 of DINOv3's
|
|
|
|
| 46 |
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
|
| 47 |
image = load_image(url)
|
| 48 |
|
| 49 |
+
repo = "Motif-Technologies/Motif-Vision-Encoder"
|
| 50 |
processor = AutoImageProcessor.from_pretrained(repo)
|
| 51 |
model = AutoModel.from_pretrained(repo, trust_remote_code=True, dtype=torch.bfloat16).to("cuda").eval()
|
| 52 |
|
|
|
|
| 94 |
## Evaluation
|
| 95 |
|
| 96 |
Compared against the strongest publicly reported self-supervised / vision backbones. Higher is
|
| 97 |
+
better for every column. Best comparable value per column in bold, second best <u>underlined</u>.
|
|
|
|
| 98 |
|
| 99 |
DAVIS S/M/L follow the DINOv3 protocol (J&F-mean at video short side 420/480, 840/960,
|
| 100 |
1260/1440 px). V-JEPA 2.1 is not part of the DINOv3 Table 5 tracking benchmark, so only its
|
|
|
|
| 109 |
| SigLIP2 | 10B | 56.1 | 62.3 | 62.9 | <u>89.1</u> | 45.4 | 86.9 |
|
| 110 |
| OpenCLIP | 2B | β | β | β | β | β | β |
|
| 111 |
| V-JEPA 2.1 | 0.022B | 69.0 | β | β | 85.5 | 47.9 | 87.7 |
|
|
|
|
|
|
|
|
|
|
| 112 |
|
| 113 |
Protocol: DINOv3-style linear/attentive probes for image tasks; V-JEPA 2-style protocol for
|
| 114 |
+
video. Baseline DAVIS / ADE20K / K400 figures are taken from the DINOv3 technical report's
|
| 115 |
+
unified evaluation (Tab. 3, 5, 6) and ImageNet from Tab. 7; OpenCLIP is not in that report and
|
| 116 |
+
its benchmarks are not reported under a comparable protocol.
|
|
|
|
| 117 |
|
| 118 |
Motif is state of the art on DAVIS video tracking at every resolution (74.0 / 80.5 / 83.5 J&F)
|
| 119 |
and stays competitive on the other image and video benchmarks, using roughly 1/3 of DINOv3's
|