metadata
license: mit
base_model: facebook/vjepa2-vitl-fpc16-256-ssv2
tags:
- coreai
- apple
- video-classification
- v-jepa
- world-model
- on-device
pipeline_tag: video-classification
Mirror of
mlboydaisuke/VJEPA2-ViTL-SSv2-CoreAI— the canonical repo (CoreAI Model Zoo). Updates land there first.
V-JEPA 2 (ViT-L, SSv2 action recognition) — Apple Core AI
V-JEPA 2 (Meta AI) running natively on the Apple Core AI engine — the zoo's first world model: a self-supervised video encoder that learns by predicting in representation space (JEPA), here with the Something-Something v2 action head (174 classes of physical interactions — put/lift/push/roll/cover/pretend…).
- One bundle: ViT-L backbone (3D RoPE attention) + attentive pooler + classifier, ~375M params, fp16 ~675 MB.
- I/O:
pixel_values_videos [1,16,3,256,256](16 frames, RGB 0..1, ImageNet mean/std) →logits [1,174](labels.json). - Verified: engine vs PyTorch reference cosine 0.999996, top-5 identical; a synthetic motion probe (square moving up vs down) flips the predicted direction correctly.
- Speed: ~150–180 ms per 16-frame clip on an M4 Max (GPU) — real-time video understanding.
Files
| path | what |
|---|---|
macos/vjepa2_ssv2_fp16.aimodel |
fp16 bundle (macOS / JIT) |
ios/vjepa2_ssv2_fp16.h18p.aimodelc |
iOS AOT bundle (iPhone, A18 Pro+ GPU) |
macos/labels.json, ios/labels.json |
174 SSv2 class names |
macos/metadata.json |
I/O + preprocessing spec |
Live demo app: coreai-video — camera → live top-3 actions. iPhone 17 Pro: ~0.34 s per 16-frame clip.
Preprocessing
Sample 16 frames uniformly from the clip, resize+center-crop to 256×256, scale to 0..1, normalize
with ImageNet mean [0.485,0.456,0.406] / std [0.229,0.224,0.225], layout [1,16,3,256,256].
Credits
- Meta AI — V-JEPA 2 (MIT).
- Conversion + Core AI port: coreai-model-zoo.