mlboydaisuke commited on
Commit
3014e93
·
verified ·
1 Parent(s): e0d417a

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +45 -0
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ base_model: facebook/vjepa2-vitl-fpc16-256-ssv2
4
+ tags:
5
+ - coreai
6
+ - apple
7
+ - video-classification
8
+ - v-jepa
9
+ - world-model
10
+ - on-device
11
+ pipeline_tag: video-classification
12
+ ---
13
+
14
+ # V-JEPA 2 (ViT-L, SSv2 action recognition) — Apple Core AI
15
+
16
+ [V-JEPA 2](https://huggingface.co/facebook/vjepa2-vitl-fpc16-256-ssv2) (Meta AI) running natively on
17
+ the Apple Core AI engine — the zoo's first **world model**: a self-supervised video encoder that
18
+ learns by predicting in representation space (JEPA), here with the Something-Something v2 action
19
+ head (174 classes of *physical interactions* — put/lift/push/roll/cover/pretend…).
20
+
21
+ - **One bundle**: ViT-L backbone (3D RoPE attention) + attentive pooler + classifier, ~375M params,
22
+ fp16 ~675 MB.
23
+ - **I/O**: `pixel_values_videos [1,16,3,256,256]` (16 frames, RGB 0..1, ImageNet mean/std) →
24
+ `logits [1,174]` (`labels.json`).
25
+ - **Verified**: engine vs PyTorch reference cosine 0.999996, top-5 identical; a synthetic
26
+ motion probe (square moving up vs down) flips the predicted direction correctly.
27
+ - **Speed**: ~150–180 ms per 16-frame clip on an M4 Max (GPU) — real-time video understanding.
28
+
29
+ ## Files
30
+
31
+ | path | what |
32
+ |---|---|
33
+ | `macos/vjepa2_ssv2_fp16.aimodel` | fp16 bundle (macOS / JIT) |
34
+ | `macos/labels.json` | 174 SSv2 class names |
35
+ | `macos/metadata.json` | I/O + preprocessing spec |
36
+
37
+ ## Preprocessing
38
+
39
+ Sample 16 frames uniformly from the clip, resize+center-crop to 256×256, scale to 0..1, normalize
40
+ with ImageNet mean `[0.485,0.456,0.406]` / std `[0.229,0.224,0.225]`, layout `[1,16,3,256,256]`.
41
+
42
+ ## Credits
43
+
44
+ - **Meta AI** — [V-JEPA 2](https://github.com/facebookresearch/vjepa2) (MIT).
45
+ - Conversion + Core AI port: [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo).