mlboydaisuke commited on
Commit
e289b14
Β·
verified Β·
1 Parent(s): 5de65b6

gen-cards: regenerate Use-it block

Browse files
Files changed (1) hide show
  1. README.md +41 -0
README.md CHANGED
@@ -27,6 +27,47 @@ head (174 classes of *physical interactions* β€” put/lift/push/roll/cover/preten
27
  - **Speed**: ~150–180 ms per 16-frame clip on an M4 Max (GPU) β€” real-time video understanding.
28
 
29
  <!-- gen-cards:use-it begin id=vjepa2-vitl-ssv2 (managed by scripts/gen-cards β€” edit cards.json / QuickStart.swift, not this block) -->
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  <!-- gen-cards:use-it end -->
31
 
32
  ## Files
 
27
  - **Speed**: ~150–180 ms per 16-frame clip on an M4 Max (GPU) β€” real-time video understanding.
28
 
29
  <!-- gen-cards:use-it begin id=vjepa2-vitl-ssv2 (managed by scripts/gen-cards β€” edit cards.json / QuickStart.swift, not this block) -->
30
+ ## Use it
31
+
32
+ ▢️ **Run it (source)** β€” the [ActionCamera runner](https://github.com/john-rocky/coreai-kit/tree/main/Examples/ActionCamera)
33
+ (live camera action recognition, one app for every video model in the catalog):
34
+
35
+ ```bash
36
+ git clone https://github.com/john-rocky/coreai-kit
37
+ open coreai-kit/Examples/ActionCamera/ActionCamera.xcodeproj
38
+ # β†’ Run, then pick "V-JEPA 2 ViT-L (SSv2)" in the model picker
39
+
40
+ # agents / headless (macOS):
41
+ cd coreai-kit/Examples/ActionCamera
42
+ swift run action-cli --model vjepa2-vitl-ssv2 --video sample.mp4
43
+ ```
44
+
45
+ πŸ’» **Build with it** β€” complete; the glue is kit API, copy-paste runs:
46
+
47
+ ```swift
48
+ import CoreAIKitVision
49
+
50
+ let recognizer = try await ActionRecognizer(catalog: "vjepa2-vitl-ssv2")
51
+ let actions = try await recognizer.classify(videoAt: videoURL, topK: 3)
52
+ // actions: ranked [Prediction] β€” .label ("Pushing [something] from left to right"),
53
+ // .probability; 174 SSv2 classes, fully on-device
54
+ ```
55
+
56
+ The take-home is [`Examples/ActionCamera/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/main/Examples/ActionCamera/Sources/QuickStart.swift)
57
+ β€” this exact code as one typed function, no UI; the CLI is an argument shell over it, and
58
+ the GUI classifies a rolling 16-frame clip from `CameraFeed`.
59
+ Live camera? Keep the last 16 `CameraFeed` frames and call `classify(frames:)` β€” other
60
+ frame counts are uniformly resampled to 16. The bundled `sample.mp4` is a synthetic
61
+ clip (a hand pushing a block); point `--video` at real footage for real results.
62
+
63
+ **Integration checklist**
64
+
65
+ - SPM: `https://github.com/john-rocky/coreai-kit` β†’ product **CoreAIKitVision**
66
+ - Info.plist: `NSCameraUsageDescription` β€” only for the live camera; the snippet needs none
67
+ - Entitlements: none needed
68
+ - First run downloads the model β€” 0.7 GB (Mac) / 1.4 GB (iPhone) β€” then it loads from the
69
+ local cache (Application Support; progress via the `downloadProgress` callback)
70
+ - Measure in Release β€” Debug is ~3Γ— slower on per-token host work
71
  <!-- gen-cards:use-it end -->
72
 
73
  ## Files