gen-cards: regenerate Use-it block
Browse files
README.md
CHANGED
|
@@ -27,6 +27,47 @@ head (174 classes of *physical interactions* β put/lift/push/roll/cover/preten
|
|
| 27 |
- **Speed**: ~150β180 ms per 16-frame clip on an M4 Max (GPU) β real-time video understanding.
|
| 28 |
|
| 29 |
<!-- gen-cards:use-it begin id=vjepa2-vitl-ssv2 (managed by scripts/gen-cards β edit cards.json / QuickStart.swift, not this block) -->
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
<!-- gen-cards:use-it end -->
|
| 31 |
|
| 32 |
## Files
|
|
|
|
| 27 |
- **Speed**: ~150β180 ms per 16-frame clip on an M4 Max (GPU) β real-time video understanding.
|
| 28 |
|
| 29 |
<!-- gen-cards:use-it begin id=vjepa2-vitl-ssv2 (managed by scripts/gen-cards β edit cards.json / QuickStart.swift, not this block) -->
|
| 30 |
+
## Use it
|
| 31 |
+
|
| 32 |
+
βΆοΈ **Run it (source)** β the [ActionCamera runner](https://github.com/john-rocky/coreai-kit/tree/main/Examples/ActionCamera)
|
| 33 |
+
(live camera action recognition, one app for every video model in the catalog):
|
| 34 |
+
|
| 35 |
+
```bash
|
| 36 |
+
git clone https://github.com/john-rocky/coreai-kit
|
| 37 |
+
open coreai-kit/Examples/ActionCamera/ActionCamera.xcodeproj
|
| 38 |
+
# β Run, then pick "V-JEPA 2 ViT-L (SSv2)" in the model picker
|
| 39 |
+
|
| 40 |
+
# agents / headless (macOS):
|
| 41 |
+
cd coreai-kit/Examples/ActionCamera
|
| 42 |
+
swift run action-cli --model vjepa2-vitl-ssv2 --video sample.mp4
|
| 43 |
+
```
|
| 44 |
+
|
| 45 |
+
π» **Build with it** β complete; the glue is kit API, copy-paste runs:
|
| 46 |
+
|
| 47 |
+
```swift
|
| 48 |
+
import CoreAIKitVision
|
| 49 |
+
|
| 50 |
+
let recognizer = try await ActionRecognizer(catalog: "vjepa2-vitl-ssv2")
|
| 51 |
+
let actions = try await recognizer.classify(videoAt: videoURL, topK: 3)
|
| 52 |
+
// actions: ranked [Prediction] β .label ("Pushing [something] from left to right"),
|
| 53 |
+
// .probability; 174 SSv2 classes, fully on-device
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
The take-home is [`Examples/ActionCamera/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/main/Examples/ActionCamera/Sources/QuickStart.swift)
|
| 57 |
+
β this exact code as one typed function, no UI; the CLI is an argument shell over it, and
|
| 58 |
+
the GUI classifies a rolling 16-frame clip from `CameraFeed`.
|
| 59 |
+
Live camera? Keep the last 16 `CameraFeed` frames and call `classify(frames:)` β other
|
| 60 |
+
frame counts are uniformly resampled to 16. The bundled `sample.mp4` is a synthetic
|
| 61 |
+
clip (a hand pushing a block); point `--video` at real footage for real results.
|
| 62 |
+
|
| 63 |
+
**Integration checklist**
|
| 64 |
+
|
| 65 |
+
- SPM: `https://github.com/john-rocky/coreai-kit` β product **CoreAIKitVision**
|
| 66 |
+
- Info.plist: `NSCameraUsageDescription` β only for the live camera; the snippet needs none
|
| 67 |
+
- Entitlements: none needed
|
| 68 |
+
- First run downloads the model β 0.7 GB (Mac) / 1.4 GB (iPhone) β then it loads from the
|
| 69 |
+
local cache (Application Support; progress via the `downloadProgress` callback)
|
| 70 |
+
- Measure in Release β Debug is ~3Γ slower on per-token host work
|
| 71 |
<!-- gen-cards:use-it end -->
|
| 72 |
|
| 73 |
## Files
|