Add SAM 2.1 video-tracking graphs (memory attention/encoder, 4 per-frame GPU graphs) + card section

README.md

@@ -79,6 +79,58 @@ enc.invoke()
 flat = enc.get_tensor(enc.get_output_details()[0]["index"]).flatten()  # image_embed | fpn0 | fpn1
 ```
 
+
+## Video tracking (SAM 2 video path)
+
+The full SAM 2.1 tracking loop — memory attention, memory encoder, object pointers and the
+prompt-conditioned mask decoder — as four fixed-shape per-frame graphs on the CompiledModel
+GPU. The rolling memory bank and per-frame orchestration run on the host (Kotlin/Swift/Python);
+only tensor math touches the GPU. Tap once on the first frame and the mask follows the object.
+
+| File | Size (fp16) | In → Out |
+|---|---|---|
+| `sam2v_encode.tflite` | 80 MB | image `[1,3,1024,1024]` → `pix_raw \| hi0 \| hi1` |
+| `sam2v_memcond7.tflite` / `sam2v_memcond2.tflite` | 26 MB | `pix_raw \| memory bank \| temporal pos \| pointers \| key mask` → `pix_feat` (7- / 2-slot bank) |
+| `sam2v_decode.tflite` | 18 MB | `pix_feat \| hi0 \| hi1 \| sparse \| nomem` → `masks \| iou \| obj_ptr \| obj_score` |
+| `sam2v_memorize.tflite` | 3 MB | `pix_raw \| mask_for_mem \| occ` → spatial memory `[4096, 64]` |
+| `sam2v_prompt.bin`, `sam2v_track_sparse.bin`, `sam2v_mtpe.bin`, `sam2v_no_obj_ptr.bin`, `sam2v_tpos_proj.bin` | ≤64 KB | host-side constants (prompt encoder, temporal PE, pointer projection) | |
+
+**Why it works on the GPU:** SAM 2's memory attention runs its RoPE attention with the batch
+dim collapsed (rank 3), which the ML Drift delegate silently mis-computes — the graphs here are
+re-authored **batch-first (rank 4)**, numerically identical on the host and correct on the GPU
+(exact under fp32 GPU compute). The residual fp16 accumulation over the memory keys does not
+reach the mask.
+
+**Fidelity:** the assembled loop matches the PyTorch `Sam2VideoModel` reference at min
+mask-IoU **0.9999** over a 10-frame clip (7- and 2-slot banks). All four graphs are fully
+GPU-resident with no CPU fallback — Pixel 8a (Mali): encode 828/828, memcond 480/480, decode
+462/462, memorize 145/145 nodes; iPhone 17 Pro (Metal): all `fullyGPU`.
+
+**Per tracked frame** (encode + memcond + decode + memorize): iPhone 17 Pro **~471 ms**
+(2-slot) / ~751 ms (7-slot); Pixel 8a ~1.0–1.5 s.
+
+### Usage (Python, CompiledModel)
+
+```python
+import numpy as np
+from ai_edge_litert.compiled_model import CompiledModel
+
+enc = CompiledModel.from_file("sam2v_encode.tflite")
+ins, outs = enc.create_input_buffers(0), enc.create_output_buffers(0)
+ins[0].write(np.ascontiguousarray(frame_nchw.ravel().astype(np.float32)))
+enc.run_by_index(0, ins, outs)
+flat = outs[0].read(4_194_304, np.float32)  # pix_raw | hi0 | hi1
+# memcond -> decode -> memorize per frame; the full host loop (bank assembly,
+# best-IoU pick, no-object handling) is verify_video.py in the recipe below.
+```
+
+### Usage (Kotlin)
+
+The Android tracker (`Sam2VideoTracker.kt`: filesDir load, rolling bank, per-frame loop) and
+the demo app live in [LiteRT-Models](https://github.com/john-rocky/LiteRT-Models) → `sam2/`.
+The conversion + verification recipe is in litert-samples:
+[`models/sam2/sam2_hiera_tiny_video/converted`](https://github.com/google-ai-edge/litert-samples/pull/283).
+
 ## Conversion
 
 Converted with `litert-torch` from the Hugging Face `transformers` SAM 2 model. The full

sam2v_decode.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c07b021f9e8e689e326d61735e03fd8d80fb0f08be1679ae1fe0cc2b5332abe
+size 17846160

sam2v_encode.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6c457a9904e4f3e9d5be0320c37dec08e7e32c275e4dfce6e81d5151b1de2a3
+size 80107568

sam2v_memcond2.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75d00d674194a881d2a4cdd4c80c382cd603ab2dab407a648788270f2d1b6373
+size 25626848

sam2v_memcond7.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc6d3889a5f4c508183c8c5a336ef45eee7c98461ba9629c0afc835c5cbf6316
+size 25626848

sam2v_memorize.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c14d93b934614def046141fb88d63f0e489c255a518cf5767bf1ad8c2b7d84a
+size 2856400

sam2v_mtpe.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a693877939b52b8c3c4cb7c8a6e902611db9757c2c8def4c7fdece3b1c10787
+size 1792

sam2v_no_obj_ptr.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f4f6ab65e1471b59c6e80f6684f156d87b3cc9e34a0528037056b1b8879d6f6
+size 1024

sam2v_prompt.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3510e3807703bc6375d742678025be7edc6e49b1e1214d26abaa408d51b60dff
+size 4096

sam2v_tpos_proj.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5efef57e4473602bbae4a38c85e04f984d0755fee341a23c9479843322dfa2a6
+size 65792

sam2v_track_sparse.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f361f79c5cbf6d28a16e0b62cb3ca09a9603fb95b457426e4266943b1fa34b5
+size 2048