Upload README.md with huggingface_hub

README.md

@@ -11,14 +11,14 @@ base_model:
 ---
 # LFM2.5-VL-450M — ExecuTorch
 
-`lfm2_5_vl_450m_xnnpack.pte` (2.07 GB)
+`lfm2_5_vl_450m_xnnpack_8da4w.pte` (566 MB)
 
 - **Source**: LiquidAI/LFM2.5-VL-450M — SigLIP 2 vision tower (768 wide, 12 layers) and an
   LFM2 decoder (1024 wide, 16 layers)
 - **License**: LFM Open License v1.0
 - **Input**: a 512×512 tile as `[1, 1024, 768]` patches plus `[1, 1024]` attention mask,
   and token ids for the words around it
-- **Output**: logits over the 65,536-token vocabulary
+- **Output**: logits over the 64,400-token vocabulary
 
 One file, three entry points — the shape ExecuTorch's multimodal runner asks for:
 
@@ -26,13 +26,18 @@ One file, three entry points — the shape ExecuTorch's multimodal runner asks f
 |---|---|---|
 | `vision_encoder` | patches, mask | `[1, 256, 1024]` rows in the decoder's embedding space |
 | `token_embeddings` | token ids | embeddings |
-| `text_model` | embeddings, positions | logits |
+| `text_model` | embeddings, positions | logits, and its own cache |
 
 A decoder that only takes token ids cannot be told about a picture. Splitting a
 vision-language model into a vision `.pte` and a text `.pte` runs aground there;
 `MultimodalPrefiller::load` asks one module for `token_embeddings` and `text_model` and uses
 `vision_encoder` if it finds it.
 
+## On an iPhone 17 Pro
+
+XNNPACK, CPU, Release build: opens in **0.3 s**, answers at **37 tok/s** (96 tokens in
+2.6 s). Measured in the sample app below, not extrapolated from a single inference.
+
 ## Verification (Mac arm64, 2026-08-21)
 
 Greedy, through the three methods, on photographs from `convert/calib_images`:
@@ -40,14 +45,14 @@ Greedy, through the three methods, on photographs from `convert/calib_images`:
 | picture | answer |
 |---|---|
 | a London street | "A bustling street scene with people walking, outdoor seating, and various storefronts, including a prominent Pizza Express." |
-| a man with a dog | "A man with a dog is standing in front of a stack of logs with a water tower in the background." |
+| a man with a dog | "A man in a hat and overalls stands next to a wagon loaded with logs, with a water tower in the background." |
 | a studio portrait | "A man in a gray long-sleeve shirt poses against a white background…" |
 
 The shop sign is read correctly, which is the check that matters: a caption that fits any
 street would not tell you the vision half was wired up right.
 
-The vision encoder alone agrees with the untouched model at **corr 0.99996** (XNNPACK fp32,
-against `get_image_features` on real imagery).
+The vision half of this file agrees with the untouched model at **worst corr 0.98877** over
+24 photographs.
 
 ## Square the picture first
 
@@ -64,7 +69,7 @@ python convert/run_lfm2_vl.py <image> "What is in this picture?"
 
 ## Conversion
 
-`convert/export_lfm2_vl_bundle.py`. Two things needed re-authoring:
+`convert/export_lfm2_vl_bundle.py`. Four things needed re-authoring:
 
 - **SigLIP 2 reads its grid out of a tensor** to size the position embeddings, which
   `torch.export` cannot follow. The grid is fixed here, so the resize is computed once and
@@ -72,5 +77,26 @@ python convert/run_lfm2_vl.py <image> "What is in this picture?"
 - **The projector wants the grid back.** The tower returns a flat run of patches; the
   projector's pixel-unshuffle trades resolution for channels and needs to know which patches
   are neighbours.
-
-(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))
+- **The quantization is not uniform.** The decoder is int4 over groups of 32; the vision
+  tower is int8 per output channel. At int4 throughout, the model reads the Pizza Express
+  sign as "Pocket Express" — the letters live in the tower, and four bits does not hold
+  them.
+- **The cache lives inside `text_model`.** ExecuTorch copies a mutable buffer into each
+  method that names it (`Program::load_mutable_subsegment_into` writes into the method's own
+  memory), so a prefill method and a decode method would each get their own and neither
+  would see the other's writes. One graph has to serve both, which rules out the two code
+  paths transformers keeps for LFM2's short convolution — a windowed convolution for a
+  prompt, a fused single-step update for a token, chosen in Python and therefore baked in by
+  tracing. Carrying the last `kernel - 1` columns and putting them in front of whatever
+  arrives is the same arithmetic in one branch-free path.
+
+## The number that decides whether it runs on a phone
+
+`CONTEXT`, the upper bound on the dynamic sequence dimension. The memory planner sizes its
+arena for the bound, not for what a picture costs: at 4096 that arena is **3.7 GB** and iOS
+kills the process with signal 9 before the first method has finished loading. One square
+photograph is 1024 patches, which the projector unshuffles to 256 rows, so 512 leaves room
+for a long question and a long answer and brings the arena to 164 MB.
+
+(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models) ·
+iOS sample: [executorch-samples](https://github.com/john-rocky/executorch-samples))