README.md

---
license: other
tags:
- executorch
- xnnpack
- pte
- on-device
- image-text-to-text
base_model:
- LiquidAI/LFM2.5-VL-450M
---
# LFM2.5-VL-450M — ExecuTorch

`lfm2_5_vl_450m_xnnpack_8da4w.pte` (566 MB)

- **Source**: LiquidAI/LFM2.5-VL-450M — SigLIP 2 vision tower (768 wide, 12 layers) and an
  LFM2 decoder (1024 wide, 16 layers)
- **License**: LFM Open License v1.0
- **Input**: a 512×512 tile as `[1, 1024, 768]` patches plus `[1, 1024]` attention mask,
  and token ids for the words around it
- **Output**: logits over the 64,400-token vocabulary

One file, three entry points — the shape ExecuTorch's multimodal runner asks for:

| method | in | out |
|---|---|---|
| `vision_encoder` | patches, mask | `[1, 256, 1024]` rows in the decoder's embedding space |
| `token_embeddings` | token ids | embeddings |
| `text_model` | embeddings, positions | logits, and its own cache |

A decoder that only takes token ids cannot be told about a picture. Splitting a
vision-language model into a vision `.pte` and a text `.pte` runs aground there;
`MultimodalPrefiller::load` asks one module for `token_embeddings` and `text_model` and uses
`vision_encoder` if it finds it.

## On an iPhone 17 Pro

XNNPACK, CPU, Release build: opens in **0.3 s**, answers at **37 tok/s** (96 tokens in
2.6 s). Measured in the sample app below, not extrapolated from a single inference.

## Verification (Mac arm64, 2026-08-21)

Greedy, through the three methods, on photographs from `convert/calib_images`:

| picture | answer |
|---|---|
| a London street | "A bustling street scene with people walking, outdoor seating, and various storefronts, including a prominent Pizza Express." |
| a man with a dog | "A man in a hat and overalls stands next to a wagon loaded with logs, with a water tower in the background." |
| a studio portrait | "A man in a gray long-sleeve shirt poses against a white background…" |

The shop sign is read correctly, which is the check that matters: a caption that fits any
street would not tell you the vision half was wired up right.

The vision half of this file agrees with the untouched model at **worst corr 0.98877** over
24 photographs.

## Square the picture first

The processor picks a tile grid from the aspect ratio — a 768×477 photograph becomes one
24×40 tile, a 1280×960 one becomes seven — and this graph takes 32×32. Centre-crop to a
square before the processor sees it and every picture becomes exactly one tile. Feeding a
stretched square instead is visible in the output: asked about a squashed street, the model
called the scene "distorted and warped", which was a fair description of what it had been
given.

```bash
python convert/run_vl.py <image> "What is in this picture?"
```

## Conversion

`convert/export_vl_bundle.py`. Four things needed re-authoring:

- **SigLIP 2 reads its grid out of a tensor** to size the position embeddings, which
  `torch.export` cannot follow. The grid is fixed here, so the resize is computed once and
  the constant handed to a replacement forward.
- **The projector wants the grid back.** The tower returns a flat run of patches; the
  projector's pixel-unshuffle trades resolution for channels and needs to know which patches
  are neighbours.
- **The quantization is not uniform.** The decoder is int4 over groups of 32; the vision
  tower is int8 per output channel. At int4 throughout, the model reads the Pizza Express
  sign as "Pocket Express" — the letters live in the tower, and four bits does not hold
  them.
- **The cache lives inside `text_model`.** ExecuTorch copies a mutable buffer into each
  method that names it (`Program::load_mutable_subsegment_into` writes into the method's own
  memory), so a prefill method and a decode method would each get their own and neither
  would see the other's writes. One graph has to serve both, which rules out the two code
  paths transformers keeps for LFM2's short convolution — a windowed convolution for a
  prompt, a fused single-step update for a token, chosen in Python and therefore baked in by
  tracing. Carrying the last `kernel - 1` columns and putting them in front of whatever
  arrives is the same arithmetic in one branch-free path.

## The number that decides whether it runs on a phone

`CONTEXT`, the upper bound on the dynamic sequence dimension. The memory planner sizes its
arena for the bound, not for what a picture costs: at 4096 that arena is **3.7 GB** and iOS
kills the process with signal 9 before the first method has finished loading. One square
photograph is 1024 patches, which the projector unshuffles to 256 rows, so 512 leaves room
for a long question and a long answer and brings the arena to 164 MB.

(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models) ·
iOS sample: [executorch-samples](https://github.com/john-rocky/executorch-samples))