README.md

metadata

license: other
tags:
  - executorch
  - xnnpack
  - pte
  - on-device
  - image-text-to-text
base_model:
  - LiquidAI/LFM2.5-VL-450M

LFM2.5-VL-450M — ExecuTorch

lfm2_5_vl_450m_xnnpack_8da4w.pte (566 MB)

• Source: LiquidAI/LFM2.5-VL-450M — SigLIP 2 vision tower (768 wide, 12 layers) and an LFM2 decoder (1024 wide, 16 layers)
• License: LFM Open License v1.0
• Input: a 512×512 tile as [1, 1024, 768] patches plus [1, 1024] attention mask, and token ids for the words around it
• Output: logits over the 64,400-token vocabulary

One file, three entry points — the shape ExecuTorch's multimodal runner asks for:

method	in	out
vision_encoder	patches, mask	[1, 256, 1024] rows in the decoder's embedding space
token_embeddings	token ids	embeddings
text_model	embeddings, positions	logits, and its own cache

A decoder that only takes token ids cannot be told about a picture. Splitting a vision-language model into a vision .pte and a text .pte runs aground there; MultimodalPrefiller::load asks one module for token_embeddings and text_model and uses vision_encoder if it finds it.

On an iPhone 17 Pro

XNNPACK, CPU, Release build: opens in 0.3 s, answers at 37 tok/s (96 tokens in 2.6 s). Measured in the sample app below, not extrapolated from a single inference.

Verification (Mac arm64, 2026-08-21)

Greedy, through the three methods, on photographs from convert/calib_images:

picture	answer
a London street	"A bustling street scene with people walking, outdoor seating, and various storefronts, including a prominent Pizza Express."
a man with a dog	"A man in a hat and overalls stands next to a wagon loaded with logs, with a water tower in the background."
a studio portrait	"A man in a gray long-sleeve shirt poses against a white background…"

The shop sign is read correctly, which is the check that matters: a caption that fits any street would not tell you the vision half was wired up right.

The vision half of this file agrees with the untouched model at worst corr 0.98877 over 24 photographs.

Square the picture first

The processor picks a tile grid from the aspect ratio — a 768×477 photograph becomes one 24×40 tile, a 1280×960 one becomes seven — and this graph takes 32×32. Centre-crop to a square before the processor sees it and every picture becomes exactly one tile. Feeding a stretched square instead is visible in the output: asked about a squashed street, the model called the scene "distorted and warped", which was a fair description of what it had been given.

python convert/run_vl.py <image> "What is in this picture?"

Conversion

convert/export_vl_bundle.py. Four things needed re-authoring:

• SigLIP 2 reads its grid out of a tensor to size the position embeddings, which torch.export cannot follow. The grid is fixed here, so the resize is computed once and the constant handed to a replacement forward.
• The projector wants the grid back. The tower returns a flat run of patches; the projector's pixel-unshuffle trades resolution for channels and needs to know which patches are neighbours.
• The quantization is not uniform. The decoder is int4 over groups of 32; the vision tower is int8 per output channel. At int4 throughout, the model reads the Pizza Express sign as "Pocket Express" — the letters live in the tower, and four bits does not hold them.
• The cache lives inside text_model. ExecuTorch copies a mutable buffer into each method that names it (Program::load_mutable_subsegment_into writes into the method's own memory), so a prefill method and a decode method would each get their own and neither would see the other's writes. One graph has to serve both, which rules out the two code paths transformers keeps for LFM2's short convolution — a windowed convolution for a prompt, a fused single-step update for a token, chosen in Python and therefore baked in by tracing. Carrying the last kernel - 1 columns and putting them in front of whatever arrives is the same arithmetic in one branch-free path.

The number that decides whether it runs on a phone

CONTEXT, the upper bound on the dynamic sequence dimension. The memory planner sizes its arena for the bound, not for what a picture costs: at 4096 that arena is 3.7 GB and iOS kills the process with signal 9 before the first method has finished loading. One square photograph is 1024 patches, which the projector unshuffles to 256 rows, so 512 leaves room for a long question and a long answer and brings the arena to 164 MB.

(conversion scripts: executorch-models · iOS sample: executorch-samples)