README: the demo paragraph links the 14.5 s pumpkin clip in place of the kitten one; SHA256SUMS: the README line

5e225fa verified about 8 hours ago

8.42 kB

---
license: apache-2.0
base_model: tencent/WeVisOne-VL-2B
base_model_relation: quantized
pipeline_tag: image-segmentation
library_name: coreai
tags: [coreai, apple, on-device, iphone, image-matting, referring-expression-segmentation, object-detection]
---
# WeVisOne-VL-2B — Core AI

On an iPhone 18 Pro (iOS 27.2), cutting out what a few words name in a photo — each match's box and its alpha matte —
took 0.99–1.00 s for one match and 4.1 s for seven with these graphs, measured 2026-10-25 in CoreAIKit's
`Examples/Cutout`. This repository holds Tencent's [WeVisOne-VL-2B](https://huggingface.co/tencent/WeVisOne-VL-2B)
(revision `3dad9ce8`) for Apple's Core AI as four graphs: the text decoder (int8, with its last hidden state as a
second output), the vision tower with the pixel decoder's inputs (fp16), and the 9.5M-parameter pixel decoder in two
parts (fp16).

| Variant | Path | Size | Requires | Tested on |
|---|---|---|---|---|
| GPU, specialized on the device (`.aimodel`) | `ios/` | 3,298,811,031 B | iOS 27 and an app with `com.apple.developer.kernel.increased-memory-limit` | iPhone 18 Pro, iOS 27.2 (24B5099f), 2026-10-25 |
| GPU, compiled ahead of time (`.h16c.aimodelc`) | `macos/` | 10,188,197,010 B | macOS 27 on a Mac16,x (the `h16c` architecture) | Mac M4 Max, macOS 27.0 (26A428), 2026-10-25 |

Each folder is the model folder the Cutout example reads: name it `WeVisOne-VL-2B-CoreAI` and put it in the app's
Documents (iPhone) or `~/Library/Application Support/Cutout/` (Mac).

```sh
hf download mlboydaisuke/WeVisOne-VL-2B-CoreAI --include "ios/*" --local-dir wevisone   # or "macos/*"
```

The graphs need a host that runs them in order; they are not a CoreAIKit catalog model. The Cutout example
([CoreAIKit `Examples/Cutout`](https://github.com/john-rocky/coreai-kit/tree/main/Examples/Cutout)) is that host:
pick or take a photo, type what to cut out, and save the cut-out as a PNG with the background removed.

A recorded run on an iPhone 18 Pro — a photo of three lit jack-o'-lanterns among small uncarved pumpkins, "every carved
pumpkin" typed: 3 found, boxes 0.83 s and matte 1.42 s on the screen, only the three carved ones cut out, then lined up
on one sheet and saved to Photos — is the 14.5 s clip
[wevisone-vl-2b-pumpkins-iphone-20261027.mp4](https://github.com/john-rocky/coreai-assets/blob/main/demos/wevisone-vl-2b-pumpkins-iphone-20261027.mp4)
(the photo is CC0).

## What each folder holds

| file | what | bytes (`ios/`) |
|---|---|---:|
| `wevisone_vl_2b_decode_int8hu_pf64/` | the text decoder: `main` (S = 1) and `prefill` (S = 64) sharing two KV states; outputs `logits` and `last_hidden` (the last decoder layer's output before the final norm, which the pixel decoder reads at each `<OBJ>`); linears int8 per block of 32 (body `symmetric_with_clipping`, the untied head absmax); `metadata.json` and `tokenizer/` (the source's `tokenizer.json`, `tokenizer_config.json` and `chat_template.jinja`, byte for byte) | 2,467,304,523 |
| `wevisone_vl_2b_vision_taps_g14.aimodel` | the vision tower for a 448 × 448 image (196 rows): `image_embeds`, `deepstack_embeds` and `taps`, the outputs of ViT layers 5 / 11 / 17 / 23 the pixel decoder reads; fp16 | 811,114,774 |
| `wv_mask_a_g28_n16_fp16_hf.aimodel` | pixel decoder part A: up to 16 matches per call, the box encoding (`box_pe`) given by the host; fp16 | 19,540,579 |
| `wv_mask_b_fp16.aimodel` | pixel decoder part B: one call per match, the 1024 × 1024 matte logits; fp16 | 851,155 |

`macos/` holds the same four graphs compiled for the Mac (`--platform macOS --architecture h16c`, the decoder with
`--expect-frequent-reshapes`; the compile stores an fp16 copy of the decoder's int8 linears). Each `.aimodel` is its
export with the MLIR debug locations removed (`strip_debug_info`; op counts and `coreai-build inspect` equal to the
export's). `config.json` is the source checkpoint's, unchanged; `SHA256SUMS` lists every other file.

## How a request runs

The host does what the source package's `predict(task="segmentation")` does, with the photo resized to a fixed
448 × 448 grid:

1. Pillow's bicubic resize to 448 × 448 and to 1024 × 1024 (the Cutout host writes it in integer arithmetic: the same
   bytes as Pillow on PNG input);
2. the vision tower; the source's chat template with the package's segmentation prompt (its matting task uses the same
   one) and the system line "You are a helpful assistant.";
3. the decoder, greedy, no repetition penalty; the prompt in 64-token `prefill` calls with the tail padded into one
   more; `last_hidden` kept at each `<OBJ>` it writes;
4. the boxes by the package's `parse_seg_boxes` (four per-mille coordinates before each `<OBJ>`); their Fourier
   encoding on the host in fp32 — it has to equal torch's bit for bit, including the 7 box widths where Darwin's
   `logf` rounds differently from torch's `log`;
5. part A once per 16 matches, part B once per match; sigmoid and a bilinear resize to the photo's size.

The decoder runs on Core AI's low-level runtime, not the pipelined engine, because the engine returns token ids only
and never the hidden state.

## Decoder accuracy

Against the source package in fp32 on the same fixed grid (33 rows), the int8 decoder gives the reference's argmax at
the last four prompt positions on 33 of 33 rows and 16 greedy tokens that never leave the reference's except at its
own near-ties (top-2 softmax gap under 0.1) on 33 of 33. Its logits are not within cosine 0.9999 of the reference at
every one of 16 teacher-forced steps on any row (lowest 0.99581); no decoder with fp16 weights reaches that on every
row either (fp16 31 of 33, fp16 weights with fp32 compute 32 of 33). On the task level it finds every one of the
reference's boxes at IoU ≥ 0.75 and gives every count exactly.

## Measured

- **iPhone 18 Pro** (iOS 27.2, the Cutout app, graphs specialized on the phone, warm, thermal state nominal): the
  lighthouse example (1024 × 683, one match, a 13-token answer) 0.988–0.997 s — tower 89 ms, the 237-token prompt
  141 ms, the answer 233 ms, part A 75 ms, part B 403 ms; seven chess pieces (a 50-token answer) 4.119–4.138 s, 2.77 s
  of it part B. The first launch after installing reached READY in 6.85 s and took 8.14 s for its first request;
  later launches reached READY in 1.5–2.6 s. Peak memory footprint 2.00 GB.
- **Mac M4 Max** (macOS 27.0, `macos/`): one match in a 3744 × 5616 NASA photo (decoded at 2731 × 4096) 0.545–0.554 s
  (the PNG another 0.29 s); peak footprint 2.8 GB.

## Accuracy

Against the source package in fp32 (transformers 5.12.1, CPU), the photo resized to 448 × 448 first:

- structured answers of the decoder: boxes found at IoU ≥ 0.75: 100.0 % (63); counts exact: 100.0 % (7); points
  within 2 % of the side: 89.4 % (47); OCR character error rate 0.000;
- the pixel decoder's two graphs on the reference's inputs: mattes within MAE 2.09e-4 and IoU(α > 0.5) ≥ 0.992 of the
  package's fp32 decoder (46 matches);
- the whole request on the iPhone 18 Pro, on the rows whose boxes equal the reference's (5 rows, 6 matches): mattes
  within MAE 5.81e-4 and IoU ≥ 0.9959 of the package's.

## Limits

- **The fixed 448 × 448 grid.** The source package keeps the photo's aspect ratio (up to 2025 × 32² pixels); these
  graphs stretch it. Against the package's own form, the fixed grid's fp32 answers find 85.5 % of its boxes at
  IoU ≥ 0.75, 71.4 % of its counts and 81.8 % of its points.
- **1,024 positions on iOS.** Core AI's pipelined engine stops a dynamic-KV decoder at 1,024 positions of prompt and
  answer on iOS (apple/coreai-models#124); the Cutout host binds its cache at 1,024: 237 prompt tokens and up to 787
  answer tokens.
- **No repetition penalty.** The package's greedy decoding uses 1.05; in fp32 the two agree on 26 of 33 answers, with
  every box and count the same.

## Convert / verify

The exporters, the gates and the records: the Core AI model zoo's
[`models/wevisone-vl-2b`](https://github.com/john-rocky/coreai-model-zoo/tree/main/models/wevisone-vl-2b) card and
[`conversion/wevisone/`](https://github.com/john-rocky/coreai-model-zoo/tree/main/conversion/wevisone), on
coreai-torch 0.4.1 / coreai-core 1.0.0b2.

## License

Apache-2.0, as the source model. `LICENSE` is the source repository's file, unchanged: it names Qwen3-VL-2B-Instruct
(Alibaba Cloud, Apache-2.0) as a component modified by Tencent.

	---
	license: apache-2.0
	base_model: tencent/WeVisOne-VL-2B
	base_model_relation: quantized
	pipeline_tag: image-segmentation
	library_name: coreai
	tags: [coreai, apple, on-device, iphone, image-matting, referring-expression-segmentation, object-detection]
	---

	# WeVisOne-VL-2B — Core AI

	On an iPhone 18 Pro (iOS 27.2), cutting out what a few words name in a photo — each match's box and its alpha matte —
	took 0.99–1.00 s for one match and 4.1 s for seven with these graphs, measured 2026-10-25 in CoreAIKit's
	`Examples/Cutout`. This repository holds Tencent's [WeVisOne-VL-2B](https://huggingface.co/tencent/WeVisOne-VL-2B)
	(revision `3dad9ce8`) for Apple's Core AI as four graphs: the text decoder (int8, with its last hidden state as a
	second output), the vision tower with the pixel decoder's inputs (fp16), and the 9.5M-parameter pixel decoder in two
	parts (fp16).

	\| Variant \| Path \| Size \| Requires \| Tested on \|
	\|---\|---\|---\|---\|---\|
	\| GPU, specialized on the device (`.aimodel`) \| `ios/` \| 3,298,811,031 B \| iOS 27 and an app with `com.apple.developer.kernel.increased-memory-limit` \| iPhone 18 Pro, iOS 27.2 (24B5099f), 2026-10-25 \|
	\| GPU, compiled ahead of time (`.h16c.aimodelc`) \| `macos/` \| 10,188,197,010 B \| macOS 27 on a Mac16,x (the `h16c` architecture) \| Mac M4 Max, macOS 27.0 (26A428), 2026-10-25 \|

	Each folder is the model folder the Cutout example reads: name it `WeVisOne-VL-2B-CoreAI` and put it in the app's
	Documents (iPhone) or `~/Library/Application Support/Cutout/` (Mac).

	```sh
	hf download mlboydaisuke/WeVisOne-VL-2B-CoreAI --include "ios/" --local-dir wevisone # or "macos/"
	```

	The graphs need a host that runs them in order; they are not a CoreAIKit catalog model. The Cutout example
	([CoreAIKit `Examples/Cutout`](https://github.com/john-rocky/coreai-kit/tree/main/Examples/Cutout)) is that host:
	pick or take a photo, type what to cut out, and save the cut-out as a PNG with the background removed.

	A recorded run on an iPhone 18 Pro — a photo of three lit jack-o'-lanterns among small uncarved pumpkins, "every carved
	pumpkin" typed: 3 found, boxes 0.83 s and matte 1.42 s on the screen, only the three carved ones cut out, then lined up
	on one sheet and saved to Photos — is the 14.5 s clip
	[wevisone-vl-2b-pumpkins-iphone-20261027.mp4](https://github.com/john-rocky/coreai-assets/blob/main/demos/wevisone-vl-2b-pumpkins-iphone-20261027.mp4)
	(the photo is CC0).

	## What each folder holds

	\| file \| what \| bytes (`ios/`) \|
	\|---\|---\|---:\|
	\| `wevisone_vl_2b_decode_int8hu_pf64/` \| the text decoder: `main` (S = 1) and `prefill` (S = 64) sharing two KV states; outputs `logits` and `last_hidden` (the last decoder layer's output before the final norm, which the pixel decoder reads at each `<OBJ>`); linears int8 per block of 32 (body `symmetric_with_clipping`, the untied head absmax); `metadata.json` and `tokenizer/` (the source's `tokenizer.json`, `tokenizer_config.json` and `chat_template.jinja`, byte for byte) \| 2,467,304,523 \|
	\| `wevisone_vl_2b_vision_taps_g14.aimodel` \| the vision tower for a 448 × 448 image (196 rows): `image_embeds`, `deepstack_embeds` and `taps`, the outputs of ViT layers 5 / 11 / 17 / 23 the pixel decoder reads; fp16 \| 811,114,774 \|
	\| `wv_mask_a_g28_n16_fp16_hf.aimodel` \| pixel decoder part A: up to 16 matches per call, the box encoding (`box_pe`) given by the host; fp16 \| 19,540,579 \|
	\| `wv_mask_b_fp16.aimodel` \| pixel decoder part B: one call per match, the 1024 × 1024 matte logits; fp16 \| 851,155 \|

	`macos/` holds the same four graphs compiled for the Mac (`--platform macOS --architecture h16c`, the decoder with
	`--expect-frequent-reshapes`; the compile stores an fp16 copy of the decoder's int8 linears). Each `.aimodel` is its
	export with the MLIR debug locations removed (`strip_debug_info`; op counts and `coreai-build inspect` equal to the
	export's). `config.json` is the source checkpoint's, unchanged; `SHA256SUMS` lists every other file.

	## How a request runs

	The host does what the source package's `predict(task="segmentation")` does, with the photo resized to a fixed
	448 × 448 grid:

	1. Pillow's bicubic resize to 448 × 448 and to 1024 × 1024 (the Cutout host writes it in integer arithmetic: the same
	bytes as Pillow on PNG input);
	2. the vision tower; the source's chat template with the package's segmentation prompt (its matting task uses the same
	one) and the system line "You are a helpful assistant.";
	3. the decoder, greedy, no repetition penalty; the prompt in 64-token `prefill` calls with the tail padded into one
	more; `last_hidden` kept at each `<OBJ>` it writes;
	4. the boxes by the package's `parse_seg_boxes` (four per-mille coordinates before each `<OBJ>`); their Fourier
	encoding on the host in fp32 — it has to equal torch's bit for bit, including the 7 box widths where Darwin's
	`logf` rounds differently from torch's `log`;
	5. part A once per 16 matches, part B once per match; sigmoid and a bilinear resize to the photo's size.

	The decoder runs on Core AI's low-level runtime, not the pipelined engine, because the engine returns token ids only
	and never the hidden state.

	## Decoder accuracy

	Against the source package in fp32 on the same fixed grid (33 rows), the int8 decoder gives the reference's argmax at
	the last four prompt positions on 33 of 33 rows and 16 greedy tokens that never leave the reference's except at its
	own near-ties (top-2 softmax gap under 0.1) on 33 of 33. Its logits are not within cosine 0.9999 of the reference at
	every one of 16 teacher-forced steps on any row (lowest 0.99581); no decoder with fp16 weights reaches that on every
	row either (fp16 31 of 33, fp16 weights with fp32 compute 32 of 33). On the task level it finds every one of the
	reference's boxes at IoU ≥ 0.75 and gives every count exactly.

	## Measured

	- iPhone 18 Pro (iOS 27.2, the Cutout app, graphs specialized on the phone, warm, thermal state nominal): the
	lighthouse example (1024 × 683, one match, a 13-token answer) 0.988–0.997 s — tower 89 ms, the 237-token prompt
	141 ms, the answer 233 ms, part A 75 ms, part B 403 ms; seven chess pieces (a 50-token answer) 4.119–4.138 s, 2.77 s
	of it part B. The first launch after installing reached READY in 6.85 s and took 8.14 s for its first request;
	later launches reached READY in 1.5–2.6 s. Peak memory footprint 2.00 GB.
	- Mac M4 Max (macOS 27.0, `macos/`): one match in a 3744 × 5616 NASA photo (decoded at 2731 × 4096) 0.545–0.554 s
	(the PNG another 0.29 s); peak footprint 2.8 GB.

	## Accuracy

	Against the source package in fp32 (transformers 5.12.1, CPU), the photo resized to 448 × 448 first:

	- structured answers of the decoder: boxes found at IoU ≥ 0.75: 100.0 % (63); counts exact: 100.0 % (7); points
	within 2 % of the side: 89.4 % (47); OCR character error rate 0.000;
	- the pixel decoder's two graphs on the reference's inputs: mattes within MAE 2.09e-4 and IoU(α > 0.5) ≥ 0.992 of the
	package's fp32 decoder (46 matches);
	- the whole request on the iPhone 18 Pro, on the rows whose boxes equal the reference's (5 rows, 6 matches): mattes
	within MAE 5.81e-4 and IoU ≥ 0.9959 of the package's.

	## Limits

	- The fixed 448 × 448 grid. The source package keeps the photo's aspect ratio (up to 2025 × 32² pixels); these
	graphs stretch it. Against the package's own form, the fixed grid's fp32 answers find 85.5 % of its boxes at
	IoU ≥ 0.75, 71.4 % of its counts and 81.8 % of its points.
	- 1,024 positions on iOS. Core AI's pipelined engine stops a dynamic-KV decoder at 1,024 positions of prompt and
	answer on iOS (apple/coreai-models#124); the Cutout host binds its cache at 1,024: 237 prompt tokens and up to 787
	answer tokens.
	- No repetition penalty. The package's greedy decoding uses 1.05; in fp32 the two agree on 26 of 33 answers, with
	every box and count the same.

	## Convert / verify

	The exporters, the gates and the records: the Core AI model zoo's
	[`models/wevisone-vl-2b`](https://github.com/john-rocky/coreai-model-zoo/tree/main/models/wevisone-vl-2b) card and
	[`conversion/wevisone/`](https://github.com/john-rocky/coreai-model-zoo/tree/main/conversion/wevisone), on
	coreai-torch 0.4.1 / coreai-core 1.0.0b2.

	## License

	Apache-2.0, as the source model. `LICENSE` is the source repository's file, unchanged: it names Qwen3-VL-2B-Instruct
	(Alibaba Cloud, Apache-2.0) as a component modified by Tencent.