MiniCPM-V-4.6 Core AI card

README.md

@@ -39,10 +39,17 @@ linear attention ×3 : full attention ×1, head_dim 256, vocab 248094, tied head
 
 ## Bundles
 
+**Recommended (optimized, 2026-06-25):**
+
 | path | what | dtype | size |
 |---|---|---|---|
-| `gpu-pipelined/minicpmv46_vlm_decode_int8lin/` | VLM text decoder (`input_ids → logits` + a static `image_embeds[64,1024]` buffer; in-graph gather `ids ≥ V ? image_embeds[ids-V] : embed[ids]`) | int8 (per-block-32 linear) | ~1.0 GB |
-| `gpu-pipelined/minicpmv46_vision/` | fixed-grid SigLIP vision encoder (`pixel_values[1,3,448,448] → image_features[64,1024]`) | fp16 | ~1.0 GB |
+| `gpu-pipelined/minicpmv46_vlm_decode_int8hu/` | VLM text decoder (`input_ids → logits` + static `image_embeds[64,1024]`; in-graph gather `ids ≥ V ? image_embeds[ids-V] : embed[ids]`) | int8 body + **untied int8 head** | ~1.2 GB |
+| `gpu-pipelined/minicpmv46_vision_int8lin/` | fixed-grid SigLIP vision encoder (`pixel_values[1,3,448,448] → image_features[64,1024]`) | **int8** | ~0.6 GB |
+
+The **int8 head** quantizes the big-vocab LM head (fp16 in `int8lin` = ~half the per-token read) → **+48% decode
+on iPhone 17 Pro** (46→68 tok/s). The **int8 vision** halves the encoder's size (the encode is compute-bound, so
+this is a size/memory win); pair it with a one-shot vision-graph warmup at load to hide the ~2.7 s first-photo
+cold compile. Original `…_int8lin` decoder + fp16 `minicpmv46_vision` remain for compatibility.
 
 The decoder is a complete qwen3.5-hybrid text LLM when `image_embeds` is zero — same bundle, no image needed.
 
@@ -62,8 +69,10 @@ Simpler than the [Qwen3-VL port](https://huggingface.co/mlboydaisuke/Qwen3-VL-2B
 
 ## Measured (iPhone 17 Pro, iOS 27 beta, release)
 
-- **iPhone 17 Pro decode ~51.5 tok/s** (text core 53.4) · **M4 Max ~224 tok/s** (text core, `llm-benchmark`),
-  engine cold-spec ~3–5 s, ~1.5 GB resident (jetsam-safe).
+- **iPhone 17 Pro decode: int8hu (recommended) ~68 tok/s vs int8lin ~46 (+48%, from the int8 head)** ·
+  M4 Max text core ~224 tok/s (`llm-benchmark`), engine cold-spec ~2–4 s, ~1.5 GB resident (jetsam-safe).
+- **Vision**: int8 encoder ≈ fp16 encode time (compute-bound) at ~0.6 GB (half); the ~2.7 s first-image latency
+  is the SigLIP graph's cold compile — run a dummy encode at load to make the user's first photo warm (~tens of ms).
 - **Numerics**: fp32-torch parity bit-exact (vision cos 1.000000, full overlay logits cos 1.00004);
   Core AI engine ≡ python ≡ HF (text 24/24; image path reproduces the HF description modulo one int8
   near-tie token, then reconverges).