Add model card

README.md

@@ -0,0 +1,73 @@
+---
+license: other
+license_name: youtu-llm
+license_link: https://huggingface.co/tencent/Youtu-LLM-2B/blob/main/LICENSE
+base_model: tencent/Youtu-LLM-2B
+tags:
+  - apple
+  - coreai
+  - aimodel
+  - on-device
+  - mla
+  - youtu
+---
+
+# Youtu-LLM-2B — Apple Core AI (`.aimodel`)
+
+[Youtu-LLM-2B](https://huggingface.co/tencent/Youtu-LLM-2B) (Tencent) converted to Apple
+**Core AI** for iOS 27 / macOS 27 (beta) — the **[zoo](https://github.com/john-rocky/coreai-model-zoo)'s
+first Multi-head Latent Attention (MLA) model that runs on iPhone**, and its first **dense**
+MLA (GLM-4.7-Flash brought MLA to the zoo but as a 30B Mac-only MoE).
+
+Youtu-LLM-2B is a **dense DeepSeek-V2/V3-style MLA** decoder: 1.96B params, 32 layers,
+`kv_lora_rank` 512 · `q_lora_rank` 1536 · `qk_nope` 128 + decoupled `qk_rope` 64 (head_dim
+192) · `v_head_dim` 128 · interleaved RoPE θ=1.6e6 · dense SwiGLU FFN (6144) · 128K context ·
+weight-tied head · Llama-3 tokenizer. It has a **reasoning ("thinking") mode** (`<think>…</think>`)
+and native agentic/tool-use ability.
+
+The MLA decode caches only the **compressed latent** (`[512]` + `[64]` rope key per token,
+2×`[288]` halves) instead of a full per-head K/V, and folds the KV up-projection into the
+query lift / value readout — a tiny KV cache that a custom **absorbed-MLA flash-decode Metal
+kernel** attends. Rides Apple's **`coreai-pipelined` GPU engine** via the decode-only loop-free
+export (async encode, on-GPU argmax sampling, on-device KV growth).
+
+| surface (int8 ship bundle) | prefill (S=1) | decode | numerics |
+|---|---:|---:|---|
+| **M4 Max** (release `llm-runner`, greedy) | 95.9 | **102.8 tok/s** | 16/16 top-1 = HF fp32 oracle |
+| **iPhone 17 Pro** (PipelinedBench p=128 g=256; in-app warm ~24) | 20.5 | **~19 tok/s** | **16/16 · device ≡ Mac ≡ HF** |
+
+Numerics: the authored Core AI model is **token-exact to the fp32 HF reference** (naive +
+absorbed forms, prefill cosine 1.000002, greedy 0 flips), and the int8 bundle running on the
+real GPU engine reproduces the oracle **byte-for-byte, 16/16 on both prompts, on Mac and on the
+iPhone 17 Pro** — the [zoo](https://github.com/john-rocky/coreai-model-zoo) ship gate.
+
+## Use it
+
+This bundle runs in **[CoreAIChat](https://github.com/john-rocky/coreai-model-zoo/tree/main/apps/CoreAIChat)**
+(the zoo's on-device chat app): pick **Youtu-LLM 2B** in the model picker; the app downloads
+`gpu-pipelined/youtu_llm_2b_decode_absorbed_msdpa/` from this repo into `Documents/models/` on
+first run, then loads from the local cache. 2B (≥2 GB) bundles need the app's
+`com.apple.developer.kernel.increased-memory-limit` entitlement (CoreAIChat ships it).
+
+Engine contract (decode-only static-`[1,1]` graph): set `COREAI_CHUNK_THRESHOLD=1` before engine
+creation (prefill runs as pipelined S=1 steps); don't call `engine.warmup()` (it warms query
+length 256) — a 1-token generate after load is the warmup. Chat template:
+`<|begin_of_text|><|User|>…<|Assistant|>` (thinking mode on → `<think>…</think>` then the answer).
+
+## Contents
+
+- `gpu-pipelined/youtu_llm_2b_decode_absorbed_msdpa/` — the int8 per-block-32 (body + head)
+  decode bundle with the absorbed-MLA flash-decode Metal kernel: `metadata.json`, the
+  `.aimodel` (LanguageBundle), and the tokenizer. ~2.2 GB.
+- `config.json` — the source model config (for reference).
+
+Conversion code and the full engineering notes live in the
+[model zoo](https://github.com/john-rocky/coreai-model-zoo) (`conversion/export_youtu_decode_pipelined.py`,
+`models/macos/youtu.py` + `youtu_absorbed.py`; the absorbed-MLA cache + flash-decode kernel are
+shared with GLM-4.7-Flash).
+
+## License
+
+Weights follow Tencent's **`youtu-llm`** license (see `license_link`) — commercial use and
+redistribution of derivatives permitted with attribution; **⚠️ the license states Youtu-LLM is
+NOT intended for use within the European Union.** The Core AI conversion adds no restrictions.