README.md

---
license: other
license_name: youtu-llm
license_link: https://huggingface.co/tencent/Youtu-LLM-2B/blob/main/LICENSE
base_model: tencent/Youtu-LLM-2B
tags:
  - apple
  - coreai
  - aimodel
  - on-device
  - mla
  - youtu
---

# Youtu-LLM-2B — Apple Core AI (`.aimodel`)

[Youtu-LLM-2B](https://huggingface.co/tencent/Youtu-LLM-2B) (Tencent) converted to Apple
**Core AI** for iOS 27 / macOS 27 (beta) — the **[zoo](https://github.com/john-rocky/coreai-model-zoo)'s
first Multi-head Latent Attention (MLA) model that runs on iPhone**, and its first **dense**
MLA (GLM-4.7-Flash brought MLA to the zoo but as a 30B Mac-only MoE).

Youtu-LLM-2B is a **dense DeepSeek-V2/V3-style MLA** decoder: 1.96B params, 32 layers,
`kv_lora_rank` 512 · `q_lora_rank` 1536 · `qk_nope` 128 + decoupled `qk_rope` 64 (head_dim
192) · `v_head_dim` 128 · interleaved RoPE θ=1.6e6 · dense SwiGLU FFN (6144) · 128K context ·
weight-tied head · Llama-3 tokenizer. It has a **reasoning ("thinking") mode** (`<think>…</think>`)
and native agentic/tool-use ability.

The MLA decode caches only the **compressed latent** (`[512]` + `[64]` rope key per token,
2×`[288]` halves) instead of a full per-head K/V, and folds the KV up-projection into the
query lift / value readout — a tiny KV cache that a custom **absorbed-MLA flash-decode Metal
kernel** attends. Rides Apple's **`coreai-pipelined` GPU engine** via the decode-only loop-free
export (async encode, on-GPU argmax sampling, on-device KV growth).

| surface (int8 ship bundle) | prefill (S=1) | decode | numerics |
|---|---:|---:|---|
| **M4 Max** (release `llm-runner`, greedy) | 95.9 | **102.8 tok/s** | 16/16 top-1 = HF fp32 oracle |
| **iPhone 17 Pro** (PipelinedBench p=128 g=256; in-app warm ~24) | 20.5 | **~19 tok/s** | **16/16 · device ≡ Mac ≡ HF** |

Numerics: the authored Core AI model is **token-exact to the fp32 HF reference** (naive +
absorbed forms, prefill cosine 1.000002, greedy 0 flips), and the int8 bundle running on the
real GPU engine reproduces the oracle **byte-for-byte, 16/16 on both prompts, on Mac and on the
iPhone 17 Pro** — the [zoo](https://github.com/john-rocky/coreai-model-zoo) ship gate.

## Use it

**[CoreAIKit](https://github.com/john-rocky/coreai-kit)** (SPM) — one line, on-device:

```swift
import CoreAIKit

let chat = try await ChatSession(catalog: "youtu-llm-2b")
let reply = try await chat.respond(to: "What can you do, offline?")
// downloads this repo once, then runs fully on-device; the <think> reasoning
// arrives as .thinking events, reply.content is the final answer
```

Or the [ChatDemo runner](https://github.com/john-rocky/coreai-kit/tree/main/Examples/ChatDemo)
(GUI + `swift run chat-cli --model youtu-llm-2b`), or the zoo's
[CoreAIChat](https://github.com/john-rocky/coreai-model-zoo/tree/main/apps/CoreAIChat) app
(pick **Youtu-LLM 2B**). 2B (≥2 GB) bundles need the host app's
`com.apple.developer.kernel.increased-memory-limit` entitlement.

Engine contract (decode-only static-`[1,1]` graph): set `COREAI_CHUNK_THRESHOLD=1` before engine
creation (prefill runs as pipelined S=1 steps); don't call `engine.warmup()` (it warms query
length 256) — a 1-token generate after load is the warmup. Chat template:
`<|begin_of_text|><|User|>…<|Assistant|>` (thinking mode on → `<think>…</think>` then the answer).

## Contents

- `gpu-pipelined/youtu_llm_2b_decode_absorbed_msdpa/` — the int8 per-block-32 (body + head)
  decode bundle with the absorbed-MLA flash-decode Metal kernel: `metadata.json`, the
  `.aimodel` (LanguageBundle), and the tokenizer. ~2.2 GB.
- `config.json` — the source model config (for reference).

Conversion code and the full engineering notes live in the
[model zoo](https://github.com/john-rocky/coreai-model-zoo) (`conversion/export_youtu_decode_pipelined.py`,
`models/macos/youtu.py` + `youtu_absorbed.py`; the absorbed-MLA cache + flash-decode kernel are
shared with GLM-4.7-Flash).

## License

Weights follow Tencent's **`youtu-llm`** license (see `license_link`) — commercial use and
redistribution of derivatives permitted with attribution; **⚠️ the license states Youtu-LLM is
NOT intended for use within the European Union.** The Core AI conversion adds no restrictions.