| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-Coder-0.5B-Instruct |
| base_model_relation: quantized |
| pipeline_tag: text-generation |
| library_name: coreai |
| widget: |
| - text: "Explain on-device AI in one sentence." |
| - text: "Write a haiku about Apple Silicon." |
| tags: |
| - coreai |
| - core-ai |
| - coreai-fabric |
| - aimodel |
| - coreml |
| - apple |
| - apple-silicon |
| - on-device |
| - iphone |
| - metal |
| - text-generation |
| - llm |
| - int8 |
| --- |
| |
| > **Canonical:** [`kevinqz/Qwen2.5-Coder-0.5B-Instruct-CoreAI`](https://huggingface.co/kevinqz/Qwen2.5-Coder-0.5B-Instruct-CoreAI) — source of truth. |
|
|
| # Qwen2.5-Coder 0.5B Instruct (fabric, int8) |
|
|
| **Apple Core AI chat model — runs fully on-device on Apple Silicon |
| (iPhone / iPad / Mac, macOS/iOS 27+).** |
|
|
| A quantized **stateful KV-cache chat** `.aimodel` — an Apple Core AI conversion of |
| [Qwen/Qwen2.5-Coder-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct), with an embedded |
| tokenizer + chat template. Produced by |
| [coreai-fabric](https://github.com/kevinqz/coreai-fabric/blob/main/recipes/qwen2.5-coder-0.5b-instruct-int8.yaml) and indexed by |
| [coreai-catalog](https://github.com/kevinqz/coreai-catalog). |
|
|
| ## Model facts |
|
|
| | Field | Value | |
| |---|---| |
| | Parameters | — | |
| | Architecture | transformer | |
| | Capabilities | code-generation, text-generation | |
| | Quantization / precision | int8 / float16 | |
| | Context length | — | |
| | On-disk size | 501 MB | |
| | Asset kind | stateful KV-cache chat bundle; embedded tokenizer + chat template | |
| | assetVersion | 2.0 | |
|
|
|
|
|
|
| ## Use it |
|
|
| Install via the catalog, then run it with Apple's Foundation Models runtime: |
|
|
| ```bash |
| pip install coreai-catalog && coreai-catalog install qwen2.5-coder-0.5b-instruct-int8 |
| ``` |
|
|
| ```swift |
| import CoreAILanguageModels |
| import FoundationModels |
| |
| // modelURL = the installed macos/ bundle directory for this model |
| let model = try await CoreAILanguageModel(resourcesAt: modelURL) |
| let session = LanguageModelSession(model: model) |
| let reply = try await session.respond(to: "Explain on-device AI in one sentence.") |
| print(reply) |
| ``` |
|
|
| A complete, buildable example lives at |
| [coreai-catalog/examples/llm-chat](https://github.com/kevinqz/coreai-catalog/tree/main/examples/llm-chat). |
|
|
| ## Requirements |
|
|
| - **Deployment: macOS 27.0+ / iOS 27.0+, Xcode 27+.** The asset serializes with |
| `minimum_os v27`, so the on-device Swift runtime requires macOS/iOS 27+. |
| - A Mac on **macOS 26 can convert and inspect** the asset but **cannot run** it |
| on-device (the Swift runtime needs the 27 SDK). |
| - Apple Silicon. |
|
|
| ## Intended use & limitations |
|
|
| - **Intended use:** general on-device chat / text generation. Inherits the base |
| model's capabilities, languages, and biases. |
| - **Limitations:** int8 quantized — the high-fidelity tier, near-lossless vs. fp16. See the Evaluation section for the measured |
| greedy fidelity vs the fp16 reference. |
|
|
| ## Evaluation (parity) |
|
|
| - **Gate A (structure): passed** — the bundle's layout + metadata were |
| validated on real hardware (Apple Silicon); the asset loads and generates. |
| - **Gate B (numeric accuracy): not_run.** Task-accuracy evaluation (e.g. tinyMMLU) is pending *upstream*: Apple's `coreai.llm.eval` is a stub in coreai-models 0.1.0 that cannot score a stateful KV-cache asset. Greedy fidelity vs fp32 can be measured on-device via the parity runner. fabric never fakes a parity number. |
| - **Runtime throughput (tok/s):** to be published once measured on the on-device (macOS/iOS 27) Swift runtime via `coreai-fabric bench`. Not estimated — real numbers or none. |
| |
| ## Provenance |
| |
| | Field | Value | |
| |---|---| |
| | Base model | [Qwen/Qwen2.5-Coder-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct) @ `ea3f2471cf1b1f0db85067f1ef93848e38e88c25` | |
| | Converted by | `coreai.llm.export` (version not reported) | |
| | Recipe | [qwen2.5-coder-0.5b-instruct-int8](https://github.com/kevinqz/coreai-fabric/blob/main/recipes/qwen2.5-coder-0.5b-instruct-int8.yaml) (recipe_source: fabric) | |
| | Precision / quantization | float16 / int8 | |
| | Conversion date | 2026-07-10 | |
| |
| Machine-readable, in this repo: |
| [`parity-report.json`](./parity-report.json) (gate results) · |
| [`reproduce-manifest.json`](./reproduce-manifest.json) (exact tool + stack + pinned |
| revision to reproduce this conversion) · [`LICENSE`](./LICENSE) (upstream terms). |
| |
| ## License and attribution |
| |
| Weights © 2024 Alibaba Cloud, licensed **apache-2.0** — see the bundled `LICENSE`. This artifact is a **converted + quantized derivative** of the base |
| model (the Apache-2.0 §4(b) change notice): weights were converted to Apple Core |
| AI format and quantized to int8. The conversion itself is community work. |
|
|
| ## Links |
|
|
| - **Base model:** [Qwen/Qwen2.5-Coder-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct) |
| - **Reproduce:** [recipe `qwen2.5-coder-0.5b-instruct-int8`](https://github.com/kevinqz/coreai-fabric/blob/main/recipes/qwen2.5-coder-0.5b-instruct-int8.yaml) · [runnable example](https://github.com/kevinqz/coreai-catalog/tree/main/examples/llm-chat) |
| - **Index:** [coreai-catalog](https://github.com/kevinqz/coreai-catalog) — the neutral registry that ties upstream ↔ this asset ↔ mirror together |
| - [HF Collection](https://huggingface.co/collections/kevinqz/coreai-apple-on-device-6a4879f21c7e1a87c99bcf5a) |
|
|
|
|
| ## The on-device Core AI ecosystem |
|
|
| This conversion is part of a broader open ecosystem for running models on Apple's |
| on-device stack — useful references if you're building here: |
|
|
| - [coreai-fabric](https://github.com/kevinqz/coreai-fabric) — the reproducible |
| recipe → `.aimodel` pipeline that produced this asset. |
| - [coreai-catalog](https://github.com/kevinqz/coreai-catalog) — the index of Core |
| AI models across the community, with provenance and integration snippets. |
| - [apple/coreai-models](https://github.com/apple/coreai-models) — Apple's official |
| exporters and runtimes. |
| - [CoreAI Model Zoo](https://github.com/john-rocky/coreai-model-zoo) and the wider |
| [coreai-community](https://huggingface.co/coreai-community) — community |
| conversions across many model families. |
|
|
| ## Not affiliated with Apple |
|
|
| Community conversion. Not produced, hosted, or endorsed by Apple. Apple and |
| Core AI are trademarks of Apple Inc., used here only to describe the target |
| runtime/format. This is an independent community conversion. |
|
|
