README.md

---
license: gemma
base_model: google/gemma-3-4b-it
tags:
- coreai
- apple
- aimodel
- apple-silicon
---

# Gemma 3 4B IT — official Apple Core AI export

Pre-converted **`.aimodel` bundles from Apple's official
[coreai-models](https://github.com/apple/coreai-models) export recipe — unmodified**,
with the exact environment, hashes, and measured performance published.

```bash
uv run coreai.llm.export gemma3-4b-it          # bfloat16 compute per registry preset
```

<!-- gen-cards:use-it begin id=gemma-3-4b-it (managed by scripts/gen-cards — edit cards.json / QuickStart.swift, not this block) -->
## Use it

▶️ **Run it (source)** — the [ChatDemo runner](https://github.com/john-rocky/coreai-kit/tree/main/Examples/ChatDemo)
(GUI + CLI, one app for every chat model in the catalog):

```bash
git clone https://github.com/john-rocky/coreai-kit
open coreai-kit/Examples/ChatDemo/ChatDemo.xcodeproj
# → Run, then pick "Gemma 3 4B" in the model picker

# agents / headless (macOS):
cd coreai-kit/Examples/ChatDemo
swift run chat-cli --model gemma-3-4b-it --prompt "What can you do, offline?"
```

💻 **Build with it** — complete; the glue is kit API, copy-paste runs:

```swift
import CoreAIKit

let chat = try await ChatSession(catalog: "gemma-3-4b-it")
let reply = try await chat.respond(to: prompt)
// reply: the answer, generated fully on-device
```

The take-home is [`Examples/ChatDemo/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/main/Examples/ChatDemo/Sources/QuickStart.swift)
— this exact code as one typed function, no UI; the CLI is an argument shell over it, and
the GUI drives the same `ChatSession` across turns for its transcript.
Multi-turn? Hold the `ChatSession` and call `respond(to:)` per turn — it keeps the
conversation history; `streamResponse(to:)` yields tokens as they decode.

**Integration checklist**

- SPM: `https://github.com/john-rocky/coreai-kit` → product **CoreAIKit**
- Info.plist: none needed
- Entitlements: none needed (macOS)
- First run downloads the model — 2.2 GB (Mac) — then it loads from the
  local cache (Application Support; progress via the `downloadProgress` callback)
- Measure in Release — Debug is ~3× slower on per-token host work
<!-- gen-cards:use-it end -->

## Why pre-converted bundles?

1. **The conversion needs a big-RAM Mac** (the 20B export was done on 128 GB);
   running only needs enough RAM to mmap the artifact.
2. **An `.aimodel` is a build artifact, not a pure function of the recipe** — the
   same export command produced a 2.2× slower artifact across the macOS 26 → 27β
   boundary ([forensics](https://github.com/john-rocky/apple-silicon-llm-bench/blob/main/methodology/coreai-export-lowering.md)).
   Hosted artifacts + hashes are the reproducible ground truth; every bundle here
   is exactly the one measured in
   [apple-silicon-llm-bench](https://github.com/john-rocky/apple-silicon-llm-bench).

## Bundles & integrity

| Bundle | Contents | SHA-256 (`main.mlirb`) |
|---|---|---|
| `macos/` | macOS dynamic, int4 (bf16 compute) | `68b103aa2994ed50b9bb14f32bd9f746afc7eebae4b65d28c913a72dbd5fce91` |

## Measured (Apple's official `llm-benchmark`, greedy)

| Bundle | Protocol | Decode tok/s | Prefill | Load (warm) | Peak RSS |
|---|---|---:|---:|---:|---:|
| macos | M4 Max, 512p/1024g | 141.5 | 1,669 | 0.32 s | 4.5 GB |

## Export environment

- macOS 27.0 beta (build 26A5353q) · Xcode 27.0 (27A5194q)
- `coreai-core 1.0.0b1` · `coreai-torch 0.4.0` · `coreai-opt 0.2.0` · `torch 2.9.0`
- apple/coreai-models @ `b1cb71b` (export code identical to upstream `0c1055f`)

## Run it

```bash
# CLI (from a coreai-models checkout)
swift run -c release llm-runner --model <downloaded-bundle-dir> --prompt "Hello"
swift run -c release llm-benchmark --model <downloaded-bundle-dir>
```

Or chat with it in [CoreAIChatMac](https://github.com/john-rocky/coreai-samples)
(point "Choose Models Folder…" at the download directory).

iOS static bundles must be AOT-compiled before device use:
`xcrun coreai-build compile <ir>.aimodel --platform iOS --preferred-compute neural-engine --architecture h18p`
(h18p = iPhone 17 Pro), then set `metadata.json` `assets.main` to the `.aimodelc`.

Use of this model is subject to the [Gemma Terms of Use](https://ai.google.dev/gemma/terms). This repository redistributes a converted derivative of google/gemma-3-4b-it under those terms; by downloading you agree to them.

---
Maintained alongside [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo)
(community models) and [coreai-samples](https://github.com/john-rocky/coreai-samples) (apps).