README.md

metadata

base_model: meta-llama/Llama-3.2-1B-Instruct
license: llama3.2
tags:
  - coreml
  - llama
  - kv-cache
  - stateful

Llama 3.2 1B — Stateful KV Cache Core ML (unified model)

Single unified Core ML bundle: input_ids, causal_mask, state keyCache / valueCache.

Structure

ctx{N}_fp16/Llama32_1B_KVCache/   — FP16 unified model
ctx{N}_int4/Llama32_1B_KVCache/   — INT4 unified model (optional)

Usage

• OS: macOS 15 / iOS 18+
• Compute units: CPU_AND_GPU
• Inference: make_state(); prefill + decode with the same model; see notebook Step 9 for causal_mask shapes.

Memory estimate (FP16 KV): 0.00 GB for ctx=4096.