APPLY.md

# LFM2 / LFM2.5-MoE export defs for coreai-models (North Star, 2026-07-14)

The missing LiquidAI model definitions for Apple's CoreAI exporter, written by us and
**verified bit-exact vs the HF reference** (`verify_lfm2_def.py`: prefill Δ=2.6e-5,
decode-with-state Δ=1.0e-5 on logit scale ~30, fp32, 4-layer truncation covering conv +
attention + MLP + tied embeddings + state continuation).

## What's in here
- `lfm2.py` → `python/src/coreai_models/models/macos/lfm2.py` (NEW) — dense LFM2/2.5
  hybrid: gated short-conv as L explicit taps (no aten.conv1d — converter-safe), conv
  state as a mutable state tensor (`conv_state`, zeros = sequence start so one graph
  serves prefill+decode), GQA attention with fused qkv + fused per-head qk RMSNorm,
  KV cache sized to attention layers only (6 not 16).
- `lfm2_moe.py` → `python/src/coreai_models/models/macos/lfm2_moe.py` (NEW) — the 8B-A1B:
  same hybrid + 32-expert top-4 SwiGLU via **SwitchGLU/GatherMM (the routed kernel —
  only the 4 routed experts' slabs are read per token)**. Router matches the MLX-Swift
  reference: softmax(fp32) → +expert_bias → top-4 → renorm. First `num_dense_layers`
  are dense MLP. Expert stacking in `_mutate_state_dict` (experts.{e}.w1/w3/w2 →
  switch_mlp.gate/up/down (1,E,out,in)).
- `registry.py` → `python/src/coreai_models/models/registry.py` — adds "lfm2" + "lfm2_moe".
- `macos.py` → `python/src/coreai_models/export/macos.py` — export hook: a model class may
  provide `build_reference_inputs(...)` + `state_names()` to declare extra mutable state
  (the conv state) beyond k_cache/v_cache.
- `verify_lfm2_def.py` → repo root — the numerical vet harness. Run it after ANY change.

## Apply to a fresh clone of john-rocky/coreai-models
1. `git clone --depth 1 https://github.com/john-rocky/coreai-models && cd coreai-models`
2. Bump pins in `python/pyproject.toml`: `coreai-core==1.0.0b2`, `coreai-torch==0.4.1`
   (b1 artifacts are rejected by the macOS 27 beta3+ strict loader — FB23666783), then
   `uv lock --upgrade-package coreai-core --upgrade-package coreai-torch`.
3. Copy the four .py files to the paths above (+ verify_lfm2_def.py to the root).
4. Vet: `uv run python verify_lfm2_def.py` → must print MATCH / MATCH.
5. Export 1.2B: `uv run coreai.llm.export LiquidAI/LFM2.5-1.2B-Instruct --experimental
   --compute-precision float16 --max-context-length 8192 --output-dir ~/models/lfm25-1.2b-coreai-b2`
   ⚠️ `--max-context-length 8192` is MANDATORY on an 8GB box. Without it the export bakes in the
   model's full 128000 context; the CoreAI engine's load/specialization then spikes RAM and CRASHED
   the 8GB Mac (2026-07-14). Reference bundles all use 8192. (lesson #42)
6. Bench: `Tools/CoreAIBench/.build/release/CoreAIBench <bundle-dir> 600`
   (build CoreAIBench with `DEVELOPER_DIR=/Library/Developer/CommandLineTools swift build
   -c release` + coreai-kit's Package.resolved copied in).

## 8B-A1B (lfm2_moe) status
Def is written and registered; NOT yet exported: (a) torch.export of 8.3B fp16 needs
~17GB+ RAM — impossible on the 8GB box, run it on a ≥32GB machine or HF Jobs;
(b) `AutoConfig` for model_type=lfm2_moe needs transformers ≥5.x (venv has 4.57 — bump
only for the 8B export, or construct the config by hand from config.json).