AileyNitro-2B (r2, September 2026)

AileyNitro-2B is the fast, always-resident companion model of A!ley (OpenM!nded / Simon van de Loo): a fine-tune of Google Gemma 4 E2B on the QAT-4bit checkpoint, running locally on Apple Silicon via MLX. It handles titles, suggestions, search terms, arithmetic expressions and stands in for the 12B model while that one is busy — and it holds the A!ley persona on its own.

• Developed by: OpenM!nded (Simon van de Loo)
• Base model: mlx-community/gemma-4-E2B-it-qat-4bit
• Model type: Multimodal (text + image input, text output), decoder-only
• License: Apache License 2.0
• Languages: German, English
• Quantization: 4-bit (QAT), unchanged; adapter merged without loss (see below)

What is new in r2

r1 was a fine-tune on the plain 4-bit E2B whose identity collapsed without a system prompt (2/7 answers "I am Gemma, made by Google DeepMind"). r2 starts from Google's QAT checkpoint (validation loss 2.92 vs 3.84 for the plain 4-bit on the same data), learns Gemma 4's native reasoning channel (<|channel>thought … <channel|>) and keeps identity with nothing but "Du bist Ailey." (7/7).

QAT adapters — why the merge is lossless

Merging a low-learning-rate adapter into 4-bit weights normally rounds most of the delta away (we measured 87 % loss on the 12B at 6-bit). Here the adapters were trained as QAT adapters: the forward pass sees fake_quant(W_dequant + ΔW) with a straight-through gradient, so the model only learns deltas that survive the rounding — and fuse() yields bit-identical weights to what was trained. Measured: live adapter val 1.774 (n=60), merged val 1.730 (n=120), base on the same 120 samples 2.973.

Setting	Value
Method	DoRA (q_proj, v_proj) + LoRA (gate_proj, up_proj, down_proj), QAT
Rank / Alpha	32 / 64
Data	6 027 curated samples (595 validation): conversation, tool use, RAG/web context, identity — native channel format, no idle-loop residue, deduplicated
Epochs	1 of 2 (epoch 2 only memorised — val 1.801/1.798 vs 1.774 — and was stopped)
lr / seq / grad accum	2e-4 / 1024 / 8, gradient checkpointing
Selected checkpoint	epoch_1 (best)
Not shipped	an ORPO stage (195 identity pairs, margin +6.8): val rose to 2.001 and the reasoning channel vanished (12/14 → 1/14)
Hardware	Apple M4, 24 GB, ~62 s per update

How to use (MLX)

Gemma 4 is a unified multimodal architecture → load with mlx_vlm.

from mlx_vlm import load, generate
model, processor = load("OpenMinded-Labs/AileyNitro-2B")
prompt = "<bos><|turn>user\nWer bist du?<turn|>\n<|turn>model\n"
print(generate(model, processor, prompt, max_tokens=300, temperature=0.7, top_p=0.8,
               repetition_penalty=1.05, verbose=False).text)

The model answers in the native channel format:

<|channel>thought
…brief reasoning…
<channel|>…answer…<turn|>

Two things a 2B does badly, and what we do about them in A!ley's runtime: mental arithmetic (it calls a calculate tool — offer one, or precompute) and world knowledge (it confabulates brand names and mechanisms; keep it to the jobs above). If you do not offer tools, ban the <|tool_call> token or strip the call — otherwise an arithmetic question may return only a tool call.

Limitations & biases

Inherited from Gemma 4 plus the fine-tune: factually unreliable on knowledge questions, reflects its training data, is a persona and not a knowledge base. Keep a human in the loop for consequential use.

License & attribution

Derivative Work of Google Gemma 4 (Apache License 2.0). Distributed under Apache 2.0; modifications are documented in AILEY_MERGE_INFO.json; LICENSE and NOTICE included. Gemma is a trademark of Google LLC; this project is independent and not endorsed by Google.

Copyright 2026 OpenM!nded / Simon van de Loo
Portions © Google LLC (Gemma 4), Apache License 2.0