mlx-community/d1a-e4b-it-qat-4bit

D1A-E4B-it QAT · MLX 4-bit

This repository mirrors JohnP1/d1a-e4b-it-qat-mlx-q4 (revision 508ba4af) byte for byte (the same files, checked by LFS sha256); only this card differs.

What it does: decisions (/v1/systemone), chat (OpenAI-format /v1/chat/completions, streaming included), and photos, voice clips and video (/v1/systemone/media). One model on Apple Silicon (MLX), 4.4 GB plus 0.9 GB of encoders (D1A-E4B-it v0.6 in 8-bit: 6.1 GB plus 0.9 GB).

Scorecard (2026-10-28): level with D1A-E4B-it v0.6 overall, stronger on PR labels and routing, weaker on one external suite (below). Released 2026-10-27 by the owner's decision, before its gates finished; both gates were set before training and have since run on every D1A suite, the demos and the PR labeler's data.

A small decision model for Apple Silicon: give it a document and a few questions, and it picks the answers. Each question is yes/no, multiple choice or a score; D1A returns a calibrated probability for every option in one forward pass, with no generated text, so every answer is a valid option.

This build sits exactly on Google's quantization-aware-trained int4 weights, google/gemma-4-E4B-it-qat-w4a16-ct (revision 6cd26aaa). All 343 of its int4 linear layers are stored here bit for bit as MLX 4-bit: the same packed words and scales, dequantization error 0.0. Nothing is re-quantized from bf16. The D1A adapter was retrained on this exact trunk, so the 4-bit model is the model that was trained, not a rounded copy of it.

Why it was retrained:

• The QAT weights differ from the bf16 ones the v0.6 adapter was trained on. On E2B, the v0.6 adapter moved onto its QAT trunk unchanged lost accuracy and changed 9.6% of its clear-cut answers (D1A-E2B-it QAT).
• D1A-E4B-it v0.6 was trained with PyTorch on an Apple GPU, where a PyTorch bug read Gemma 4's per-layer embeddings as zeros for every token id from 199,729: every punctuation and digit token, 28.5% of its training tokens (jonpol01/d1a#353, pytorch/pytorch#188756). This retrain reads them correctly (d1a with jonpol01/d1a#354).

Scorecard

Against D1A-E4B-it v0.6 (MLX 8-bit): level overall, pooled −0.16 points [−0.88, +0.51] over 21 lines, with no veto from D1A's release rules. Against v0.6's adapter moved unchanged onto this same QAT trunk: +1.19 [+0.50, +1.87], 6 significant wins and no significant loss, so the retrain is what makes the 4-bit trunk work.

• Stronger than v0.6: PR labels on the test set +1.6, PR severity with the per-class offsets +6.0, PR blast radius +7.2, routing +1.7 and +1.5.
• Weaker than v0.6: the external typesafe suite, −7.8 (jonpol01/d1a#361). It comes from the 4-bit trunk, not the retrain: v0.6's own adapter on this trunk loses nearly as much (−6.9). Calibration is looser on transfer-v4 (ECE 0.081 → 0.105). With the severity offsets on, it catches fewer P1 pull requests (29 of 47 against 39 of 47) at a higher precision (0.30 against 0.22).
• Speed: about the same per decision as v0.6 in 8-bit (ratio 1.01, inside D1A's latency limit), at 5.3 GB instead of 7.0 GB.
• Photos, voice and video: of the 28 answers to the 14 demo requests, 26 are the same as v0.6's. The two that differ: an intact parcel in a mailbox read as "in a delivery locker", and a voice clip about a damaged parcel marked urgent.

Every suite, accuracy in %, Δ in points with paired 95% bootstrap intervals (bold where the interval excludes 0); measured with scripts/quality_gate.py --all-suites and scripts/decide.py on an Apple M1 Max. ECE and the method: the full scorecard.

Suite	n	This build	Δ vs D1A-E4B-it v0.6	Δ vs v0.6's adapter on this trunk
devtools-v1 (card)	1,074	74.6	+0.2 [-1.1, +1.5]	+0.3 [-1.0, +1.6]
documents-v1 (card)	920	88.9	+0.3 [-1.1, +1.7]	+1.1 [-0.2, +2.4]
external semif-v1	144	88.9	-2.1 [-5.6, +0.7]	-1.4 [-4.9, +2.1]
external typesafe-v1	102	79.4	-7.8 [-11.7, -3.6]	-1.0 [-5.8, +4.3]
external wanli-v1	256	72.3	-1.6 [-3.9, +0.8]	+0.4 [-2.3, +3.1]
external wanli-v2	1,002	72.1	-0.5 [-1.9, +0.9]	+1.6 [+0.1, +3.1]
hard-v1 (card)	1,083	76.8	+1.2 [-0.4, +2.9]	+3.4 [+1.7, +5.4]
ja-jglue development	1,500	80.7	-0.1 [-1.3, +1.1]	+1.0 [-0.2, +2.3]
ja-jglue test	1,500	79.7	+1.2 [+0.0, +2.4]	+1.7 [+0.4, +3.0]
night2 assertion	1,020	94.1	+0.9 [-0.3, +2.0]	+1.7 [+0.7, +2.7]
night2 dates	900	90.0	-1.2 [-2.7, +0.2]	+0.9 [-0.2, +2.0]
night2 unknowable	525	67.2	+0.6 [-0.8, +1.9]	+0.6 [-1.0, +2.3]
pr-labels development	2,030	80.5	+0.9 [-0.1, +1.9]	+1.7 [+0.4, +3.0]
pr-labels development-ja	198	78.8	+3.5 [-0.5, +7.2]	-0.5 [-5.0, +4.1]
pr-labels real17	51	64.7	-3.9 [-11.8, +3.9]	+0.0 [-7.8, +7.8]
pr-labels test	2,045	81.0	+1.6 [+0.5, +2.7]	+4.4 [+3.0, +5.9]
pr-labels test-ja	193	76.2	-0.5 [-2.7, +1.6]	+3.1 [-1.0, +7.4]
routing factory-development	640	95.2	+1.7 [+0.3, +3.0]	+0.6 [-0.9, +2.2]
routing generic-development	270	98.9	+1.5 [+0.4, +3.0]	+1.1 [+0.0, +2.6]
routing handlabelled-45	45	100.0	+0.0 [+0.0, +0.0]	+0.0 [+0.0, +0.0]
transfer-v4 (card)	656	76.1	-1.2 [-3.4, +0.8]	+0.0 [-1.8, +1.7]
decision-v7 (card)	1,264	86.0	+0.2 [-0.8, +1.3]	+0.7 [-0.2, +1.6]
PR severity, with offsets (test)	953	75.3	+6.0 [+3.8, +8.3]	+8.6 [+6.1, +11.0]
PR blast radius (test)	139	64.0	+7.2 [+1.4, +13.7]	+7.9 [+0.7, +14.4]
PR labeler replay (recent PRs)	193	72.0	+1.0 [-4.2, +6.2]	+2.1 [-3.1, +7.2]
owner's PR labels	109	71.6	+0.9 [-4.4, +6.5]	+4.6 [+0.0, +9.3]

What's inside

• The trunk: Gemma 4 E4B-it QAT, its 343 int4 linear layers exactly as Google trained them (symmetric, groups of 32); token and per-layer embeddings 4-bit in groups of 64 (QAT fixes only the Linears' groups of 32). 4.4 GB.
• The D1A adapter, unmerged (adapter/, LoRA rank 16, 133 MB). With the adapter off the model is Gemma 4 E4B-it QAT itself, so the same weights also chat.
• The pointer head (head.safetensors).
• Calibration temperature 1.66 in d1a_config.json, fitted on this build's own answers: 3,178 rows from the decision-v7 calibration split and the PR-label development set, the same recipe as v0.6. Expected calibration error 0.069 → 0.016 out of fold; 90% interval of the temperature 1.59–1.74. Both sets come from corpora the model trained on, which the fit records.
• media/: Gemma 4's vision and audio encoders from the QAT checkpoint itself (bf16, 0.9 GB).

Training

D1A-E4B-it v0.6's recipe, continued on the QAT trunk (jonpol01/d1a#341):

• warm start from JohnP1/d1a-e4b-it v0.6;
• one epoch of v0.6's all-skills mix: 23,427 records from 11 sources (PR labels in English and Japanese, blast radius, recent PRs, hard, devtools, documents, JGLUE, routing, decision-v7);
• the trunk frozen at the QAT checkpoint's values, dequantized exactly (scale × integer in fp32), the embedding tables in bf16 as the checkpoint stores them (--embedding_dtype bf16, jonpol01/d1a#355); learning rate 2e-5, batch 8;
• 2,929 steps, 36.2 hours on an Apple M1 Max (MPS).

Other checks

• Exactness: 343 of 343 quantized linear layers identical to the QAT checkpoint (words, scales and biases).
• Serving (d1a.serving.serve, Apple M1 Max): the 145 playground demo requests (the 14 photo, voice and video requests included) and 92 PR-labeler states all answered; the example request below answers billing (0.40, shipping 0.37); with the adapter off, chat answers.

Run it

Needs d1a with jonpol01/d1a#261:

pip install "d1a[serve,mlx,media] @ git+https://github.com/jonpol01/d1a"
python -m d1a.serving.serve --run JohnP1/d1a-e4b-it-qat-mlx-q4 --port 8009

A decision:

curl -s localhost:8009/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
  "questions": {"team": {"type": "choice", "instructions": "Which team should handle this?",
                         "criteria": {"returns": null, "shipping": null, "billing": null}}}
}'

A photo, voice clip or video: the same request with a media field. type is image (JPEG, PNG, WebP), audio (WAV, FLAC or OGG, up to 30 s) or video (MP4, MOV or WebM, up to 32 MB; 16 frames sampled evenly, the sound track not used):

curl -s localhost:8009/v1/systemone/media -H 'content-type: application/json' -d '{
  "media": {"type": "image", "data": "'"$(base64 -i parcel.jpg)"'"},
  "questions": {"damaged": {"type": "noul", "instructions": "Is the parcel damaged?"}}
}'

Chat (the adapter is switched off for it, so this is Gemma 4 E4B-it QAT itself):

curl -s localhost:8009/v1/chat/completions -H 'content-type: application/json' -d '{
  "messages": [{"role": "user", "content": "Name three things to check when a parcel arrives damaged."}]
}'

or in-process: from d1a import D1A; D1A.load("JohnP1/d1a-e4b-it-qat-mlx-q4").decide(state, questions).

Limits

• The external typesafe suite: −7.8 points against D1A-E4B-it v0.6 (above, jonpol01/d1a#361). Pin JohnP1/d1a-e4b-it-mlx-q8 v0.6 where answers must match v0.6's.
• Photos, voice and video are measured on the 14 demo requests only, zero-shot as on every D1A-E4B-it build.
• <bos>: the model was trained with id 2 at the start of every record; d1a adds it (leading_ids: [2]). An app that encodes requests itself must prepend it.
• Questions are in English. The documents can be English or Japanese.
• Everything on the D1A-E4B-it card applies: experimental, and agent tool calls go through D1A's fail-safe thresholds (d1a.agents.presets.advise).

License and data

Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: github.com/jonpol01/d1a, built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.