Permea2.0-1.7B — Ternary Qwen3, Healed (Doubt-Cut Distillation)

A ~0.5 GB ternary Qwen3-1.7B whose reasoning quality was restored by QAT healing — the flagship result of a systematic 16-arm research campaign. Weights are 2-bit ternary (grid identical to prism-ml/Ternary-Bonsai-1.7B); the healing moved ~0.1–0.2 % of grid codes via straight-through-estimator training around the frozen ternary grid.

Variants

File	Focus	Size
Permea2.0-1.7B-P.Q2_0.gguf	Benchmark champion — best external scores	0.49 GB
Permea2.0-1.7B-N.Q2_0.gguf	Balanced — strongest instruction-holdout	0.49 GB

The core finding: doubt loops, and how to cut them

Naive ternary quantization does not destroy knowledge — it destroys commitment. The quantized model reaches correct answers in its reasoning trace, then cycles "But wait, let me check again…" until any token budget is exhausted. We showed this failure mode is not fixable by larger budgets or anti-repetition samplers — only by training: the student mines its own truncated loop traces on training data, each trace is cut at the first provably-correct commitment point, and the repaired traces are distilled back. ~200 such repair pairs recover most of the gap to the bf16 base.

Benchmarks (zero-shot, identical harness/items/budget for all rows)

Measured with our open evaluation harness: 550 items (GSM8K 200 / MMLU 200 / ARC-Challenge 150, official test splits), generation budget 3072 tokens, greedy, answer extraction with boxed/statement fallbacks. Numbers are paired — every model saw the identical items through the identical grader.

Model	Size	GSM8K	MMLU	ARC-C	Overall
Permea2.0-1.7B-P (this)	0.49 GB	71.5	46.5	69.3	61.8
Qwen3-0.6B (Q4_K_M)	0.40 GB	61.0	48.5	69.3	58.7
SmolLM2-360M (Q8)	0.39 GB	4.5	31.0	22.7	19.1
Ternary-Bonsai-1.7B (unhealed)	0.5 GB	68.5	46.5	64.0	59.3
Qwen3-1.7B bf16 (teacher/base)	3.4 GB	81.0	61.5	88.0	75.8
Qwen3-0.6B (Q8)	0.64 GB	69.5	48.0	68.7	61.5
Llama-3.2-1B-Instruct (Q8)	1.31 GB	33.5	29.5	35.3	32.5
Gemma-3-1B-it (Q8)	1.07 GB	3.0	33.5	45.3	25.6

Reasoning-capable models (ours, Qwen3-0.6B) were measured in thinking mode; Llama-3.2-1B and Gemma-3-1B are non-reasoning models and were measured with their standard chat templates — same items, budget and grader for every row. Competitor GGUFs are pinned to fixed revisions.

Notes: published third-party numbers are not directly comparable (few-shot prompts, different harnesses). The honest comparison is the paired delta inside this table. Our harness reproduces published Qwen3-1.7B anchors within ~1–5 points (GSM8K 81.0 vs 75.4 published, MMLU 61.5 vs 62.6).

Throughput (single stream, RTX 5090, 384 fixed tokens, greedy)

Model	tok/s
This model (0.49 GB)	767
Qwen3-0.6B (Q4)	788
Qwen3-1.7B bf16 (base)	363

2.1× faster than the bf16 base at (measured-)identical healed quality.

Limitations

• Thinking mode strongly recommended. The no-think fast path shows larger quality loss (inherited from ternarization).

• A knowledge gap vs the bf16 base remains on MMLU/ARC (~10–20 points) — this is ternarization damage, not a reasoning defect.

• German instruction-following was trained/evaluated on a proprietary internal suite; the public benchmarks above are English.

• BitNet-b1.58-2B could not be measured in the identical harness (its i2_s GGUF requires the BitNet fork); we do not quote third-party numbers in this table.

• Speculative decoding with this model as a draft for the bf16 base is blocked by the trimmed vocabulary (151669 vs 151936) in stock llama.cpp — vocabulary-padded export is future work.

• The published GGUFs quantize the (frozen, never-trained) embedding and output tensors on the identical ternary grid as the upstream pack; behavioral identity to the full-precision-embedding variant was verified on held-out (identical scores, same environment).

Lineage & method

Qwen3-1.7B (Apache 2.0) → ternary packing by prism-ml (Apache 2.0) → QAT healing + doubt-cut distillation (this work). Training data: ~140 verified teacher traces + ~200 self-repair pairs mined from official train splits (GSM8K/ARC/OpenBookQA train — zero eval-set contact, text-overlap verified). Full research log available on request.