paper/ICE_Technical_Report.md

ICE: Quantization by Error-Propagation Class in Sparse Mixture-of-Experts Models

Gökhan Buz

Independent researcher, gbuzhf on Hugging Face

Version 1.0, August 2026

ICE stands for Isolation of Compounding Error: the method finds the tensors whose quantization error compounds beyond the token that produced it, and isolates them out of the bit budget entirely.

---

Abstract

Every quantizer in the GGUF ecosystem minimizes the same objective for every tensor: the importance-matrix-weighted mean squared error of that tensor's output, for the current token. That objective is correct for tensors whose error dies with the token that produced it. It is the wrong objective for three other kinds of tensor, and no published method separates them.

This report describes ICE, which classifies every tensor by how far its quantization error travels rather than by how large its activations are, and allocates bits accordingly. Errors that flip a discrete routing argmax, that enter a recurrent state decay, or that are written into the KV cache do not decay with the token: they are re-read or compounded by every later token, so their cost grows with context length. ICE holds that set exact and spends everything else on the expert bank.

The set is small enough that this is a line item rather than a trade-off. On the model studied here the entire propagating set is 47 M parameters, 0.14% of the checkpoint. Roughly half of it is already held exact by upstream llama.cpp; the part ICE actually pins is 27.0 M parameters, 0.078% of the model, for about 0.03 GB.

The method is evaluated against two published mixed-precision ladders, Unsloth Dynamic 2.0 and LocalAI APEX, on a common harness: mean KL divergence against the bf16 checkpoint on WikiText-2. Across a twelve-tier comparison, nine tiers are Pareto-optimal and three are strictly dominated. At matched size the 23 GB ICE tier is 5.0% closer to bf16 than the nearest UD tier and 13.0% closer than the nearest APEX tier, while being smaller than both.

Four quantitative laws came out of the measurements, and they matter more than the ladder:

• Law 1, sparsity. In an MoE with E experts and k active, a bit spent on the always-on core is worth E/k bits spent on the expert bank. Measured: 270 against 8.5 units of work per GB-per-bpw, a ratio of 31.8 against a predicted 32.
• Law 2, convexity. Quantization error falls as 4^-b. By Jensen, cleverness in allocation is capped: water-filling over the real routing histogram beats uniform by only +0.139 bpw, so ICE does no depth grading at all. The same convexity says that at a fixed average, the narrower type bracket always wins, measured at +16.6% and +7.0% for widening.
• Law 3, placement. Within a fixed bracket, assigning the high type shallow-first is worth about −8.5% KLD per bpw of gap (r = −0.894) and reverses below a gap of 0.47 bpw. The value of a bit falls with depth as g(t) ≈ exp(−t/9.95), so a block-0 bit is worth about 50 times a block-39 bit.
• Law 4, the floor is epistemic. Fitting KLD ≈ k₀ + c_d·4^(−b_dense) + c_e·4^(−b_expert) on two independent harnesses gives k₀ = 0.0205 and 0.0219. The best tier measured sits 8% above that floor. Above roughly 5.5 expert bpw you are buying calibration noise. You cannot out-bit a wrong prior.

The negative results are reported at the same weight as the positive ones, because they are the expensive part to rediscover. Ranking expert tensors by imatrix-weighted sensitivity, the standard approach, made things monotonically worse (+48.0% per tensor, +23.9% block-coupled). Bumping ffn_down above its sibling projections, which is standard practice and which this work originally derived from first principles, was 11.3% worse than uniform at identical size and was retracted. And lifting the entire dense path to bf16 bought 0.00122 KLD/GB against an expert-side rate of 0.0027 KLD/GB, which closes what had looked like the largest remaining direction.

---

1. Introduction

1.1 The problem

A 35-billion-parameter MoE checkpoint with 3 billion active parameters behaves like neither a 35B dense model nor a 3B dense model. Its byte distribution and its compute distribution disagree by a factor of nearly three:

role group	params	% of file	% of active compute
routed experts (8 of 256 fire)	32.21 B	92.94%	34.2%
attention	1.03 B	2.96%	34.9%
output head	0.51 B	1.47%	17.3%
SSM / state	0.26 B	0.74%	8.7%
shared expert	0.13 B	0.36%	4.3%
token_embd	0.51 B	1.47%	0% (get_rows)

65.8% of the active parameters are not routed. Active total is 2.95 B, which is what the "A3B" label means and a useful check that the accounting is right.

Every quantization decision on such a model is a decision about how to split a byte budget between a very large, very sparsely used store of weights and a small, densely used one. Published ladders take reasonable but different positions on that split, and until they are measured on one harness against one reference there is no way to tell which position is correct.

1.2 Why activation magnitude is the wrong ranking axis

The signal available to a llama.cpp practitioner is the importance matrix (imatrix), which accumulates E[x·xᵀ] per tensor over a calibration corpus. It is a good and cheap statistic and it is what llama.cpp's own mixes and both comparison ladders build on.

But E[x·xᵀ] is the input side of the Gauss-Newton (Fisher) approximation to the loss Hessian only. The full second-order term needs the output-side sensitivity as well, and the two are not interchangeable. Two failure modes follow, both of which were measured here (Section 7.1):

1. A depth bias that is mechanical, not informational. Activation norms grow with depth in a residual stream, so any ranking proportional to input energy nominates deep tensors. On this model the imatrix's top-8 blocks were [32..39]; direct measurement puts the highest-value blocks at [0..7].
2. A role bias against ffn_down. The down-projection's input is post-activation and small in norm, but its output is written straight into the residual stream. Input energy under-weights it in exact proportion to how much the activation function suppressed its input.

There is a deeper problem than either bias. The imatrix objective is defined per token. A tensor whose error is still doing damage a thousand tokens later is scored as if it were not.

1.3 The claim

ICE asks a different question about each tensor. Not how strongly it is driven, but how long the error it introduces survives.

In one line: freeze what propagates, spend everything else on the library.

1.4 Contributions

1. A four-class taxonomy of quantization error by propagation distance, with a procedure for finding each class in an arbitrary architecture (Section 3.1).
2. Four measured laws governing MoE bit allocation: the sparsity exchange rate, the convexity cap, the placement rule, and the epistemic floor (Section 4). Law 4 is cross-validated against an independent third-party harness.
3. A twelve-tier controlled comparison against two published ladders on one harness and one reference, with a Pareto analysis that eliminates three tiers as dominated (Sections 6.1, 6.2).
4. A direct measurement of the depth gain function by sliding a fixed-width band across depth at constant file size (Section 6.3).
5. An error-budget decomposition proving the dense path is correctly fed and closing it as an optimization direction (Section 6.4).
6. Seven documented negative results, including the retraction of a rule this work originally derived and shipped (Section 7).

---

2. Background

2.1 The model

All measurements are on Ornith-1.5-35B-A3B (qwen35moe in llama.cpp) and its abliterated derivative. The structure below is read from the checkpoint's own tensor table.

blocks	41 = 30 linear/SSM + 10 full-attention (indices 3, 7, ..., 39) + 1 MTP (blk.40)
residual width	2048
experts	256, 8 active, so sparsity ratio E/k = 32
KV	2 heads, head_dim 256, on 10 blocks only, about 20 KB/token
vocab	248,320, so output.weight is 508.6 M in a single tensor
trunk parameters	34,660,610,688

Every block is 95.6% expert weight (805.3 M of about 842 M). The surgical field is the remaining 35 M per block plus the two 508 M end caps.

Two structural facts drive the method. First, the hybrid 3:1 layout means only 10 of 40 trunk blocks maintain a KV cache, so the cached class is a quarter the size it would be on a uniformly attentive model of the same depth. Second, blk.40 is a multi-token-prediction draft head whose proposals are verified by the target model, which changes what its errors cost.

Per-role parameter counts, which the whole allocation problem turns on:

role	tensors	params	% of model
ffn_{down,gate,up}_exps	120	32,212,254,720	92.94%
token_embd	1	508,559,360	1.47%
output	1	508,559,360	1.47%
attn_qkv (fused, SSM blocks)	30	503,316,480	1.45%
attn_gate	30	251,658,240	0.73%
ssm_out	30	251,658,240	0.73%
attn_q	10	167,772,160	0.48%
attn_output	10	83,886,080	0.24%
shared experts	120	125,829,120	0.36%
ffn_gate_inp (routers)	40	20,971,520	0.06%
attn_k	10	10,485,760	0.03%
attn_v	10	10,485,760	0.03%
ssm_alpha, ssm_beta	60	3,932,160	0.01%
ssm_conv1d	30	983,040	0.00%
norms and misc	231	~176,128	0.00%

2.2 Quantization in llama.cpp

Two format families are relevant:

• k-quants (Q3_K, Q4_K, Q5_K, Q6_K): block-scaled with a super-block hierarchy, nominal 3.4375 / 4.5 / 5.5 / 6.5625 bpw.
• IQ quants (IQ3_S, IQ4_XS, IQ4_NL): codebook based, nominal 3.44 / 4.25 / 4.5 bpw.

A recipe is a --tensor-type-file: a list of regex-to-type rules applied per tensor. This is what makes controlled experiments possible. Two recipes that assign the same number of tensors to each type produce files of identical size, so any KLD difference between them is attributable purely to which tensors got which type. Every comparison in Sections 6 and 7 is constructed that way.

Some tensors are never quantized by upstream regardless of recipe: ffn_gate_inp is excluded by name, ssm_conv1d has a first dimension of 4 and is not a multiple of the block size, and 1-D tensors including all norms are skipped. This matters for honest accounting and is treated in Section 3.2.

2.3 The metric

All quality numbers are mean KL divergence against the bf16 checkpoint's own token distribution, via llama-perplexity --kl-divergence on WikiText-2 raw test. KLD is preferred to perplexity because it measures distance from the model being approximated rather than from the ground truth text, because it stays sensitive in the regime of interest (tiers below differ by 40% in KLD while their perplexity ratios differ in the third decimal), and because it is deterministic.

2.4 Prior ladders

Unsloth Dynamic 2.0 (UD) assigns bits per tensor with a calibration-driven policy and, in the variants measured, pins the dense path at Q8_0 across every size tier. This gives high active bpw at a given file size, and on the measured ladder the UD tiers are the strongest files at the large end.

LocalAI APEX uses a sensitivity-driven allocation with a documented methodology and a published technical report, which is the structural model for this document. Its tiers span a wider size range, down to 14.24 GB.

AtomicChat AD appears once, in Section 4.4, as an independent harness whose published measurements were used to cross-validate Law 4. Its tiers are not otherwise compared here.

The comparison in Section 6.1 is not an argument that any ladder is badly designed. It is a measurement of where each sits on a size-versus-fidelity plane when all of them are evaluated identically, which is information no individual publication can provide because each uses its own harness and its own reference.

2.5 Related literature

The problem addressed here is the format-constrained version of the second-order weight-sensitivity problem studied by OBD and OBS and, in the modern LLM setting, by GPTQ, AWQ and SqueezeLLM. Those methods solve for values within a fixed format. ICE solves for format assignment across tensors and takes the values from the stock quantizer, so the two are complementary. Full references are in Section 10.

---

3. Method

3.1 The four error-propagation classes

Classify every tensor by how far an error travels, not by how big its activations are.

class	mechanism	failure shape	treatment
DISCRETE	error flips an argmax, so a different computation runs	categorical, not graded	exact (F32)
RECURRENT	error enters a state decay and compounds multiplicatively along the sequence	grows with context length	exact (F32)
CACHED	error is written once and re-read by later tokens	frozen in, cannot be re-decided	near-exact (F16)
INSTANT	error affects this token's output only	graded, bounded	this is where the budget lives

How to find them in an arbitrary architecture.

• DISCRETE: anything feeding a top_k, argmax or hard gate. In an MoE that is the router (ffn_gate_inp), plus any hard routing or branching signal.
• RECURRENT: anything multiplying or parameterizing a carried state. SSM and Mamba decay and timestep terms (ssm_alpha, ssm_beta, ssm_a, ssm_dt), gated linear-attention decay, any RNN gate. Not the state's output projection, which is INSTANT.
• CACHED: whatever is written into the KV cache, so attn_k and attn_v. Note that attn_q is not cached, which is the trap discussed in Section 7.4.
• INSTANT: everything else.

The three non-INSTANT classes share one property that the per-token imatrix objective cannot see: their cost is a function of context length. That is the property the method is named for.

3.2 Rule 1: freeze the propagating set

On this model the entire propagating set is 47 M parameters, 0.14% of the checkpoint, and holding all of it exact costs 0.15 GB. It is not a trade-off, it is a line item you can simply pay. Expect the same on any sparse MoE, because routers and state gates are structurally tiny.

An honest accounting splits that 47 M in two.

Already exact without any recipe rule (22.3 M params). ffn_gate_inp is excluded by name in src/llama-quant.cpp; ssm_conv1d has a first dimension of 4, not a multiple of the block size; ssm_a, ssm_dt and all norms are 1-D and never quantized. Pinning any of these would be inert, and ICE's recipe validator rejects inert rules. The propagation argument still explains why no ladder should try to reclaim those bytes, but ICE does not get credit for them.

Actually pinned by ICE (27.0 M params, 0.078%):

tensor	blocks	params	class	assigned	bytes
ssm_alpha	30	1,966,080	RECURRENT	F32	7.86 MB
ssm_beta	30	1,966,080	RECURRENT	F32	7.86 MB
attn_k	11	11,534,336	CACHED	F16	23.07 MB
attn_v	11	11,534,336	CACHED	F16	23.07 MB
total	82 tensors	27,000,832			61.86 MB

The same tensors at the dense path's Q8_0 would occupy 28.7 MB, so Rule 1's incremental cost is 33 MB, about 0.03 GB.

Everything always-on but INSTANT (attn_q, attn_output, attn_qkv, attn_gate, ssm_out, shared experts, output, token_embd) is held at Q8_0. Section 4.1 explains why that is not generosity but arithmetic.

3.3 Rule 2: the draft head follows the tier

A draft head does not need target-model precision. blk.40 is an MTP block whose proposals are verified by the target model, so its errors cost acceptance rate, not correctness. ICE pins only the draft block's projections at Q8_0 and lets its experts follow the tier.

The saving is exact: blk.40 holds 3 × 268,435,456 = 805,306,368 expert parameters, which occupy 855 MB at Q8_0 and 453 MB at the 23G tier's Q4_K. Rule 2 frees 402 MB per tier, an order of magnitude more than Rule 1 spends.

Other ladders pin the whole block at Q8_0, drop it to Q4_0, or delete it. The question of what a drafter actually needs does not appear to have been asked. Measured draft acceptance with the shrunk block is 96.04% (388 of 404), so the saving is not paid for out of throughput. See Section 6.6 for why this is not a clean A/B.

3.4 Rule 3: uniform width across FFN roles

ffn_down, ffn_gate and ffn_up receive the same bit-width.

This contradicts standard practice and it contradicts what this work originally shipped. The first ladder put ffn_down 0.79 bpw above its siblings, derived from the imatrix participation ratio (Section 4.2). A controlled test at identical size refuted it: 0.041192 uniform against 0.046449 bumped, 11.3% better uniform, reproduced exactly on a rebuild. Rebuilding the whole ladder uniform improved every rung, by 7.3%, 11.3%, 3.3% and 3.1% at 19, 21, 23 and 25 GB.

The retraction is discussed in Section 7.2. Practical rule until someone measures better: allocate the expert stack uniformly, despite convention.

3.5 Rule 4: choose the narrowest bracket the budget allows

A tier's expert budget is met by mixing two types, a high and a low, in whatever ratio hits the target size. There is usually more than one pair that can hit a given average. Always choose the pair with the smallest bit-width separation, by the convexity argument in Section 4.2. The enumeration is over a small discrete set, so it is exhaustive rather than heuristic.

3.6 Rule 5: place the high type shallow-first, only if the gap exceeds 0.47 bpw

Given the pair from Rule 4 and the ratio that hits the budget, the remaining freedom is which blocks get the high type. Measurement (Section 4.3) says assign them shallow-first, but only when the two types are far enough apart to make position meaningful. Below about half a bit of separation the attempt costs more than it returns.

This is a conditional rule and the condition is checkable before any file is built. On the one tier in the ladder whose gap is 0.25 bpw, applying it anyway made the result 1.6% worse.

3.7 The algorithm

Input:  target file size S, model M, imatrix I (for the quantizer, not for ranking)
Output: a tensor-type-file recipe

1. classify every tensor into DISCRETE / RECURRENT / CACHED / INSTANT
   pin the RECURRENT and CACHED tensors that upstream would otherwise quantize
   reject any rule that would be inert                          # Rule 1

2. always-on INSTANT tensors -> Q8_0                            # Law 1
   S_remaining <- S - size(everything assigned so far)

3. blk.40 projections -> Q8_0; blk.40 experts join the pool     # Rule 2

4. b_target <- S_remaining / n_expert_params
   enumerate all (high, low) type pairs bracketing b_target
   choose the pair minimizing (bpw_high - bpw_low)              # Rule 4 / Law 2
   n_high <- the count that hits b_target

5. if (bpw_high - bpw_low) > 0.47:                              # Rule 5 / Law 3
       high type -> blocks [0 .. n_high-1]                       # shallow-first
   else:
       high type -> even Bresenham dither across depth
   gate, up and down always move together                       # Rule 3

6. emit; verify rule count, zero double-matches, zero uncovered, zero inert

Step 6 is not optional. On all four shipped tiers it reports 0 double-matched, 0 uncovered, 0 inert, and it is the check that catches a regex which did not match what it was intended to match, something llama-quantize will not warn about.

3.8 Adapting ICE to a new MoE

1. Inventory. Dump the tensor list, group by role, compute params, share of file, and share of active compute (weighting routed experts by k/E).
2. Sparsity ratio E/k. This is the always-on-to-expert exchange rate and it sets the entire shape of the recipe (Law 1).
3. Classify every tensor into the four classes. Sum the propagating set. If it is well under 1% of the model, freeze all of it.
4. Confirm grading is not worth it by water-filling over the routing histogram (Law 2).
5. Fixed body = propagating set + always-on core + draft projections. Everything else is one dial: size = body + E_params · bpw / 8.
6. Bound the ladder. Fit Law 4 on any measured tiers to find k₀. The useful band runs from the cliff, where the expert term explodes, to saturation, where the floor dominates. On this model that was 17 to 27 GB.

3.9 What ICE deliberately does not do

No per-layer graded bit ladder. Refused explicitly, not overlooked. Law 2 caps the gain at +0.139 bpw and the direction of a depth profile was unresolved at the time: the participation ratio says the middle blocks need more bits because they are flattest, convention says the edges do because they matter more functionally.

Note this does not conflict with Rule 5. Rule 5 is a binary partition between two already-chosen types at a fixed count, which changes no bit-widths. A graded ladder changes the bit-widths themselves and pays the convexity penalty for it.

No imatrix-derived tensor ranking. Three variants were tested and all three lost (Section 7.1).

---

4. The four measured laws

4.1 Law 1, sparsity: always-on bits are worth E/k routed bits

For anything that fires on every token, compute share and storage cost are both proportional to parameter count, so work per byte is a constant. Routed experts divide that constant by the routing fraction.

Measured on this model, with 8 of 256 experts active: every always-on role scores about 270 units of work per GB-per-bpw; the experts score 8.5. The ratio is 31.8, against a predicted E/k = 32.

A bit spent on the always-on core is worth E_total / E_active bits spent on the expert bank. Compute this ratio first for any new MoE.

This is why the always-on core sits at Q8_0 in every tier and why the ladder's entire dynamic range lives in the experts. It also explains a negative result: lifting attn_gate or ssm_out to Q6_K is harmful, because they are always-on and the 32× law says those bytes belong in the experts.

4.2 Law 2, convexity: allocation cleverness is capped

Quantization error falls as 4^-b, which is convex. By Jensen's inequality, non-uniform allocation beats uniform only when sensitivity varies enormously, and the gain scales with the logarithm of the spread.

Measured by water-filling over the real per-expert routing histogram:

sensitivity spread	optimal gain over uniform
46× (real expert routing)	+0.139 bpw
3.7× (depth spread of ffn_down)	less
~250× (hypothetical)	~1 bpw

Routing on this model is well balanced (209.5 effective experts of 256, Gini 0.331), which is precisely why the spread is only 46×.

Corollary: stop tuning ladders. This is why every published recipe lands within about 0.3 bpw of every other one. Per-expert allocation, depth grading and per-layer bands are all worth less than 0.15 bpw.

The second consequence of the same convexity is the bracket rule. At a fixed average bit-width, a tight bracket incurs strictly less expected cost than a wide one. Measured at constant file size:

tier	narrow pair	wide pair	penalty for widening
21G	Q4_K / IQ4_XS (gap 0.25) 0.039182	Q4_K / Q3_K (gap 1.06) 0.045673	+16.6%
19G	IQ4_XS / IQ3_S (gap 0.81) 0.060015	Q4_K / Q3_K (gap 1.06) 0.064213	+7.0%

Both wide alternatives hit the same file size, so the penalty is entirely bracket width. This is Rule 4.

An empirical check on the 4^-b form: fitting the four shipped tiers after subtracting the Law 4 floor gives a decay of roughly 0.65 to 0.75 bpw per halving of excess KLD, against 0.5 bpw for a pure 4^-b. The residual difference is the dense path, which does not move across the ladder.

4.3 Law 3, placement: gain scales with the type gap

Within a fixed bracket, where the high type goes still matters. Two measurements establish the rule, both byte-identical by construction.

The depth gain function. A 15-block window of the high type was slid across depth at constant count, on the 23G tier:

band position	mean KLD
blocks [0..14]	0.032407
blocks [6..20]	0.033014
even dither (reference)	0.034162
blocks [18..32]	0.036107
blocks [25..39]	0.036136
blocks [12..26]	0.036760

Fitting an exponential gives g(t) ≈ exp(−t/9.95), RMS residual 0.00077 against roughly 0.0005 error bars. A bit in block 0 is worth about 50 times a bit in block 39.

Two things a simple decay model does not capture. First, [12..26] and the even dither share a centre of mass but differ by 0.0026, so concentration matters independently of position. Second, [12..26] is the worst row despite being centred, because it leaves blocks [0..11] entirely on the low type: starving the shallow region costs more than enriching the deep region helps.

The conditional. Applying shallow-first across six byte-identical pairs:

tier	pair	gap (bpw)	dither	shallow	gain
21G	Q4_K / IQ4_XS	0.25	0.039182	0.039814	+1.6%
19G	IQ4_XS / IQ3_S	0.81	0.060015	0.058676	−2.2%
25G	Q5_K / Q4_K	1.00	0.028955	0.028131	−2.8%
23G	Q5_K / Q4_K	1.00	0.034290	0.032536	−5.1%
21G	Q4_K / Q3_K	1.06	0.045673	0.043766	−4.2%
19G	Q4_K / Q3_K	1.06	0.064213	0.059623	−7.1%

Linear fit: gain ≈ −8.5% per bpw of gap, r = −0.894, zero-crossing at 0.47 bpw.

The 21G row is the important one. It is the only tier with a sub-0.5 bpw gap and the only one where shallow-first made things worse. The law predicted the sign before that measurement was taken, which is the reason to treat it as a law rather than a fit through six points.

Laws 2 and 3 pull against each other. Widening the gap makes placement more valuable (Law 3) but costs more than placement can recover (Law 2). The resolution is ICE's allocation policy: use the narrowest bracket the budget allows, then apply shallow-first only where that bracket is still wider than about 0.5 bpw. Law 2 dominates; placement is a second-order correction inside an already-chosen bracket, never a reason to widen one.

4.4 Law 4, the floor is epistemic, not numeric

Fitting KLD ≈ k₀ + c_d·4^(−b_dense) + c_e·4^(−b_expert) on two independent harnesses:

harness	floor k₀	c_e/c_d
this work (WikiText-2, ctx 2048)	0.0205	5.91
AtomicChat AD (eval_neutral, ctx 4096)	0.0219	7.22

The best file measured, UD-Q6_K, scores 0.0221 against a 0.0205 floor, so it is 8% above the floor: the top of the ladder is saturated. Above roughly 5.5 expert bpw you are buying calibration noise, not accuracy.

You cannot out-bit a wrong prior. Quantization is lossy compression guided by a belief about importance. Improve the imatrix before adding bits.

A direct probe later confirmed the fit from a completely different direction. Holding the experts at Q8_0 and quantizing only the dense path measures 0.018297 on the abliterated checkpoint (Section 6.4), against a fitted k₀ of 0.0205 and a card estimate of about 0.0185 on the clean one. An extrapolated intercept and a direct measurement agreeing to within about 10% is the strongest evidence available that the floor is real.

Caveat, stated when the law was formulated and still true. This fitted model ranks the extremes well and gets close calls wrong: it misorders AtomicChat's Q4_K_M against AD-Q4_K-IQ4_XS. Use it to find the floor and the saturation point, never to choose between two candidate recipes.

---

5. Experimental setup

item	value
base checkpoints	Ornith-1.5-35B-A3B (bf16) and its abliterated derivative (bf16)
quantizer	upstream llama.cpp, pinned commit, unmodified
importance matrix	one generic imatrix, identical across every file compared
corpus	WikiText-2 raw test
context	2048
chunks	16 for the twelve-tier ladder comparison, 64 for every controlled experiment
metric	mean KLD against the corresponding bf16 checkpoint
hardware	rented single-GPU instances, plus CPU-only for the bf16 reference pass

5.1 Controls

Three disciplines make the comparisons interpretable, and their absence is the usual reason quantization comparisons are not:

1. One reference per model. Every KLD in a given table is against the same bf16 logits file. Numbers are never compared across base models; where both checkpoints appear they are in separate tables and labelled.
2. Byte-identical variants. Every experiment in Sections 6 and 7 permutes type assignments rather than changing type counts, so the variant matches its control in size, usually to the byte. A KLD difference is then a pure recipe effect.
3. One harness. Same binary, corpus, context and chunk count throughout.

5.2 Reproducibility

The same recipe measured through three different paths:

path	chunks	device	mean KLD
published model card	16	CPU	0.0345
controlled study, phase 2	64	CPU	0.034162
controlled study, phase 5	64	GPU	0.034290

Spread under 1%. KLD against a fixed reference is deterministic; chunk count tightens the error bar without moving the mean, and the CPU/GPU difference is floating-point accumulation order.

Error bars. At 16 chunks the reported bars are 0.0008 to 0.0012; at 64 chunks they are 0.0005 to 0.0007. All significance statements below use the 64-chunk figure.

---

6. Results

6.1 The twelve-tier comparison

All twelve files, one harness, one reference, 16 chunks, clean base model. Sorted by mean KLD.

tier	size	mean KLD	99% KLD	99.9% KLD	PPL ratio	same top-1	active bpw	file bpw
UD-Q6_K	30.20 GB	0.0221	0.220	0.809	0.9957	93.85%	8.063	6.804
UD-Q5_K_S	25.83 GB	0.0272	0.269	1.030	0.9862	93.51%	7.693	5.820
25G-ICE	24.84 GB	0.0303	0.296	1.049	0.9814	93.16%	7.686	5.597
APEX-I-Balanced	26.28 GB	0.0345	0.328	1.401	0.9907	92.55%	6.913	5.922
23G-ICE	22.83 GB	0.0361	0.332	1.272	0.9885	92.65%	7.523	5.143
UD-Q4_K_XL	23.21 GB	0.0380	0.389	1.405	0.9740	92.47%	7.471	5.230
21G-ICE	20.84 GB	0.0412	0.392	1.665	0.9924	92.03%	7.357	4.695
APEX-I-Quality	23.84 GB	0.0415	0.424	1.386	0.9843	91.98%	6.699	5.371
19G-ICE	18.82 GB	0.0608	0.615	1.877	1.0030	90.32%	7.192	4.240
UD-IQ4_XS	18.68 GB	0.0723	0.758	2.464	1.0526	89.46%	6.762	4.209
APEX-I-Compact	17.56 GB	0.0954	0.899	2.962	1.0101	87.83%	5.228	3.956
APEX-I-Mini	14.24 GB	0.2608	2.489	5.915	1.2281	80.49%	4.180	3.208

Read by size-matched pairs, which is the only fair way to read it:

• 23G-ICE (22.83 GB) against UD-Q4_K_XL (23.21 GB): 0.38 GB smaller, 5.0% closer to bf16.
• 23G-ICE against APEX-I-Quality (23.84 GB): 1.01 GB smaller, 13.0% closer.
• 25G-ICE (24.84 GB) against APEX-I-Balanced (26.28 GB): 1.44 GB smaller, 12.2% closer.
• 19G-ICE (18.82 GB) against UD-IQ4_XS (18.68 GB): 15.8% closer at +0.14 GB.
• At the top, UD-Q5_K_S and UD-Q6_K are the two best files measured and nothing in the ICE ladder reaches them. This agrees with Law 4: above about 25 GB the expert term is saturated and the dense path is what remains, which is the regime the UD policy of pinning everything dense at Q8_0 is built for.
• At the bottom, APEX-I-Compact and APEX-I-Mini are the only tiers below 18.5 GB. ICE does not operate there and offers no evidence about that regime.

The advantage grows as size falls, from −1.9% at 25 GB to −18.6% at 21 GB against the UD curve, which is Law 4 again: at the top the floor dominates and no allocation choice can move it.

A note on active bpw. The column is included because it explains where the bits went, but it must not be used to rank. Tested against measured KLD it misorders 4 of the 8 UD and APEX tiers (Spearman +0.95), and 10 of AtomicChat's 13 on a second ladder (+0.71). It is a linear average of bit-widths while error is not linear in bits, so Q8_0-dense-path designs come out flattered.

6.2 Pareto analysis: twelve tiers reduce to nine

A tier is worth publishing only if nothing else is both smaller and closer to bf16.

Pareto-optimal (nine): APEX-I-Mini (14.24 / 0.2608), APEX-I-Compact (17.56 / 0.0954), UD-IQ4_XS (18.68 / 0.0723), 19G-ICE (18.82 / 0.0608), 21G-ICE (20.84 / 0.0412), 23G-ICE (22.83 / 0.0361), 25G-ICE (24.84 / 0.0303), UD-Q5_K_S (25.83 / 0.0272), UD-Q6_K (30.20 / 0.0221).

Dominated (three):

covered tier	covered by
UD-Q4_K_XL (23.21 GB / 0.0380)	23G-ICE: 0.38 GB smaller, 5.0% better
APEX-I-Quality (23.84 GB / 0.0415)	23G-ICE: 1.01 GB smaller, 13.0% better
APEX-I-Balanced (26.28 GB / 0.0345)	UD-Q5_K_S: 0.45 GB smaller, 21.1% better; also 25G-ICE: 1.44 GB smaller, 12.2% better

This is the most useful output of the comparison, and it is worth being precise about its scope. It says that at these three operating points, on this model, with this harness, another published file is strictly better on both axes. It does not say those ladders are worse in general: the same analysis keeps two UD tiers as the best files on the board and two APEX tiers as the only options below 18.5 GB.

6.3 Applying Law 3 to the shipped ladder

Each recipe byte-identical to the tier it replaces, abliterated base, 64 chunks:

tier	gap	shipped	revised	change	decision
19G	0.81	0.060015	0.058676	−2.2% (1.4σ)	adopt
21G	0.25	0.039182	0.039814	+1.6%	do not change
23G	1.00	0.034290	0.032536	−5.1% (3.6σ)	adopt
25G	1.00	0.028955	0.028131	−2.8% (1.3σ)	adopt

Only 23G is individually significant at conventional thresholds. 19G and 25G are directionally consistent at about 1.3σ each and are adopted because the law they confirm was fitted on data including an independent significant point and a correctly predicted failure. 21G is left unchanged, which is the honest output of a conditional rule when the condition does not hold. Its revised recipe file has the same SHA-256 as the one it replaces.

At the local slope of each tier's size-versus-quality curve these gains are worth approximately 0.14, 0, 0.49 and 0.34 GB.

6.4 The error budget, and why the dense path is closed

Decomposing total damage at the 23G operating point by holding one side exact:

	mean KLD	share of damage	bytes	share of bytes
dense path	0.018297	53.6%	2.7 GB	12%
routed experts at 4.875 bpw	0.015993	46.4%	19.6 GB	88%

The dense path causes more than half the damage in an eighth of the bytes, which looks like an obvious misallocation. It is not, and the deciding measurement is direct:

variant	mean KLD	size
control 23G	0.034290	22.34 GB
all 238 dense tensors lifted to BF16	0.031543	24.59 GB

Gain 0.002747 for +2.25 GB is 0.00122 KLD/GB, against a measured expert-side rate of 0.0027 KLD/GB. Those bytes are worth 2.2× more in the experts.

The dense path is not underfed, it is correctly fed. The 0.018297 figure is the intrinsic cost of quantizing the dense path at all, which is Law 4's floor seen from another angle, not a misallocation waiting to be recovered. The ICE split of 88% of bytes to experts is already on the right side of the trade and this direction is closed.

Sub-additivity. Dense-only damage (0.002747) plus expert-only damage (0.015993) is 0.018740, while the two together give 0.034290. The perturbations are strongly sub-additive, so per-role "share of the floor" was never a meaningful quantity, and per-role attribution of a joint quantization error should be treated with suspicion generally.

6.5 Draft acceptance

Measured acceptance with the shrunk blk.40: 96.04% (388 of 404). The baseline with the block fully pinned at Q8_0 was 92.97%, but on a different tier, so this is not a clean A/B and is not claimed as one. What it establishes is that the smaller draft head still drafts and there is no collapse, which is the condition Rule 2 needs.

6.6 Falsifiable predictions, scored

Three predictions were registered before measurement.

1. 25G-ICE beats UD-Q5_K_S on measured KLD at about 1 GB less. FAILED. 0.0303 against 0.0272. ICE is 0.99 GB smaller but not better. The prediction rested on the 0.79 bpw ffn_down gap, which was then refuted outright (Section 7.2).
2. 19G-ICE beats UD-IQ4_XS. HELD, 0.0608 against 0.0723, 15.8% better.
3. Un-pinning blk.40 costs under 2 pp of acceptance. HELD, direction unverified, for the reason in Section 6.5.

One of three failed, and the failure took a derived rule down with it. That is recorded here rather than quietly dropped because a method that only reports its successful predictions is not making predictions.

---

7. Negative results

Seven results reported at the same weight as the positive ones. Four of them refute standard practice and one retracts a rule this work itself shipped.

7.1 Imatrix-weighted sensitivity ranking (three variants, all lost)

Score each of the 120 routed-expert tensors by s_t = Σ_j a_j ‖ΔW‖² using the imatrix activations, then spend the high type on the top scorers. On the 23G tier, byte-identical:

variant	result against control
per-tensor ranking, roles free	+48.0%
block-coupled (gate, up, down move together)	+23.9%
propagation-corrected g(t)·s_t, roles free	−1.2% (0.6σ, noise)

The first two are catastrophic; the third is indistinguishable from doing nothing. The diagnosis is Section 1.2: s_t grows with depth mechanically, its top-8 blocks being [32..39] while the measured truth is [0..7]. Coupling the roles halves the damage without fixing the direction.

The third variant is the most informative failure. Multiplying by the measured g(t) fixes the depth axis and still does not beat the control, because s_t is also wrong on the role axis: it starves all 40 ffn_down tensors and gives back exactly what the depth correction won.

Conclusion: imatrix activation energy is not a usable ranking signal for inter-tensor bit allocation on this architecture. It remains the right input to the quantizer itself, which is how ICE uses it.

7.2 The ffn_down bump, and the retraction of the leverage rule

The imatrix stores per-channel importance h. Its concentration can be measured with the participation ratio PR = (Σh)² / (n·Σh²). Spiky h (low PR) is a gift because the quantizer knows where to spend its block scale; flat h (high PR) is the hard case. So flat tensors should need more bits. Measured:

tensor	PR/n
attn_k, attn_q, attn_v	0.0069 (about 14 of 2048 channels)
attn_qkv, attn_gate, ssm_alpha, ssm_beta	0.0128
ffn_gate_exps, ffn_up_exps	0.2048
ffn_down_exps	0.6104, flattest in the model

This gave a clean derivation: down should sit log4(0.6104/0.2048) = 0.79 bpw above gate and up. It agreed with convention, it agreed with the best published rungs (UD-Q4_K_XL and AtomicChat's AD-Q5_K-Q4_K both run about 1.0), and it shipped.

It is wrong. A controlled test at identical size:

expert allocation	size	mean KLD
down = gate/up + 0.79	20.82 GB	0.046449
uniform	20.85 GB	0.041192

Uniform is 11.3% better, reproduced exactly on a rebuild. The whole ladder was rebuilt uniform, improving every rung by 7.3%, 11.3%, 3.3% and 3.1%.

What PR actually predicts is unknown. It measures how concentrated the input importance is, which is not how much a tensor's error reaches the output. ffn_down writes straight into the residual stream while gate and up errors are attenuated through the gating nonlinearity first. Position in the circuit is not captured by PR and on this model it evidently dominates.

This is the same defect as Section 7.1, seen from a different direction: an input-side statistic used as an output-side one.

7.3 Per-expert allocation by routing frequency

Load balancing works on this model (209.5 effective experts of 256, Gini 0.331) and Law 2 caps the achievable gain at +0.139 bpw. A naive "hot 25% at 6.5 bpw, cold at 2.5" split is worse than uniform, 0.00586 against 0.00552. Dead.

7.4 Demoting attn_q

The tempting corollary of the CACHED class: if K and V persist and Q does not, demote Q below the always-on rate. The arithmetic kills it. Each query distributes attention weight summing to 1, so the total k-influence accumulated over a sequence is O(N), the same as q. Buys 63 MB and costs 0.17 active bpw. Dropped before shipping.

This is why Rule 1 promotes K and V rather than demoting Q, and the distinction is not cosmetic.

7.5 Dense role lifting

Experts held at Q8_0, one dense role at a time lifted to BF16, reference D = 0.019018:

role lifted	mean KLD	bytes added	rate (KLD/GB)
output head	0.018941	+0.48 GB	0.00016
token_embd	0.019217	+0.48 GB	negative
attention	0.017510	+0.94 GB	0.0016

Against an expert-side rate of 0.0027 KLD/GB none is worth doing, and token_embd is worth undoing. Section 6.4 generalizes this to the whole dense path.

Separately, demoting token_embd to Q4_K did not replicate: it helps at 21G and hurts at 19G. Lifting attn_gate or ssm_out to Q6_K is harmful, which is Law 1 doing its job.

7.6 Same-family type swaps

An early hypothesis held that mixing quantization families was itself harmful, since a k-quant's and an IQ quant's error distributions are not commensurate. Both 19G and 21G have same-family alternatives at their exact budgets:

tier	pair	dither	shallow
21G	Q4_K / IQ4_XS (shipped, mixed)	0.039182	0.039814
21G	Q4_K / Q3_K (same family)	0.045673	0.043766
19G	IQ4_XS / IQ3_S (shipped, same family)	0.060015	0.058676
19G	Q4_K / Q3_K (same family)	0.064213	0.059623

Both swaps lose, but this is not clean evidence for the family hypothesis, because forcing the same family also forced a much wider bpw gap, and Law 3 shows the gap is the real variable. The family hypothesis is confounded and unresolved, and is listed as open work rather than confirmed.

7.7 up to down scale migration

scale(up row j, s) and scale(down col j, 1/s) is exactly output-preserving, so variance could in principle be migrated out of down's columns (which pollute every row's quantization range) into up's rows (which get their own k-quant scales for free). Dead on this model: down's columns are already uniform, p95/p05 = 1.09. There is nothing to migrate.

---

8. Limitations

1. One model family. Every measurement is on qwen35moe at 35B-A3B. The decay constant 9.95, the 0.47 bpw zero-crossing, the +0.139 bpw cap and the floor k₀ are fitted on this architecture. The form of the laws should transfer; the constants should be re-measured. Law 1 is the exception, being arithmetic rather than a fit.
2. One evaluation corpus. WikiText-2 raw test only. KLD against bf16 is a faithfulness metric, not a capability metric, and says nothing directly about code, multilingual or long-context behaviour. This last gap is awkward, since the method's central claim is about context length.
3. One imatrix, and a generic one. Every experiment reuses the same generic importance matrix. By Law 4 this is the largest untested variable in the work.
4. Error bars. 1σ is 0.0005 to 0.0007 at 64 chunks. Only the 23G placement result clears 3σ; 19G and 25G are adopted on the strength of the law rather than their individual significance, which is stated rather than hidden.
5. Rule 1 is not expressible on a fused-QKV checkpoint. It needs attn_k and attn_v as separate tensors. On this model 30 of 41 blocks fuse them into attn_qkv; those blocks happen to carry no KV cache, but on an architecture that fuses QKV and attends, Rule 1 would require splitting the tensor at conversion time, which a --tensor-type-file cannot do.
6. Rule 2 applies only to checkpoints carrying a draft head, and its acceptance evidence is not a clean A/B (Section 6.5).
7. The family-versus-gap confound in Section 7.6 is unresolved.
8. Comparison scope. Two published ladders at twelve tiers on one model. Not a survey.

---

9. Conclusion and open work

ICE classifies tensors by how far their quantization error travels rather than by how strongly they are activated. The three classes whose error outlives the current token turn out to be 0.14% of the checkpoint, so freezing them is a line item rather than a trade-off, and the recovered budget goes to the expert bank.

The resulting ladder is Pareto-optimal at its operating points against two published ladders measured on one harness, dominating three of their tiers, and the advantage grows as the budget falls. It does not win at the top of the ladder, where Law 4 says the floor dominates and the measurements agree.

The four laws are more portable than the ladder. They say: compute E/k first because it sets everything; stop tuning allocations because convexity caps the gain at about 0.14 bpw; place shallow-first only when the bracket is wide enough to make position mean anything; and stop adding bits near the floor because you cannot out-bit a wrong prior.

They also close two directions that looked promising: imatrix-weighted sensitivity ranking, and moving bytes from the experts to the dense path.

Open work, ranked by expected value.

1. A domain-matched imatrix. Untouched throughout, and Law 4 says it is worth more than any remaining allocation change.
2. Orthogonal rotation of the residual stream. RMSNorm is invariant to rotation but not to scaling, so a rotation folds into the norm weights exactly and for free, and yields a standard GGUF. With attention at PR 0.0069 the residual stream has about 14 dominant dimensions of 2048, and spreading that energy is worth an estimated 0.9 bpw, which is larger than everything in this report combined. Nobody has published rotation results for a hybrid SSM/MoE and the fixed SSM state may not tolerate it.
3. Shared basis plus per-expert delta. Store W̄ once and Δ_i per expert. If ‖Δ‖/‖W‖ ≈ ¼ that is about 2 bpw for 0.4% overhead. Needs a llama.cpp op, but the deciding measurement is free: compute ‖Δ_i‖/‖W̄‖ from the bf16. The counter-argument is that this architecture already has a shared expert which may have absorbed the common component.
4. Graded placement profiles on the wide-gap tiers, since every placement variant tested is a step function while the measured g(t) is smooth.
5. Disentangling family from gap (Section 7.6).
6. Re-measuring the constants on a second architecture, the only way to learn whether g(t)'s decay length is a property of this model or of residual transformers generally.

---

10. References

Quantization formats and tooling

1. G. Gerganov and the llama.cpp contributors. llama.cpp: LLM inference in C/C++. https://github.com/ggml-org/llama.cpp The k-quant and IQ formats, llama-quantize, the --tensor-type-file mechanism and the --kl-divergence evaluation path are all from this project. Every file measured here was produced by an unmodified build of it.
2. I. Kawrakow. K-quant format design (Q3_K through Q6_K), llama.cpp PR #1684 and subsequent work.
3. I. Kawrakow. IQ-quant and codebook format design (IQ3_S, IQ4_XS, IQ4_NL), llama.cpp.
4. The importance-matrix mechanism in llama.cpp, which accumulates per-tensor activation second moments over a calibration corpus and feeds the quantizer's scale search.

Second-order weight sensitivity, the basis for Sections 1.2 and 7.2

5. Y. LeCun, J. Denker and S. Solla. Optimal Brain Damage. NeurIPS 1989.
6. B. Hassibi and D. Stork. Second Order Derivatives for Network Pruning: Optimal Brain Surgeon. NeurIPS 1992.
7. E. Frantar, S. Ashkboos, T. Hoefler and D. Alistarh. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. ICLR 2023. The Hessian-based formulation whose Gauss-Newton approximation the imatrix represents the input half of.
8. J. Lin et al. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. MLSys 2024. The activation-magnitude approach that Section 7.1 finds insufficient for inter-tensor allocation on MoE.
9. S. Kim et al. SqueezeLLM: Dense-and-Sparse Quantization. ICML 2024. Sensitivity-weighted non-uniform quantization, the closest prior work in spirit, though solving for values rather than format assignment.

Allocation theory

10. C. Shannon. Coding Theorems for a Discrete Source with a Fidelity Criterion. IRE 1959. The 4^-b rate-distortion behaviour underlying Law 2.
11. Reverse water-filling for Gaussian sources, as in T. Cover and J. Thomas, Elements of Information Theory, ch. 10. The allocation procedure used to compute Law 2's +0.139 bpw bound.

Mixture-of-Experts, state-space models and speculative decoding

12. N. Shazeer et al. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. ICLR 2017. The top-k routing whose discreteness defines the DISCRETE class.
13. W. Fedus, B. Zoph and N. Shazeer. Switch Transformers. JMLR 2022.
14. A. Gu and T. Dao. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. 2023. The state decay terms that define the RECURRENT class.
15. Y. Leviathan, M. Kalman and Y. Matias. Fast Inference from Transformers via Speculative Decoding. ICML 2023. The verification property that makes Rule 2 sound.
16. F. Gloeckle et al. Better & Faster Large Language Models via Multi-token Prediction. ICML 2024. The MTP head structure used by blk.40.

Comparison ladders

17. Unsloth. Unsloth Dynamic 2.0 GGUFs. https://docs.unsloth.ai A calibration-driven per-tensor mixed-precision policy. Used here as a comparison ladder; its Q5_K_S and Q6_K tiers are the two best files in the Section 6.1 measurement.
18. E. Di Giacinto, R. Palethorpe and the LocalAI team. APEX: Adaptive Precision EXpert quantization. LocalAI, 2025. https://github.com/localai-org/apex-quant A documented sensitivity-driven allocation method with a published technical report, which is the structural model for this document.
19. AtomicChat AD GGUF ladder, used in Section 4.4 as an independent harness for cross-validating Law 4 and in Section 6.1 for the active-bpw ranking check.

Data and checkpoints

20. S. Merity, C. Xiong, J. Bradbury and R. Socher. Pointer Sentinel Mixture Models. ICLR 2017. WikiText-2, raw test split, as distributed with the llama.cpp CI dataset.
21. ornith-ai/Ornith-1.5-35B-A3B, the bf16 checkpoint and its MTP head, and the abliterated derivative used for the controlled experiments.

---

Appendix A: recipe format and the shipped rules

Each ICE tier is a --tensor-type-file of 443 rules, written out per tensor rather than as wildcards, so the file is a complete and auditable statement with no regex precedence to reason about. Excerpts are verbatim from the shipped 23G recipe.

A.1 Global rules (2):

^output\.weight$=Q8_0
token_embd\.weight=Q8_0

A.2 Rule 1, the pinned propagating set. 60 state-gate rules across the 30 SSM blocks, 22 K/V rules across the 11 attention blocks:

blk\.0\.ssm_alpha\.weight=F32
blk\.0\.ssm_beta\.weight=F32
...
blk\.11\.attn_k\.weight=F16
blk\.11\.attn_v\.weight=F16

Routers, ssm_conv1d, ssm_a, ssm_dt and the norms do not appear. Upstream holds them exact already, so a rule would be inert and the validator rejects it.

A.3 Rule 2, the draft block. All 11 of blk.40's rules, from the 23G tier:

blk\.40\.attn_k\.weight=F16          <- Rule 1 applies to the draft block too
blk\.40\.attn_v\.weight=F16
blk\.40\.attn_q\.weight=Q8_0
blk\.40\.attn_output\.weight=Q8_0
blk\.40\.nextn\.eh_proj\.weight=Q8_0
blk\.40\.ffn_down_shexp\.weight=Q8_0
blk\.40\.ffn_gate_shexp\.weight=Q8_0
blk\.40\.ffn_up_shexp\.weight=Q8_0
blk\.40\.ffn_down_exps\.weight=Q4_K  <- follows the tier, not pinned Q8_0
blk\.40\.ffn_gate_exps\.weight=Q4_K
blk\.40\.ffn_up_exps\.weight=Q4_K

The three _exps lines are the whole of Rule 2. They read IQ3_S at 19G, IQ4_XS at 21G, Q4_K at 23G and Q5_K at 25G, so the draft block is a full member of the expert pool and is eligible for the high type when the tier is generous enough to reach it.

A.4 Rules 3 to 5, the expert body. One rule per routed-expert tensor, with gate, up and down always receiving the same type within a block (Rule 3). The pair comes from Rule 4 and the block assignment from Rule 5. Verified against the shipped files:

tier	pair (high / low)	gap	expert bpw	high-type blocks	count
19G	IQ4_XS / IQ3_S	0.81	3.899	[0..22] shallow-first	23 / 41
21G	Q4_K / IQ4_XS	0.25	4.383	0 2 4 6 7 9 11 13 15 17 19 20 22 24 26 28 30 32 34 35 37 39 (even dither)	22 / 41
23G	Q5_K / Q4_K	1.00	4.868	[0..14] shallow-first	15 / 41
25G	Q5_K / Q4_K	1.00	5.353	[0..33] plus blk.40, shallow-first	35 / 41

21G is the tier where Rule 5's condition fails, so its block set is the even dither, byte-for-byte identical to the previous revision. That row is the method declining to change something, which is the point of having a condition.

A.5 Validating a recipe before building. Four checks, all cheap, all of which have caught a real error at least once:

1. Rule count. All four tiers have exactly 443 rules.
2. Zero double-matched. No tensor matched by two rules.
3. Zero uncovered. No intended tensor left to the base ftype by accident.
4. Zero inert. No rule that assigns a type the quantizer would have used anyway, which would silently misrepresent what the recipe does.

Plus one build-time check: sum the per-tensor sizes implied by the rules and compare to the built file. A mismatch means a regex did not match what it was meant to, and llama-quantize will not warn about that.

---

Appendix B: measurement protocol

B.1 The bf16 reference, once per base model, reused for every tier:

llama-perplexity -m base-bf16.gguf -f wiki.test.raw \
    -c 2048 --chunks 64 --kl-divergence-base base.kld

CPU-bound on a 69 GB file, roughly 25 minutes on a 30-core machine.

B.2 A tier:

llama-perplexity -m tier.gguf -f wiki.test.raw \
    -c 2048 --chunks 64 --kl-divergence-base base.kld --kl-divergence

B.3 Constructing a byte-identical variant. Count how many blocks receive the high type in the control. Produce the variant by permuting which blocks those are, never by changing the count. The files then match in size, usually to the byte. If they do not, the variant is invalid and must not be compared.

B.4 Error bars. 1σ is 0.0005 to 0.0007 at 64 chunks and 0.0008 to 0.0012 at 16. Differences below about 1.5% on a 0.03 mean KLD are not interpretable as single measurements, only as confirmations of a rule established elsewhere, and are labelled as such in Section 6.3.

---

Appendix C: index of evidence

appendix/original-ICE/PRINCIPLE.md   the model-agnostic method, as first written
appendix/original-ICE/RECIPE.md      its instantiation for this model, arithmetic shown
appendix/recipes/                    every tensor-type-file cited
appendix/measurements/               raw llama-perplexity output for every KLD quoted
appendix/SHA256SUMS.txt              checksum for every file above

appendix/measurements/ also holds sensitivity.jsonl, the 600-row per-tensor sensitivity map used to construct the attacks that Section 7.1 reports as failures, and ice_final_ledger.txt, the original build log with its error bars.

Recipes for the three dominated tiers are not included: they were dropped after the Pareto analysis and no recipe was retained. Their measured numbers are in Section 6.1.

---

Appendix D: the propagating set in full

tensor	blocks	params	class	status
ffn_gate_inp, ffn_gate_inp_shexp	80	21,053,440	DISCRETE	already F32, excluded by name upstream
ssm_conv1d	30	983,040	RECURRENT	already F32, first dim 4
ssm_a, ssm_dt, all norms	100+	~264,000	RECURRENT / n/a	already F32, 1-D
ssm_alpha	30	1,966,080	RECURRENT	pinned F32 by ICE
ssm_beta	30	1,966,080	RECURRENT	pinned F32 by ICE
attn_k	11	11,534,336	CACHED	pinned F16 by ICE
attn_v	11	11,534,336	CACHED	pinned F16 by ICE
whole propagating set		~47.3 M (0.14%)		0.15 GB
of which ICE pins	82 tensors	27,000,832 (0.078%)		61.86 MB, +33 MB over Q8_0

Excluded and why: attn_q (167.8 M) and attn_output (83.9 M) are INSTANT and held at the always-on Q8_0; see Section 7.4 for the arithmetic that killed demoting them further. attn_qkv (503.3 M) fuses Q, K and V on the 30 SSM blocks, so the class cannot be separated out there; those blocks carry no KV cache, but it is a structural limit rather than a design choice (Section 8.5).

---

This report describes work done independently. The comparison ladders in Section 6 are the published work of their respective authors and are measured here, not reproduced or modified.