ASHQ1 Technical Charter & Precision Guarantees

This document establishes the architectural principles, mathematical formulation, and invariant guarantees governing the AutoRound + ASHQ1 quantization suite.

📐 1. Architectural Principles

1.1 Dual-Stage Optimization Model

The suite operates on a two-stage sequential precision boundary:

Weight-Space Reconditioning (AutoRound): Minimizes local layer reconstruction error over calibration activations: $$\min_{\mathbf{W}_q} |\mathbf{W}\mathbf{X} - \mathbf{W}_q\mathbf{X}|_F^2 + \lambda \mathcal{R}(\mathbf{W}_q)$$ where $\mathbf{W}_q$ represents the discrete integer matrix with continuous scale and zero adjustments.
Activation-Space Precision Allocation (ASHQ1): Given an empirical Fisher Information Proxy $\mathbf{F} = \text{diag}(\mathbf{H})$ derived from activation variances in imatrix.gguf (legacy imatrix.dat files are read as-is), ASHQ1 solves a constrained knapsack problem: $$\max_{{t_i}} \sum_{i} \mathbf{F}i \cdot \Delta \mathcal{Q}(t_i) \quad \text{subject to} \quad \sum{i} \text{Size}(t_i, \text{params}i) \le \text{Budget}{\text{VRAM}}$$ where $t_i \in \text{TIER_ORDER}$ and $\Delta \mathcal{Q}(t_i) = 2^{-2 \cdot \text{eff_bpw}(t_i)}$.

🛡️ 2. Invariant Precision Guarantees

To prevent numerical drift and safeguard autoregressive context integrity, ASHQ1 enforces absolute tensor classification floors:

┌──────────────────────────────┬───────────────┬────────────────────────────────────────────┐
│ Tensor Class                 │ Minimum Floor │ Rationale                                  │
├──────────────────────────────┼───────────────┼────────────────────────────────────────────┤
│ Norms & Scales (ln, rms)     │ F16 / F32     │ Preserves dynamic scale variance           │
│ Attention Gates (gate_proj)  │ Q6_K          │ Prevents non-linear token routing collapse │
│ Recurrent State (ssm_alpha)  │ Q8_0          │ Guarantees stable hidden state accumulation│
│ Speculative Heads (MTP/NextN)│ Q6_K / Q8_0   │ Maximizes draft token acceptance rate      │
│ Uncalibrated (no imatrix)    │ IQ4_XS        │ Keeps llama-quantize within safe tiers     │
│ Vision Projectors (mmproj)   │ F32 (critical)│ Protects high-resolution visual grounding  │
└──────────────────────────────┴───────────────┴────────────────────────────────────────────┘

2.1 Pinned Structural Layers

1D Vectors & Bias Tensors: All vectors with fewer than 4096 elements or non-matrix shapes remain strictly in F32 or F16.
Block Alignment Rules: Tensors must adhere to integer multiples of the GGML block size (32 or 256 elements). Unaligned shapes are automatically promoted to the nearest valid container (F16/Q8_0).
Speculative Block Norms: enorm, hnorm and any 1-D vector inside the MTP/NextN block stay pinned in F32 and receive no tier rule, keeping the draft head numerically aligned with the trunk.
Imatrix Coverage Floor: Every tensor whose statistics are absent from the merged importance matrix holds at IQ4_XS or above, both at assignment time and inside the downgrade queue.

2.2 Tied-Weight Group Preservation

When tied layers are detected (such as shared input/output embeddings or symmetric MoE gates), ASHQ1 binds the corresponding parameters into unified equivalence sets. The optimizer updates their quantization tiers simultaneously to prevent directional divergence.

🔬 3. AutoRound Lineage & Saturation Bounds

When weights are derived from an AutoRound W4A16 source (recorded via .provenance.json sidecars):

Information Saturation: Because the underlying weights have been pre-conditioned for 4-bit representations, assigning container formats above Q5_K yields negligible signal recovery ($\Delta \mathcal{Q} \to 0$).
Automated Capping: The ASHQ1 optimizer caps attention and FFN blocks at Q5_K, attention gates at Q6_K, and MTP heads at Q6_K, while recurrent memory states hold Q8_0. The bit-budget freed by these ceilings flows to the imatrix-ranked tensors that convert it into measurable error reduction.
Tier Pruning: On AutoRound int4 lineage, generation of both the 48% Fidelity and 36% Quality tiers is skipped by default, focusing compute resources on optimal Compact (33%), Mini (27%), and Nano (24% in the 2026-08-20 validation run) variants.
Cap Saturation: On AutoRound int4 lineage, Compact (33%) drives every attention and FFN projection to the Q5_K cap. Measured on a 9B qwen35 source (17 091 MiB BF16): Compact ships 33.06%, Quality ships 35.49% against a 36% target. Beyond Compact, the perplexity difference remains negligible across all model sizes (1B to 9B, with Δ PPL ≤ 0.0358 on 9B), as blocks 1 through 31 remain identical between tiers. Compact therefore represents the maximum standard tier for this lineage.
Discrete Ceiling: Once every capped class saturates, the upgrade queue holds only large indivisible moves — output Q5_K→Q6_K at +129 MiB, token_embd Q6_K→Q8_0 at +235 MiB. Reachable footprints become discrete: on the reference 9B source, 35.49%, then 36.25%, then 37.62%. A target landing between two steps delivers the lower one, which the greedy scheduler reports as remaining slack.

📊 4. Validation Evidence

The 2026-08-20 validation run measured the AutoRound int4-lineage Nano-24pc artifact at 4,106 MiB on disk, with a quantizer-reported effective size of 3.84 BPW. On wiki.test.raw at ctx=2048, 64 chunks, batch 512, 15 threads, and Flash-Attention, it measured PPL 10.3148 at 1,190.7 tok/s over 90.3 s. Against the documented Quality-36pc baseline of 8.0932, this is +2.2216 PPL.

📈 5. Calibration Dataset Standard

Calibration quality directly determines activation matrix fidelity. The suite implements an exact multi-domain quota:

55% Agentic Execution: Tool-use traces, JSON schema parameters, and multi-turn function call loops.
20% Formal Logic & Code: Formal mathematical proofs (Lean 4), algorithmic Python implementations, and syntactic structures.
15% Semantic Diversity: Multilingual sequences, encyclopedic knowledge, and conversational discourse.
10% Frontier Reasoning: Extended thinking traces and complex reasoning chains.

All samples undergo token-count round-trip verification to ensure consistent sequence boundaries during llama-imatrix evaluation.