## [1.3.0] - 2026-08-20
Browse files# Changelog
All notable changes to this project are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.3.0] - 2026-08-20
### Added
- **Nano 24% tier validation** for the AutoRound int4-lineage Ornith-1.5-9B run. The generated `model-AutoRound-ASHQ1-Nano-24pc.gguf` measured **4,106 MiB** on disk and **3.84 BPW** in the quantizer report.
- Recorded Nano perplexity at **10.3148** on `wiki.test.raw` (`ctx=2048`, 64 chunks, batch 512, 15 threads, Flash-Attention), at **1,190.7 tok/s** over **90.3 s**.
- Recorded the Nano result as **+2.2216 PPL** versus the documented `Quality-36pc` baseline of **8.0932**.
### Notes
- The run completed successfully. It reported a 5 MiB dry-run over-target condition and disabled free embedding allocation because the 1,032 MiB embedding exceeded 15% of the Nano target; final output was nevertheless produced at 4,106 MiB.
|
@@ -1412,7 +1412,7 @@ def _show_floors():
|
|
| 1412 |
# βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 1413 |
|
| 1414 |
TIER_RATIOS={
|
| 1415 |
-
"nano":0.
|
| 1416 |
"mini":0.27,
|
| 1417 |
"compact":0.33,
|
| 1418 |
"quality":0.39,
|
|
|
|
| 1412 |
# βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 1413 |
|
| 1414 |
TIER_RATIOS={
|
| 1415 |
+
"nano":0.24,
|
| 1416 |
"mini":0.27,
|
| 1417 |
"compact":0.33,
|
| 1418 |
"quality":0.39,
|
|
@@ -52,13 +52,17 @@ When tied layers are detected (such as shared input/output embeddings or symmetr
|
|
| 52 |
When weights are derived from an AutoRound W4A16 source (recorded via `.provenance.json` sidecars):
|
| 53 |
* **Information Saturation**: Because the underlying weights have been pre-conditioned for 4-bit representations, assigning container formats above `Q5_K` yields negligible signal recovery ($\Delta \mathcal{Q} \to 0$).
|
| 54 |
* **Automated Capping**: The ASHQ1 optimizer caps attention and FFN blocks at `Q5_K`, attention gates at `Q6_K`, and MTP heads at `Q6_K`, while recurrent memory states hold `Q8_0`. The bit-budget freed by these ceilings flows to the imatrix-ranked tensors that convert it into measurable error reduction.
|
| 55 |
-
* **Tier Pruning**: On AutoRound int4 lineage, generation of the 48% `Fidelity` tier is skipped by default, focusing compute resources on optimal `Quality` (36%), `Compact` (33%), `Mini` (27%), and `Nano` (
|
| 56 |
* **Cap Saturation**: On AutoRound int4 lineage, `Compact` (33%) drives every attention and FFN projection to the `Q5_K` cap. Measured on a 9B `qwen35` source (17 091 MiB BF16): `Compact` ships 33.06%, `Quality` ships 35.49% against a 36% target. Beyond `Compact`, the budget buys `token_embd` (+280 MiB), `output` (+121 MiB) and the layer-0 FFN pair (+15 MiB); blocks 1 through 31 stay identical across both tiers. `Quality` therefore retargets to 36% on this lineage; other lineages keep 39%.
|
| 57 |
* **Discrete Ceiling**: Once every capped class saturates, the upgrade queue holds only large indivisible moves β `output` Q5_KβQ6_K at +129 MiB, `token_embd` Q6_KβQ8_0 at +235 MiB. Reachable footprints become discrete: on the reference 9B source, 35.49%, then 36.25%, then 37.62%. A target landing between two steps delivers the lower one, which the greedy scheduler reports as remaining slack.
|
| 58 |
|
| 59 |
---
|
| 60 |
|
| 61 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
|
| 63 |
Calibration quality directly determines activation matrix fidelity. The suite implements an exact multi-domain quota:
|
| 64 |
|
|
|
|
| 52 |
When weights are derived from an AutoRound W4A16 source (recorded via `.provenance.json` sidecars):
|
| 53 |
* **Information Saturation**: Because the underlying weights have been pre-conditioned for 4-bit representations, assigning container formats above `Q5_K` yields negligible signal recovery ($\Delta \mathcal{Q} \to 0$).
|
| 54 |
* **Automated Capping**: The ASHQ1 optimizer caps attention and FFN blocks at `Q5_K`, attention gates at `Q6_K`, and MTP heads at `Q6_K`, while recurrent memory states hold `Q8_0`. The bit-budget freed by these ceilings flows to the imatrix-ranked tensors that convert it into measurable error reduction.
|
| 55 |
+
* **Tier Pruning**: On AutoRound int4 lineage, generation of the 48% `Fidelity` tier is skipped by default, focusing compute resources on optimal `Quality` (36%), `Compact` (33%), `Mini` (27%), and `Nano` (24% in the 2026-08-20 validation run) variants.
|
| 56 |
* **Cap Saturation**: On AutoRound int4 lineage, `Compact` (33%) drives every attention and FFN projection to the `Q5_K` cap. Measured on a 9B `qwen35` source (17 091 MiB BF16): `Compact` ships 33.06%, `Quality` ships 35.49% against a 36% target. Beyond `Compact`, the budget buys `token_embd` (+280 MiB), `output` (+121 MiB) and the layer-0 FFN pair (+15 MiB); blocks 1 through 31 stay identical across both tiers. `Quality` therefore retargets to 36% on this lineage; other lineages keep 39%.
|
| 57 |
* **Discrete Ceiling**: Once every capped class saturates, the upgrade queue holds only large indivisible moves β `output` Q5_KβQ6_K at +129 MiB, `token_embd` Q6_KβQ8_0 at +235 MiB. Reachable footprints become discrete: on the reference 9B source, 35.49%, then 36.25%, then 37.62%. A target landing between two steps delivers the lower one, which the greedy scheduler reports as remaining slack.
|
| 58 |
|
| 59 |
---
|
| 60 |
|
| 61 |
+
## π 4. Validation Evidence
|
| 62 |
+
|
| 63 |
+
The 2026-08-20 validation run measured the AutoRound int4-lineage `Nano-24pc` artifact at **4,106 MiB** on disk, with a quantizer-reported effective size of **3.84 BPW**. On `wiki.test.raw` at `ctx=2048`, 64 chunks, batch 512, 15 threads, and Flash-Attention, it measured **PPL 10.3148** at **1,190.7 tok/s** over **90.3 s**. Against the documented `Quality-36pc` baseline of **8.0932**, this is **+2.2216 PPL**.
|
| 64 |
+
|
| 65 |
+
## π 5. Calibration Dataset Standard
|
| 66 |
|
| 67 |
Calibration quality directly determines activation matrix fidelity. The suite implements an exact multi-domain quota:
|
| 68 |
|
|
@@ -62,7 +62,7 @@ All tiers maintain strict byte-budget percentages relative to the original unqua
|
|
| 62 |
|
| 63 |
| Tier | File Ratio | Base Type | Typical Use Case | Target Preservation |
|
| 64 |
| :--- | :---: | :---: | :--- | :--- |
|
| 65 |
-
| **Nano** | **
|
| 66 |
| **Mini** | **27%** | `IQ4_XS` | Efficient high-throughput serving | Balanced `IQ4_XS`/`IQ3_S` distribution |
|
| 67 |
| **Compact** | **33%** | `IQ4_XS` | Balanced daily-driver footprint | Down-proj `Q4_K`, Gate/Up `IQ4_XS` |
|
| 68 |
| **Quality** | **39%** | `Q5_K_M` | Near-lossless general deployment | Full `Q4_K`/`Q5_K` attention coverage |
|
|
@@ -72,7 +72,7 @@ All tiers maintain strict byte-budget percentages relative to the original unqua
|
|
| 72 |
|
| 73 |
> **Int4 lineage tier ladder**: `Compact` (33%) already drives every attention and FFN projection to the `Q5_K` cap. Budget beyond that point buys `token_embd` and `output` precision only, so `Quality` retargets to **36%** on this lineage and writes `-Quality-36pc.gguf`. `Fidelity` stays pruned.
|
| 74 |
>
|
| 75 |
-
> Measured on a 9B `qwen35` source (17 091 MiB BF16): Nano **
|
| 76 |
|
| 77 |
### Perplexity Benchmarks (Ornith-1.5-9B)
|
| 78 |
|
|
@@ -83,8 +83,10 @@ Evaluated on `wiki.test.raw` (Wikitext-2), `n_ctx=2048`, 64 chunks, Flash-Attent
|
|
| 83 |
| **Quality-36pc** | 6.06 GiB | ~7.5 GiB | **8.0932** | baseline | ~1241 tok/s |
|
| 84 |
| **Compact-33pc** | 5.65 GiB | ~7.0 GiB | **8.1290** | +0.0358 | ~1241 tok/s |
|
| 85 |
| **Mini-27pc** | 4.62 GiB | ~5.8 GiB | **9.5101** | +1.4169 | ~1442 tok/s |
|
| 86 |
-
| **Nano-
|
| 87 |
|
|
|
|
|
|
|
| 88 |
> **Takeaways:**
|
| 89 |
> - `Quality-36pc` provides near-lossless perplexity for production inference.
|
| 90 |
> - `Compact-33pc` loses only **0.0358 PPL** while saving ~416 MiB, ideal for 8 GB VRAM setups.
|
|
|
|
| 62 |
|
| 63 |
| Tier | File Ratio | Base Type | Typical Use Case | Target Preservation |
|
| 64 |
| :--- | :---: | :---: | :--- | :--- |
|
| 65 |
+
| **Nano** | **24%** | `IQ3_XXS` | Maximum compression, edge & mobile VRAM | Core gates `Q6_K`, Down-proj `IQ2_S` |
|
| 66 |
| **Mini** | **27%** | `IQ4_XS` | Efficient high-throughput serving | Balanced `IQ4_XS`/`IQ3_S` distribution |
|
| 67 |
| **Compact** | **33%** | `IQ4_XS` | Balanced daily-driver footprint | Down-proj `Q4_K`, Gate/Up `IQ4_XS` |
|
| 68 |
| **Quality** | **39%** | `Q5_K_M` | Near-lossless general deployment | Full `Q4_K`/`Q5_K` attention coverage |
|
|
|
|
| 72 |
|
| 73 |
> **Int4 lineage tier ladder**: `Compact` (33%) already drives every attention and FFN projection to the `Q5_K` cap. Budget beyond that point buys `token_embd` and `output` precision only, so `Quality` retargets to **36%** on this lineage and writes `-Quality-36pc.gguf`. `Fidelity` stays pruned.
|
| 74 |
>
|
| 75 |
+
> Measured on a 9B `qwen35` source (17 091 MiB BF16): Nano **24.02%**, Mini **27.01%**, Compact **33.06%**, Quality **35.49%**. `Quality` stops short of its nominal target because the surviving upgrades are indivisible blocks of 129 MiB and above.
|
| 76 |
|
| 77 |
### Perplexity Benchmarks (Ornith-1.5-9B)
|
| 78 |
|
|
|
|
| 83 |
| **Quality-36pc** | 6.06 GiB | ~7.5 GiB | **8.0932** | baseline | ~1241 tok/s |
|
| 84 |
| **Compact-33pc** | 5.65 GiB | ~7.0 GiB | **8.1290** | +0.0358 | ~1241 tok/s |
|
| 85 |
| **Mini-27pc** | 4.62 GiB | ~5.8 GiB | **9.5101** | +1.4169 | ~1442 tok/s |
|
| 86 |
+
| **Nano-24pc** | 4.01 GiB | ~4.5 GiB | **10.3148** | **+2.2216** | 1190.7 tok/s |
|
| 87 |
|
| 88 |
+
> **Run note:** The Nano-24pc result above is the 2026-08-20 validation run: 4,106 MiB on disk, 3.84 effective quantizer BPW, `ctx=2048`, 64 chunks, batch 512, 15 threads, and Flash-Attention.
|
| 89 |
+
>
|
| 90 |
> **Takeaways:**
|
| 91 |
> - `Quality-36pc` provides near-lossless perplexity for production inference.
|
| 92 |
> - `Compact-33pc` loses only **0.0358 PPL** while saving ~416 MiB, ideal for 8 GB VRAM setups.
|
|
@@ -98,6 +98,8 @@ Interactive CLI tool to benchmark GGUF files against `wiki.test.raw` (auto-downl
|
|
| 98 |
- Real-time ETA and chunk progression streaming.
|
| 99 |
- Generates a comparative summary table with Ξ PPL.
|
| 100 |
|
|
|
|
|
|
|
| 101 |
---
|
| 102 |
|
| 103 |
## π― 3. Recommended Minimum Tiers by Model Size
|
|
|
|
| 98 |
- Real-time ETA and chunk progression streaming.
|
| 99 |
- Generates a comparative summary table with Ξ PPL.
|
| 100 |
|
| 101 |
+
The 2026-08-20 Nano validation used `model-AutoRound-ASHQ1-Nano-24pc.gguf` (4,106 MiB; quantizer-reported 3.84 BPW) and measured PPL **10.3148** at **1,190.7 tok/s** in **90.3 s** with `ctx=2048`, 64 chunks, batch 512, 15 threads, and Flash-Attention.
|
| 102 |
+
|
| 103 |
---
|
| 104 |
|
| 105 |
## π― 3. Recommended Minimum Tiers by Model Size
|