Soulfate24 commited on
Commit
81df991
Β·
verified Β·
1 Parent(s): 7ff3e96

## [1.3.0] - 2026-08-20

Browse files

# Changelog

All notable changes to this project are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.3.0] - 2026-08-20

### Added

- **Nano 24% tier validation** for the AutoRound int4-lineage Ornith-1.5-9B run. The generated `model-AutoRound-ASHQ1-Nano-24pc.gguf` measured **4,106 MiB** on disk and **3.84 BPW** in the quantizer report.
- Recorded Nano perplexity at **10.3148** on `wiki.test.raw` (`ctx=2048`, 64 chunks, batch 512, 15 threads, Flash-Attention), at **1,190.7 tok/s** over **90.3 s**.
- Recorded the Nano result as **+2.2216 PPL** versus the documented `Quality-36pc` baseline of **8.0932**.

### Notes

- The run completed successfully. It reported a 5 MiB dry-run over-target condition and disabled free embedding allocation because the 1,032 MiB embedding exceeded 15% of the Nano target; final output was nevertheless produced at 4,106 MiB.

Files changed (4) hide show
  1. ASHQ1.py +1 -1
  2. CHARTER.md +6 -2
  3. README.md +5 -3
  4. USAGE.md +2 -0
ASHQ1.py CHANGED
@@ -1412,7 +1412,7 @@ def _show_floors():
1412
  # ═══════════════════════════════════════════════════════════════════════════
1413
 
1414
  TIER_RATIOS={
1415
- "nano":0.21,
1416
  "mini":0.27,
1417
  "compact":0.33,
1418
  "quality":0.39,
 
1412
  # ═══════════════════════════════════════════════════════════════════════════
1413
 
1414
  TIER_RATIOS={
1415
+ "nano":0.24,
1416
  "mini":0.27,
1417
  "compact":0.33,
1418
  "quality":0.39,
CHARTER.md CHANGED
@@ -52,13 +52,17 @@ When tied layers are detected (such as shared input/output embeddings or symmetr
52
  When weights are derived from an AutoRound W4A16 source (recorded via `.provenance.json` sidecars):
53
  * **Information Saturation**: Because the underlying weights have been pre-conditioned for 4-bit representations, assigning container formats above `Q5_K` yields negligible signal recovery ($\Delta \mathcal{Q} \to 0$).
54
  * **Automated Capping**: The ASHQ1 optimizer caps attention and FFN blocks at `Q5_K`, attention gates at `Q6_K`, and MTP heads at `Q6_K`, while recurrent memory states hold `Q8_0`. The bit-budget freed by these ceilings flows to the imatrix-ranked tensors that convert it into measurable error reduction.
55
- * **Tier Pruning**: On AutoRound int4 lineage, generation of the 48% `Fidelity` tier is skipped by default, focusing compute resources on optimal `Quality` (36%), `Compact` (33%), `Mini` (27%), and `Nano` (21%) variants.
56
  * **Cap Saturation**: On AutoRound int4 lineage, `Compact` (33%) drives every attention and FFN projection to the `Q5_K` cap. Measured on a 9B `qwen35` source (17 091 MiB BF16): `Compact` ships 33.06%, `Quality` ships 35.49% against a 36% target. Beyond `Compact`, the budget buys `token_embd` (+280 MiB), `output` (+121 MiB) and the layer-0 FFN pair (+15 MiB); blocks 1 through 31 stay identical across both tiers. `Quality` therefore retargets to 36% on this lineage; other lineages keep 39%.
57
  * **Discrete Ceiling**: Once every capped class saturates, the upgrade queue holds only large indivisible moves β€” `output` Q5_Kβ†’Q6_K at +129 MiB, `token_embd` Q6_Kβ†’Q8_0 at +235 MiB. Reachable footprints become discrete: on the reference 9B source, 35.49%, then 36.25%, then 37.62%. A target landing between two steps delivers the lower one, which the greedy scheduler reports as remaining slack.
58
 
59
  ---
60
 
61
- ## πŸ“ˆ 4. Calibration Dataset Standard
 
 
 
 
62
 
63
  Calibration quality directly determines activation matrix fidelity. The suite implements an exact multi-domain quota:
64
 
 
52
  When weights are derived from an AutoRound W4A16 source (recorded via `.provenance.json` sidecars):
53
  * **Information Saturation**: Because the underlying weights have been pre-conditioned for 4-bit representations, assigning container formats above `Q5_K` yields negligible signal recovery ($\Delta \mathcal{Q} \to 0$).
54
  * **Automated Capping**: The ASHQ1 optimizer caps attention and FFN blocks at `Q5_K`, attention gates at `Q6_K`, and MTP heads at `Q6_K`, while recurrent memory states hold `Q8_0`. The bit-budget freed by these ceilings flows to the imatrix-ranked tensors that convert it into measurable error reduction.
55
+ * **Tier Pruning**: On AutoRound int4 lineage, generation of the 48% `Fidelity` tier is skipped by default, focusing compute resources on optimal `Quality` (36%), `Compact` (33%), `Mini` (27%), and `Nano` (24% in the 2026-08-20 validation run) variants.
56
  * **Cap Saturation**: On AutoRound int4 lineage, `Compact` (33%) drives every attention and FFN projection to the `Q5_K` cap. Measured on a 9B `qwen35` source (17 091 MiB BF16): `Compact` ships 33.06%, `Quality` ships 35.49% against a 36% target. Beyond `Compact`, the budget buys `token_embd` (+280 MiB), `output` (+121 MiB) and the layer-0 FFN pair (+15 MiB); blocks 1 through 31 stay identical across both tiers. `Quality` therefore retargets to 36% on this lineage; other lineages keep 39%.
57
  * **Discrete Ceiling**: Once every capped class saturates, the upgrade queue holds only large indivisible moves β€” `output` Q5_Kβ†’Q6_K at +129 MiB, `token_embd` Q6_Kβ†’Q8_0 at +235 MiB. Reachable footprints become discrete: on the reference 9B source, 35.49%, then 36.25%, then 37.62%. A target landing between two steps delivers the lower one, which the greedy scheduler reports as remaining slack.
58
 
59
  ---
60
 
61
+ ## πŸ“Š 4. Validation Evidence
62
+
63
+ The 2026-08-20 validation run measured the AutoRound int4-lineage `Nano-24pc` artifact at **4,106 MiB** on disk, with a quantizer-reported effective size of **3.84 BPW**. On `wiki.test.raw` at `ctx=2048`, 64 chunks, batch 512, 15 threads, and Flash-Attention, it measured **PPL 10.3148** at **1,190.7 tok/s** over **90.3 s**. Against the documented `Quality-36pc` baseline of **8.0932**, this is **+2.2216 PPL**.
64
+
65
+ ## πŸ“ˆ 5. Calibration Dataset Standard
66
 
67
  Calibration quality directly determines activation matrix fidelity. The suite implements an exact multi-domain quota:
68
 
README.md CHANGED
@@ -62,7 +62,7 @@ All tiers maintain strict byte-budget percentages relative to the original unqua
62
 
63
  | Tier | File Ratio | Base Type | Typical Use Case | Target Preservation |
64
  | :--- | :---: | :---: | :--- | :--- |
65
- | **Nano** | **21%** | `IQ3_XXS` | Maximum compression, edge & mobile VRAM | Core gates `Q6_K`, Down-proj `IQ2_S` |
66
  | **Mini** | **27%** | `IQ4_XS` | Efficient high-throughput serving | Balanced `IQ4_XS`/`IQ3_S` distribution |
67
  | **Compact** | **33%** | `IQ4_XS` | Balanced daily-driver footprint | Down-proj `Q4_K`, Gate/Up `IQ4_XS` |
68
  | **Quality** | **39%** | `Q5_K_M` | Near-lossless general deployment | Full `Q4_K`/`Q5_K` attention coverage |
@@ -72,7 +72,7 @@ All tiers maintain strict byte-budget percentages relative to the original unqua
72
 
73
  > **Int4 lineage tier ladder**: `Compact` (33%) already drives every attention and FFN projection to the `Q5_K` cap. Budget beyond that point buys `token_embd` and `output` precision only, so `Quality` retargets to **36%** on this lineage and writes `-Quality-36pc.gguf`. `Fidelity` stays pruned.
74
  >
75
- > Measured on a 9B `qwen35` source (17 091 MiB BF16): Nano **21.10%**, Mini **27.01%**, Compact **33.06%**, Quality **35.49%**. `Quality` stops short of its nominal target because the surviving upgrades are indivisible blocks of 129 MiB and above.
76
 
77
  ### Perplexity Benchmarks (Ornith-1.5-9B)
78
 
@@ -83,8 +83,10 @@ Evaluated on `wiki.test.raw` (Wikitext-2), `n_ctx=2048`, 64 chunks, Flash-Attent
83
  | **Quality-36pc** | 6.06 GiB | ~7.5 GiB | **8.0932** | baseline | ~1241 tok/s |
84
  | **Compact-33pc** | 5.65 GiB | ~7.0 GiB | **8.1290** | +0.0358 | ~1241 tok/s |
85
  | **Mini-27pc** | 4.62 GiB | ~5.8 GiB | **9.5101** | +1.4169 | ~1442 tok/s |
86
- | **Nano-21pc** | 3.60 GiB | ~4.5 GiB | 430.7392 | collapse | ~1321 tok/s |
87
 
 
 
88
  > **Takeaways:**
89
  > - `Quality-36pc` provides near-lossless perplexity for production inference.
90
  > - `Compact-33pc` loses only **0.0358 PPL** while saving ~416 MiB, ideal for 8 GB VRAM setups.
 
62
 
63
  | Tier | File Ratio | Base Type | Typical Use Case | Target Preservation |
64
  | :--- | :---: | :---: | :--- | :--- |
65
+ | **Nano** | **24%** | `IQ3_XXS` | Maximum compression, edge & mobile VRAM | Core gates `Q6_K`, Down-proj `IQ2_S` |
66
  | **Mini** | **27%** | `IQ4_XS` | Efficient high-throughput serving | Balanced `IQ4_XS`/`IQ3_S` distribution |
67
  | **Compact** | **33%** | `IQ4_XS` | Balanced daily-driver footprint | Down-proj `Q4_K`, Gate/Up `IQ4_XS` |
68
  | **Quality** | **39%** | `Q5_K_M` | Near-lossless general deployment | Full `Q4_K`/`Q5_K` attention coverage |
 
72
 
73
  > **Int4 lineage tier ladder**: `Compact` (33%) already drives every attention and FFN projection to the `Q5_K` cap. Budget beyond that point buys `token_embd` and `output` precision only, so `Quality` retargets to **36%** on this lineage and writes `-Quality-36pc.gguf`. `Fidelity` stays pruned.
74
  >
75
+ > Measured on a 9B `qwen35` source (17 091 MiB BF16): Nano **24.02%**, Mini **27.01%**, Compact **33.06%**, Quality **35.49%**. `Quality` stops short of its nominal target because the surviving upgrades are indivisible blocks of 129 MiB and above.
76
 
77
  ### Perplexity Benchmarks (Ornith-1.5-9B)
78
 
 
83
  | **Quality-36pc** | 6.06 GiB | ~7.5 GiB | **8.0932** | baseline | ~1241 tok/s |
84
  | **Compact-33pc** | 5.65 GiB | ~7.0 GiB | **8.1290** | +0.0358 | ~1241 tok/s |
85
  | **Mini-27pc** | 4.62 GiB | ~5.8 GiB | **9.5101** | +1.4169 | ~1442 tok/s |
86
+ | **Nano-24pc** | 4.01 GiB | ~4.5 GiB | **10.3148** | **+2.2216** | 1190.7 tok/s |
87
 
88
+ > **Run note:** The Nano-24pc result above is the 2026-08-20 validation run: 4,106 MiB on disk, 3.84 effective quantizer BPW, `ctx=2048`, 64 chunks, batch 512, 15 threads, and Flash-Attention.
89
+ >
90
  > **Takeaways:**
91
  > - `Quality-36pc` provides near-lossless perplexity for production inference.
92
  > - `Compact-33pc` loses only **0.0358 PPL** while saving ~416 MiB, ideal for 8 GB VRAM setups.
USAGE.md CHANGED
@@ -98,6 +98,8 @@ Interactive CLI tool to benchmark GGUF files against `wiki.test.raw` (auto-downl
98
  - Real-time ETA and chunk progression streaming.
99
  - Generates a comparative summary table with Ξ” PPL.
100
 
 
 
101
  ---
102
 
103
  ## 🎯 3. Recommended Minimum Tiers by Model Size
 
98
  - Real-time ETA and chunk progression streaming.
99
  - Generates a comparative summary table with Ξ” PPL.
100
 
101
+ The 2026-08-20 Nano validation used `model-AutoRound-ASHQ1-Nano-24pc.gguf` (4,106 MiB; quantizer-reported 3.84 BPW) and measured PPL **10.3148** at **1,190.7 tok/s** in **90.3 s** with `ctx=2048`, 64 chunks, batch 512, 15 threads, and Flash-Attention.
102
+
103
  ---
104
 
105
  ## 🎯 3. Recommended Minimum Tiers by Model Size