Soulfate24 commited on
Commit
4ba2e75
·
verified ·
1 Parent(s): 58ce54f

## [1.1.1] — 2026-08-20 - **`Compact` tier budget raised from 30% to 33% of the source BF16 footprint.**

Browse files

# Changelog

---

## [1.1.1] — 2026-08-20

### Changed

- **`Compact` tier budget raised from 30% to 33% of the source BF16 footprint.**
The additional ~10% budget is distributed by the greedy knapsack scheduler
(`_push_upgrade`) along the highest `(importance × ΔMSE) / ΔMiB` gradient. Measured
effects on a dense 4B checkpoint:
- `ffn_down` on high-importance layers lifts from `IQ4_XS` to `Q4_K`.
- `ffn_gate` / `ffn_up` lift from `IQ3_S` to `IQ4_XS` across the majority of blocks.
- `attn_q` / `attn_k` / `attn_v` reach `Q4_K` on the first and last transformer blocks.
- Pinned classes are unaffected: `norms` and `ssm_params` stay `F16`, `gdn_state`
stays `Q8_0`, 1-D vectors and block-unaligned rows stay `F32`.
- **Tier grid is now uniform.** Steps become 21% → 27% → 33% → 39% → 48%, a constant
6-point spacing across the whole ladder, replacing the previous 3-point Mini→Compact
and 9-point Compact→Quality gap.
- Recommended minimum tier for ~4B parameter models now reads `Compact (33%)` in
`README.md`, `USAGE.md`, and the deployment guidance.
- `CHARTER.md` section 3 tier-pruning list updated to `Compact (33%)`.

### Migration

Output filenames now carry the `-33pc` suffix:

```
model-AutoRound-ASHQ1-Compact-30pc.gguf → model-AutoRound-ASHQ1-Compact-33pc.gguf
```

`02_BF16-GGUF-to-ASHQ1.py` resolves the skip-if-exists check against the exact
filename, so a previously generated `-30pc` artifact is treated as absent and the
Compact tier is regenerated. Rename or archive existing `-30pc` files before
re-running the orchestrator to keep the working directory unambiguous.

Every other tier (`Nano` 21%, `Mini` 27%, `Quality` 39%, `Fidelity` 48%) keeps its
ratio and its filename. Imatrix files, provenance sidecars, and the
`ashq1-overhead.json` calibration cache remain valid and are reused as-is.

### Notes on AutoRound int4 lineage

On checkpoints carrying an int4 AutoRound provenance sidecar, the extra budget
saturates earlier: `INT4_LINEAGE_CAP` (`Q5_K`) holds attention and FFN blocks,
`INT4_GATE_CAP` (`Q6_K`) holds attention gates, and `INT4_GDN_CAP` (`Q8_0`) holds
recurrent state. The realized file size therefore lands below the nominal 33%
target on such models, and the converger reports the shortfall in its
`[converge] Selected best result` line. This matches the information-saturation
bound stated in `CHARTER.md` section 3.

Files changed (5) hide show
  1. 02_BF16-GGUF-to-ASHQ1.py +1 -1
  2. ASHQ1.py +1 -1
  3. CHARTER.md +1 -1
  4. README.md +2 -2
  5. USAGE.md +1 -1
02_BF16-GGUF-to-ASHQ1.py CHANGED
@@ -7,7 +7,7 @@ information ceiling reached at Quality — and modules use Quality
7
  (override: ASHQ1_INCLUDE_FIDELITY=1).
8
  The COMPLETE model (including MTP head) is preferred over the no-mtp trunk:
9
  ASHQ1 keeps the nextn layer at Q8_0 (Q6_K on int4 lineage) and excludes it from the size budget.
10
- Final naming: model-AutoRound-ASHQ1-Compact-30pc.gguf (AutoRound lineage detected
11
  via step 00 sidecar), model-ASHQ1-…-NNpc.gguf otherwise; mmproj-ASHQ1-Balanced-NNpc.gguf
12
  (pct relative to source mmproj).
13
  Lineage override: ASHQ1_LINEAGE=autoround|plain.
 
7
  (override: ASHQ1_INCLUDE_FIDELITY=1).
8
  The COMPLETE model (including MTP head) is preferred over the no-mtp trunk:
9
  ASHQ1 keeps the nextn layer at Q8_0 (Q6_K on int4 lineage) and excludes it from the size budget.
10
+ Final naming: model-AutoRound-ASHQ1-Compact-33pc.gguf (AutoRound lineage detected
11
  via step 00 sidecar), model-ASHQ1-…-NNpc.gguf otherwise; mmproj-ASHQ1-Balanced-NNpc.gguf
12
  (pct relative to source mmproj).
13
  Lineage override: ASHQ1_LINEAGE=autoround|plain.
ASHQ1.py CHANGED
@@ -1410,7 +1410,7 @@ def _show_floors():
1410
  TIER_RATIOS={
1411
  "nano":0.21,
1412
  "mini":0.27,
1413
- "compact":0.30,
1414
  "quality":0.39,
1415
  "fidelity":0.48,
1416
  }
 
1410
  TIER_RATIOS={
1411
  "nano":0.21,
1412
  "mini":0.27,
1413
+ "compact":0.33,
1414
  "quality":0.39,
1415
  "fidelity":0.48,
1416
  }
CHARTER.md CHANGED
@@ -52,7 +52,7 @@ When tied layers are detected (such as shared input/output embeddings or symmetr
52
  When weights are derived from an AutoRound W4A16 source (recorded via `.provenance.json` sidecars):
53
  * **Information Saturation**: Because the underlying weights have been pre-conditioned for 4-bit representations, assigning container formats above `Q5_K` yields negligible signal recovery ($\Delta \mathcal{Q} \to 0$).
54
  * **Automated Capping**: The ASHQ1 optimizer caps attention and FFN blocks at `Q5_K`, attention gates at `Q6_K`, and MTP heads at `Q6_K`, while recurrent memory states hold `Q8_0`. The bit-budget freed by these ceilings flows to the imatrix-ranked tensors that convert it into measurable error reduction.
55
- * **Tier Pruning**: On AutoRound int4 lineage, generation of the 48% `Fidelity` tier is skipped by default, focusing compute resources on optimal `Quality` (39%), `Compact` (30%), `Mini` (27%), and `Nano` (21%) variants.
56
 
57
  ---
58
 
 
52
  When weights are derived from an AutoRound W4A16 source (recorded via `.provenance.json` sidecars):
53
  * **Information Saturation**: Because the underlying weights have been pre-conditioned for 4-bit representations, assigning container formats above `Q5_K` yields negligible signal recovery ($\Delta \mathcal{Q} \to 0$).
54
  * **Automated Capping**: The ASHQ1 optimizer caps attention and FFN blocks at `Q5_K`, attention gates at `Q6_K`, and MTP heads at `Q6_K`, while recurrent memory states hold `Q8_0`. The bit-budget freed by these ceilings flows to the imatrix-ranked tensors that convert it into measurable error reduction.
55
+ * **Tier Pruning**: On AutoRound int4 lineage, generation of the 48% `Fidelity` tier is skipped by default, focusing compute resources on optimal `Quality` (39%), `Compact` (33%), `Mini` (27%), and `Nano` (21%) variants.
56
 
57
  ---
58
 
README.md CHANGED
@@ -64,7 +64,7 @@ All tiers maintain strict byte-budget percentages relative to the original unqua
64
  | :--- | :---: | :---: | :--- | :--- |
65
  | **Nano** | **21%** | `IQ3_XXS` | Maximum compression, edge & mobile VRAM | Core gates `Q6_K`, Down-proj `IQ2_S` |
66
  | **Mini** | **27%** | `IQ4_XS` | Efficient high-throughput serving | Balanced `IQ4_XS`/`IQ3_S` distribution |
67
- | **Compact** | **30%** | `IQ4_XS` | Balanced daily-driver footprint | Down-proj `IQ4_XS`, Gate/Up `IQ3_S` |
68
  | **Quality** | **39%** | `Q5_K_M` | Near-lossless general deployment | Full `Q4_K`/`Q5_K` attention coverage |
69
  | **Fidelity** | **48%** | `Q6_K` | Maximum analytical precision (raw BF16 lineage) | High-precision `Q5_K`/`Q6_K`/`Q8_0` mix |
70
 
@@ -75,7 +75,7 @@ All tiers maintain strict byte-budget percentages relative to the original unqua
75
  Smaller parameter architectures require higher relative bit precision to prevent degradation of core reasoning representations:
76
 
77
  * **≥ 9B Parameters**: **Mini** (27% ratio) — Large parameter capacity preserves semantic integrity at lower bit rates.
78
- * **~ 4B Parameters**: **Compact** (30% ratio) — Optimal balance between memory footprint and dense layer preservation.
79
  * **~ 3B Parameters**: **Quality** (39% ratio) — Higher baseline precision protects critical routing and attention projections.
80
  * **≤ 1B Parameters**: **Fidelity** (48% ratio) — Compact architectures require maximum parameter density.
81
 
 
64
  | :--- | :---: | :---: | :--- | :--- |
65
  | **Nano** | **21%** | `IQ3_XXS` | Maximum compression, edge & mobile VRAM | Core gates `Q6_K`, Down-proj `IQ2_S` |
66
  | **Mini** | **27%** | `IQ4_XS` | Efficient high-throughput serving | Balanced `IQ4_XS`/`IQ3_S` distribution |
67
+ | **Compact** | **33%** | `IQ4_XS` | Balanced daily-driver footprint | Down-proj `Q4_K`, Gate/Up `IQ4_XS` |
68
  | **Quality** | **39%** | `Q5_K_M` | Near-lossless general deployment | Full `Q4_K`/`Q5_K` attention coverage |
69
  | **Fidelity** | **48%** | `Q6_K` | Maximum analytical precision (raw BF16 lineage) | High-precision `Q5_K`/`Q6_K`/`Q8_0` mix |
70
 
 
75
  Smaller parameter architectures require higher relative bit precision to prevent degradation of core reasoning representations:
76
 
77
  * **≥ 9B Parameters**: **Mini** (27% ratio) — Large parameter capacity preserves semantic integrity at lower bit rates.
78
+ * **~ 4B Parameters**: **Compact** (33% ratio) — Optimal balance between memory footprint and dense layer preservation.
79
  * **~ 3B Parameters**: **Quality** (39% ratio) — Higher baseline precision protects critical routing and attention projections.
80
  * **≤ 1B Parameters**: **Fidelity** (48% ratio) — Compact architectures require maximum parameter density.
81
 
USAGE.md CHANGED
@@ -93,7 +93,7 @@ python ASHQ1.py --model model-BF16.gguf --imatrix imatrix.gguf --tier quality --
93
  Smaller parameter architectures require higher relative bit precision to prevent degradation of core reasoning representations:
94
 
95
  * **≥ 9B Models**: Select **Mini** (27% ratio) or higher. Large parameter capacity preserves semantic integrity at lower bit rates.
96
- * **~ 4B Models**: Select **Compact** (30% ratio) or higher. Optimal balance between memory footprint and dense layer preservation.
97
  * **~ 3B Models**: Select **Quality** (39% ratio) or higher. Higher baseline precision protects critical routing and attention projections.
98
  * **≤ 1B Models**: Select **Fidelity** (48% ratio) or higher. Compact architectures require maximum parameter density.
99
 
 
93
  Smaller parameter architectures require higher relative bit precision to prevent degradation of core reasoning representations:
94
 
95
  * **≥ 9B Models**: Select **Mini** (27% ratio) or higher. Large parameter capacity preserves semantic integrity at lower bit rates.
96
+ * **~ 4B Models**: Select **Compact** (33% ratio) or higher. Optimal balance between memory footprint and dense layer preservation.
97
  * **~ 3B Models**: Select **Quality** (39% ratio) or higher. Higher baseline precision protects critical routing and attention projections.
98
  * **≤ 1B Models**: Select **Fidelity** (48% ratio) or higher. Compact architectures require maximum parameter density.
99