Soulfate24 commited on
Commit
c2d9522
·
verified ·
1 Parent(s): 9d4e5a7

## [1.2.2] — 2026-08-20

Browse files

# Changelog

All notable changes to the AutoRound + ASHQ1 suite are documented in this file.
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

---

## [1.2.2] — 2026-08-20

### Fixed

- **Overhead self-calibration is now keyed per tier.** The correction factor lived
under the model filename alone, so an IQ-dominant profile and a K-quant profile
overwrote each other's value on every batch run. Observed on a 9B `qwen35` source:
`Nano` settled at ×1.0018 while `Mini`, `Compact` and `Quality` settled at ×1.0000,
leaving the cache oscillating instead of converging. Cache entries move to a
`filename|profile` key; stale entries simply fall back to ×1.0 and recalibrate on
the next dry run.

- **Upgrade queue now drains completely.** The greedy scheduler stopped at the first
candidate exceeding the remaining budget, abandoning every smaller move still
queued behind it. Because `current_size` grows monotonically, a rejected group
never fits later, so the oversized candidate is skipped and the queue keeps
draining. Measured on a 9B `qwen35` AutoRound int4 source (17 091 MiB BF16):
`Quality` recovered 127 MiB (5 939 → 6 066 MiB, 34.75% → 35.49%) by upgrading
`output.weight` to `Q5_K`, and `Compact` recovered 18 MiB (5 632 → 5 650 MiB,
32.95% → 33.06%) by upgrading `blk.0.ffn_down` and `blk.1.ssm_out`.
- **Tier ratio in the final summary.** The summary table printed
`bf16_mib * ALL_RATIOS[tier]`, showing `Quality` at 39% (6 665 MiB) on int4 lineage
while the run had correctly targeted 36% (6 153 MiB). The summary now routes
through `resolve_ratio()` like the execution loop.

### Verified

- Four-tier batch on a 17 091 MiB BF16 source: `Nano` 21.10%, `Mini` 27.01%,
`Compact` 33.06%, `Quality` 35.49%.
- `Quality` holds 85 MiB of slack against its 36% target: every capped class is
saturated and the surviving moves cost 129 MiB (`output` Q5_K→Q6_K) and 235 MiB
(`token_embd` Q6_K→Q8_0). Reachable footprints on this source are discrete at
35.49%, 36.25% and 37.62%.
- `META_OVERHEAD_MIB=12` matches the +10 to +12 MiB observed between the reported
quant size and the file on disk across all four tiers.
- `-Quality-36pc.gguf` naming is emitted and re-detected correctly across runs.

Files changed (3) hide show
  1. ASHQ1.py +9 -6
  2. CHARTER.md +2 -1
  3. README.md +2 -0
ASHQ1.py CHANGED
@@ -52,17 +52,20 @@ OVERHEAD_CACHE="ashq1-overhead.json"
52
  def _overhead_cache_path(model_path:str)->str:
53
  return os.path.join(os.path.dirname(os.path.abspath(model_path)),OVERHEAD_CACHE)
54
 
55
- def load_overhead_factor(model_path:str)->float:
 
 
 
56
  p=_overhead_cache_path(model_path)
57
  if os.path.isfile(p):
58
  try:
59
  with open(p,"r",encoding="utf-8") as f:
60
- v=float(json.load(f).get(os.path.basename(model_path),1.0))
61
  if 0.9<=v<=1.1:return v
62
  except Exception:pass
63
  return 1.0
64
 
65
- def save_overhead_factor(model_path:str,factor:float):
66
  if not(0.9<=factor<=1.1):return
67
  p=_overhead_cache_path(model_path)
68
  data={}
@@ -70,7 +73,7 @@ def save_overhead_factor(model_path:str,factor:float):
70
  try:
71
  with open(p,"r",encoding="utf-8") as f:data=json.load(f)
72
  except Exception:data={}
73
- data[os.path.basename(model_path)]=round(factor,4)
74
  try:
75
  with open(p,"w",encoding="utf-8") as f:json.dump(data,f,indent=2)
76
  except Exception as e:
@@ -1265,7 +1268,7 @@ def run_main_cli(args_list=None)->int:
1265
  int4_lineage=int4_lineage_of(args.model)
1266
  if int4_lineage:
1267
  print(f" Lineage: int4 AutoRound source — weight upgrades capped at {INT4_LINEAGE_CAP}, MTP at {INT4_MTP_CAP}")
1268
- _ov=load_overhead_factor(args.model)
1269
  if abs(_ov-1.0)>1e-4:
1270
  apply_overhead_factor(_ov)
1271
  print(f" Calibrated overhead (previous run): ×{_ov:.4f}")
@@ -1342,7 +1345,7 @@ def run_main_cli(args_list=None)->int:
1342
  dry_size=run_dry_run(flags,args.model)
1343
  if dry_size and estimated>0:
1344
  ratio=dry_size/estimated
1345
- save_overhead_factor(args.model,_ov*ratio)
1346
  if abs(ratio-1.0)>0.02:
1347
  print(f" Calibration: actual/estimated overhead ×{ratio:.4f} — saved for subsequent runs")
1348
  _show_size_result(dry_size,target_mib+free_mib)
 
52
  def _overhead_cache_path(model_path:str)->str:
53
  return os.path.join(os.path.dirname(os.path.abspath(model_path)),OVERHEAD_CACHE)
54
 
55
+ def _overhead_key(model_path:str,profile:str)->str:
56
+ return f"{os.path.basename(model_path)}|{profile}"
57
+
58
+ def load_overhead_factor(model_path:str,profile:str="quality")->float:
59
  p=_overhead_cache_path(model_path)
60
  if os.path.isfile(p):
61
  try:
62
  with open(p,"r",encoding="utf-8") as f:
63
+ v=float(json.load(f).get(_overhead_key(model_path,profile),1.0))
64
  if 0.9<=v<=1.1:return v
65
  except Exception:pass
66
  return 1.0
67
 
68
+ def save_overhead_factor(model_path:str,factor:float,profile:str="quality"):
69
  if not(0.9<=factor<=1.1):return
70
  p=_overhead_cache_path(model_path)
71
  data={}
 
73
  try:
74
  with open(p,"r",encoding="utf-8") as f:data=json.load(f)
75
  except Exception:data={}
76
+ data[_overhead_key(model_path,profile)]=round(factor,4)
77
  try:
78
  with open(p,"w",encoding="utf-8") as f:json.dump(data,f,indent=2)
79
  except Exception as e:
 
1268
  int4_lineage=int4_lineage_of(args.model)
1269
  if int4_lineage:
1270
  print(f" Lineage: int4 AutoRound source — weight upgrades capped at {INT4_LINEAGE_CAP}, MTP at {INT4_MTP_CAP}")
1271
+ _ov=load_overhead_factor(args.model,args.profile)
1272
  if abs(_ov-1.0)>1e-4:
1273
  apply_overhead_factor(_ov)
1274
  print(f" Calibrated overhead (previous run): ×{_ov:.4f}")
 
1345
  dry_size=run_dry_run(flags,args.model)
1346
  if dry_size and estimated>0:
1347
  ratio=dry_size/estimated
1348
+ save_overhead_factor(args.model,_ov*ratio,args.profile)
1349
  if abs(ratio-1.0)>0.02:
1350
  print(f" Calibration: actual/estimated overhead ×{ratio:.4f} — saved for subsequent runs")
1351
  _show_size_result(dry_size,target_mib+free_mib)
CHARTER.md CHANGED
@@ -53,7 +53,8 @@ When weights are derived from an AutoRound W4A16 source (recorded via `.provenan
53
  * **Information Saturation**: Because the underlying weights have been pre-conditioned for 4-bit representations, assigning container formats above `Q5_K` yields negligible signal recovery ($\Delta \mathcal{Q} \to 0$).
54
  * **Automated Capping**: The ASHQ1 optimizer caps attention and FFN blocks at `Q5_K`, attention gates at `Q6_K`, and MTP heads at `Q6_K`, while recurrent memory states hold `Q8_0`. The bit-budget freed by these ceilings flows to the imatrix-ranked tensors that convert it into measurable error reduction.
55
  * **Tier Pruning**: On AutoRound int4 lineage, generation of the 48% `Fidelity` tier is skipped by default, focusing compute resources on optimal `Quality` (36%), `Compact` (33%), `Mini` (27%), and `Nano` (21%) variants.
56
- * **Cap Saturation**: On AutoRound int4 lineage, `Compact` (33%) drives every attention and FFN projection to the `Q5_K` cap. Measured on a 9B `qwen35` source: `Compact` delivers 32.95% of the BF16 footprint with 8 MiB of slack, while `Quality` at a 39% target exhausts its upgrade queue and ships 36.87%. `Quality` therefore retargets to 36% on this lineage; other lineages keep 39%.
 
57
 
58
  ---
59
 
 
53
  * **Information Saturation**: Because the underlying weights have been pre-conditioned for 4-bit representations, assigning container formats above `Q5_K` yields negligible signal recovery ($\Delta \mathcal{Q} \to 0$).
54
  * **Automated Capping**: The ASHQ1 optimizer caps attention and FFN blocks at `Q5_K`, attention gates at `Q6_K`, and MTP heads at `Q6_K`, while recurrent memory states hold `Q8_0`. The bit-budget freed by these ceilings flows to the imatrix-ranked tensors that convert it into measurable error reduction.
55
  * **Tier Pruning**: On AutoRound int4 lineage, generation of the 48% `Fidelity` tier is skipped by default, focusing compute resources on optimal `Quality` (36%), `Compact` (33%), `Mini` (27%), and `Nano` (21%) variants.
56
+ * **Cap Saturation**: On AutoRound int4 lineage, `Compact` (33%) drives every attention and FFN projection to the `Q5_K` cap. Measured on a 9B `qwen35` source (17 091 MiB BF16): `Compact` ships 33.06%, `Quality` ships 35.49% against a 36% target. Beyond `Compact`, the budget buys `token_embd` (+280 MiB), `output` (+121 MiB) and the layer-0 FFN pair (+15 MiB); blocks 1 through 31 stay identical across both tiers. `Quality` therefore retargets to 36% on this lineage; other lineages keep 39%.
57
+ * **Discrete Ceiling**: Once every capped class saturates, the upgrade queue holds only large indivisible moves — `output` Q5_K→Q6_K at +129 MiB, `token_embd` Q6_K→Q8_0 at +235 MiB. Reachable footprints become discrete: on the reference 9B source, 35.49%, then 36.25%, then 37.62%. A target landing between two steps delivers the lower one, which the greedy scheduler reports as remaining slack.
58
 
59
  ---
60
 
README.md CHANGED
@@ -71,6 +71,8 @@ All tiers maintain strict byte-budget percentages relative to the original unqua
71
  *Note: Models originating from an AutoRound int4 lineage cap their weight allocations at `Q5_K`, as theoretical information saturation is fully realized. Attention gates settle at `Q6_K` and recurrent states at `Q8_0` on that lineage, and every tensor missing from the imatrix keeps `IQ4_XS` or above.*
72
 
73
  > **Int4 lineage tier ladder**: `Compact` (33%) already drives every attention and FFN projection to the `Q5_K` cap. Budget beyond that point buys `token_embd` and `output` precision only, so `Quality` retargets to **36%** on this lineage and writes `-Quality-36pc.gguf`. `Fidelity` stays pruned.
 
 
74
 
75
  ### 🎯 Recommended Minimum Tiers by Model Size
76
 
 
71
  *Note: Models originating from an AutoRound int4 lineage cap their weight allocations at `Q5_K`, as theoretical information saturation is fully realized. Attention gates settle at `Q6_K` and recurrent states at `Q8_0` on that lineage, and every tensor missing from the imatrix keeps `IQ4_XS` or above.*
72
 
73
  > **Int4 lineage tier ladder**: `Compact` (33%) already drives every attention and FFN projection to the `Q5_K` cap. Budget beyond that point buys `token_embd` and `output` precision only, so `Quality` retargets to **36%** on this lineage and writes `-Quality-36pc.gguf`. `Fidelity` stays pruned.
74
+ >
75
+ > Measured on a 9B `qwen35` source (17 091 MiB BF16): Nano **21.10%**, Mini **27.01%**, Compact **33.06%**, Quality **35.49%**. `Quality` stops short of its nominal target because the surviving upgrades are indivisible blocks of 129 MiB and above.
76
 
77
  ### 🎯 Recommended Minimum Tiers by Model Size
78