ASHQ1 v6: IQ tier spectrum, --allow-q3-or-lower, has_imatrix safety

- Full IQ tier range (IQ2_XXS through IQ4_XS) for aggressive spread
- --allow-q3-or-lower now uses IQ types instead of plain Q3_K
- has_imatrix check prevents crash on tensors without imatrix at low bitrates
- Depth factor removed (A/B test proved no PPL benefit)
- IQ4_NL MSE_BPW adjusted for free-upgrade pass
- Ornith-9B-MTP results added to README

.gitignore

@@ -0,0 +1,6 @@
+*.gguf
+*.gguf.hdr
+*.gguf.tmp
+__pycache__/
+*.pyc
+.env

README.md

@@ -0,0 +1,206 @@
+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+library_name: gguf
+tags:
+- quantization
+- gguf
+- llama-cpp
+- imatrix
+- hybrid-quantization
+- selective-quantization
+- priority-queue
+- mse
+- theoretical-optimization
+- qwen3.5
+- gemma4
+- moe
+- mtp
+---
+
+# ASHQ1 — Autonomous Selective Hybrid Quantization
+
+> ⚠️ **Experimental.** ASHQ1 is a personal research project that I will be refining over time. Use at your own risk. Results may vary between architectures and fine-tunes. Feedback and contributions welcome.
+
+ASHQ1 is a post-training quantization method for GGUF models that uses an **imatrix-driven priority queue** to maximise theoretical quality per megabyte. Instead of uniform bit-depth or heuristic layer-blocking, it treats tied tensor groups as monolithic entities and greedily upgrades them by strict mathematical utility — the product of summed importance and theoretical MSE reduction, divided by size cost.
+
+## Results
+
+| Target | Model | Arch | MTP | Actual | PPL (ctx=1024) | vs Q6_K |
+|:------:|-------|:----:|:---:|:------:|:---------------:|:-------:|
+| 5500 | Qwable-9B | Qwen3.5 | — | 5,503 MiB | 7.4334 | — |
+| 5700 | Qwythos-9B | Qwen3.5 | yes | 5,713 MiB | **7.5411** | **−0.047** |
+| 6000 | Ornith-1.0-9B | Qwen3.5 | yes | 6,015 MiB | **7.6774** | — |(1)
+| 6000 | Ornith-1.0-9B | Qwen3.5 | yes | 6,012 MiB | **7.4697** | — |(2)
+
+ASHQ1 at 5700 MiB beats uniform Q6_K by **0.047 PPL** at **19% smaller** (5713 vs 7076 MiB).
+
+> (1) Using `--allow-q3-or-lower` — aggressive IQ2→Q8_0 spread.  
+> (2) Default Q4_K floor — conservative distribution, best PPL at 6000 MiB.
+
+## Real-World Validation
+
+ASHQ1's theoretical quality advantage transfers to real agentic coding. We tested Ornith-1.0-9B ASHQ1 6500 (6.4 GB, 33% smaller than Q8_0) as the backend for [Pi](https://pi.dev/), an autonomous coding agent that uses `llama.cpp` as its LLM backend.
+
+At `temperature 0.6`, the model was tasked with building a complete personal finance dashboard as a single HTML file — Canvas charts, budget tracker, dark mode, transaction filtering, upcoming bills, responsive layout. The agent worked autonomously: planned the architecture, wrote the entire ~1100-line file, caught its own bugs (`date.now` → `date.getTime`), fixed dark mode logic, ran Node.js validation, and iterated until all checks passed. The final `finance-dashboard.html` was a polished, production-quality single-page app — no external dependencies, no hallucinations, no broken features.
+
+This is not cherry-picked. It's the first test we ran. The benchmarks didn't lie — ASHQ1 preserves enough quality that a 6.4 GB quant can drive an autonomous coding agent to build complete, working applications from scratch.
+
+## How It Works
+
+### 1. Floor Assignment
+
+Every tensor starts at a minimum tier by class. SSM params and norms lock at F16. Embeddings start at Q5_K. Weight matrices start at Q4_K (or IQ4_XS for QAT models). MTP heads deploy at Q8_0.
+
+With `--allow-q3-or-lower`, low-importance tensors (`ffn_down`, `attn_output`, `ssm_out`) start as low as IQ2_XXS, giving the priority queue more room to upgrade important tensors to Q8_0. Tensors missing imatrix data are kept at Q4_K to avoid garbage at low bitrates.
+
+### 2. Importance
+
+Imatrix `in_sum2` measures how much each weight contributes to the output variance. Layer position weighting was tested but showed no PPL benefit and has been removed.
+
+### 3. Tied Group Detection
+
+Tensors with numerically identical `in_sum2` arrays are tied (shared weights). They form a single upgrade group — all members upgrade together as one unit. Group importance is the **sum** of its members' importance, preventing large groups from being starved of budget.
+
+### 4. Priority Queue Drain
+
+All possible single-tier upgrades are pushed into a max-heap:
+
+```
+utility/MiB = sum(timp[group]) × (MSE(cur) − MSE(next)) / (size(next) − size(cur))
+```
+
+MSE per tier is theoretical: `MSE = 2^(-2 × bpw)`. K-quants get +0.1 effective bpw vs IQ-quants at the same real bpw, so IQ4_NL→Q4_K is a free quality gain.
+
+The queue pops the highest-utility upgrade, applies it, pushes the next upgrade for that group, and drains until the budget is exhausted. A final pass catches any remaining zero-cost upgrades.
+
+## Why It Works
+
+| Problem | ASHQ1 Solution |
+|---------|---------------|
+| Uniform quant wastes bits on low-importance tensors | Priority queue allocates budget where it matters |
+| Heuristic hand-tuning doesn't scale | Single knob: `--size` in MiB |
+| Hand-tuned SHQ hybrids need days of PPL sweeps | Queue converges in ~1 sec for any budget |
+| Large tied groups starved by per-tensor logic | `sum(timp)` prevents 32× group penalty |
+| IQ4_NL→Q4_K at same bpw is a no-op | Free-upgrade pass catches zero-cost quality gains |
+| No PPL-per-budget curve needed | Queue optimises for MSE directly |
+| Tensors without imatrix crash at low bitrates | `has_imatrix` check falls back to Q4_K floor |
+
+## Supported Architectures
+
+| Arch | Detection | Features |
+|------|-----------|----------|
+| `qwen35` | SSM + QKV | Hybrid attention, SSM layers, GQA, **MTP support** |
+| `mellum2` | MoE (`exps` tensors) | Mixture of Experts, GQA, router F16 |
+| `gemma4` | Layer-scale norms | QAT support, Q4_K attention floor |
+
+MTP (Multi-Token Prediction) heads are handled explicitly: MTP tensors deploy at Q8_0 and are excluded from the classifier's budget (their cost is subtracted from the target upfront). Tensor names with `nextn.*` or layers beyond `n_layers` are detected as MTP at runtime.
+
+### Looking for: Qwen3.6 support
+
+Qwen3.6 is one of the most capable local LLMs right now, but I can't handle it on my hardware. The BF16 source is ~55 GB — I don't have enough RAM to even load it, let alone quantize. If you have access to a Qwen3.6 GGUF (any quantization) and can run `llama-imatrix` on it — or if you'd like to collaborate on adding architecture detection — please reach out. I can handle the integration, I just need the raw tensor names and imatrix data to map out the class system.
+
+New architectures can be added via `ARCH_FEATURES` in `constants.py`.
+
+## Code Structure
+
+| File | Role |
+|------|------|
+| `main.py` | CLI entry point, orchestration, `--show-floors`, multiple `--imatrix` support |
+| `model_reader.py` | Reads GGUF, detects architecture/prefix/n_layers/MTP at runtime |
+| `imatrix_reader.py` | Parses imatrix GGUF, detects tied groups via `np.allclose(in_sum2)`, combines multiple imatrix |
+| `classifier.py` | Floor assignment → tied group building → priority queue drain → free upgrade pass |
+| `config_generator.py` | Generates `--tensor-type` regex rules from classified tensors (valid ECMAScript regex with pipe-alternated ranges) |
+| `quantizer.py` | Subprocess wrapper around `llama-quantize` |
+| `constants.py` | TENSOR_CLASS mapping, CLASS_HARD_FLOORS, CLASS_MAX_TIER, MSE_BPW, TIER_BPW, ARCH_FEATURES |
+
+## Usage
+
+### Quantization
+
+```bash
+pip install -r requirements.txt
+
+# Dry run (∼1 sec)
+python main.py --model model.gguf --imatrix imatrix.gguf --size 6800
+
+# Actual quant (∼10 min)
+python main.py --model model.gguf --imatrix imatrix.gguf --size 6800 --run
+
+# Show hard floors
+python main.py --show-floors
+
+# Multiple imatrix (combined with max/mean)
+python main.py --model model.gguf --imatrix i1.gguf --imatrix i2.gguf \
+  --imatrix-method max --size 6800 --run
+
+# Allow low-bit tensors (IQ2_XXS through Q8_0 spread)
+python main.py --model model.gguf --imatrix imatrix.gguf --size 6000 \
+  --allow-q3-or-lower --run
+```
+
+The `llama-quantize` binary path is set in `quantizer.py:6`.
+
+### Inference (llama-server)
+
+Recommended server flags for serving ASHQ1 quants:
+
+```bash
+./build/bin/llama-server \
+  -m model-ASHQ1.gguf \
+  -c 50000 \
+  --jinja \
+  -fit off \
+  -ngl 99 \
+  --flash-attn on \
+  --cache-type-k q8_0 \
+  --cache-type-v q8_0 \
+  --port 8080 \
+  --mmap \
+  --temp 1.0 \
+  --top-p 0.95 \
+  --min-p 0 \
+  --top-k 20 \
+  --seed -1 \
+  --parallel 1
+```
+
+## Tier Reference
+
+| Tier | BPW | MSE_BPW |
+|------|:---:|:-------:|
+| F16 | 16.0 | 16.0 |
+| Q8_0 | 8.50 | 8.50 |
+| Q6_K | 6.5625 | 6.5625 |
+| Q5_K | 5.50 | 5.50 |
+| Q4_K | 4.50 | 4.50 |
+| IQ4_NL | 4.50 | (2) |
+| IQ4_XS | 4.25 | 4.25 |
+| Q3_K | 3.4375 | 3.4375 |
+| IQ3_M | 3.66 | — |
+| IQ3_S | 3.44 | 3.44 |
+| IQ3_XXS | 3.0625 | 3.0625 |
+| IQ2_S | 2.50 | 2.50 |
+| IQ2_XS | 2.3125 | 2.3125 |
+| IQ2_XXS | 2.0625 | 2.0625 |
+| IQ1_S | 1.5625 | 1.5625 |
+
+> (2) IQ4_NL uses IQ4_XS MSE_BPW for the free-upgrade pass (same real bpw as Q4_K).
+
+## Quantization Configs
+
+Generated configs are valid `llama-quantize` arguments with ECMAScript-compatible regex patterns. Each `--tensor-type` rule matches a group of tensors that share the same target tier, with layers grouped into contiguous ranges:
+
+- `(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0` — specific attention layers at Q8_0
+- `(blk|BLK)\.((?:22|23|24|25|26))\.ffn_gate=Q6_K` — range of FFN layers at Q6_K
+- `.*output_norm.*=F16` — global catch-all
+
+Rules are sorted by specificity (specific layers, high tiers first) because `llama-quantize` uses first-match-wins.
+
+## References
+
+- [ASHQ1 repo](https://huggingface.co/wepiqx/ASHQ1)
+- [GGUF specification](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md)
+- [llama.cpp](https://github.com/ggerganov/llama.cpp)

classifier.py

@@ -0,0 +1,244 @@
+import heapq
+import warnings
+from functools import lru_cache
+from typing import Dict, List, Set, Tuple, Any
+from dataclasses import dataclass, field
+
+from constants import (
+    TIER_ORDER, TIER_BPW, GGUF_OVERHEAD_FACTOR, CLASS_MAX_TIER,
+    CAN_Q3, ALLOW_LOWER_FLOOR, MTP_DEPLOY_TIER, get_tensor_class, get_tensor_type,
+    is_mtp_tensor,
+)
+
+# Выносим делитель в константу (8 бит * 1024 байт * 1024 кбайт)
+BITS_IN_MIB = 8 * 1024 * 1024.0
+
+# Предвычисляем множители размеров для каждого тира (ускорение математики)
+TIER_SIZE_MULTIPLIER = {
+    tier: (bpw / BITS_IN_MIB) * GGUF_OVERHEAD_FACTOR 
+    for tier, bpw in TIER_BPW.items()
+}
+
+# K-quants используют блоки по 256 элементов и требуют выравнивания (padding) 3D-тензоров MoE
+K_QUANTS = {"Q3_K", "Q4_K", "Q5_K", "Q6_K"}
+MOE_PAD_TYPES = {"ffn_gate_exps", "ffn_up_exps", "ffn_down_exps", "ffn_down"}
+
+MSE_BPW = {
+    "IQ1_S": 1.5625, "IQ2_XXS": 2.0625, "IQ2_XS": 2.3125,
+    "IQ2_S": 2.5,
+    "IQ3_XXS": 3.0625,
+    "Q3_K": 3.4375, "IQ3_S": 3.44,
+    "IQ4_XS": 4.25, "IQ4_NL": 4.25, "Q4_K": 4.50,
+    "Q5_K": 5.50, "Q6_K": 6.5625, "Q8_0": 8.50, "F16": 16.0,
+}
+
+@dataclass(order=True, slots=True)
+class UpgradeItem:
+    """Элемент очереди апгрейдов. Сравнивается только по neg_utility."""
+    neg_utility: float
+    group_id: int = field(compare=False)
+    next_tier: str = field(compare=False)
+    cost_delta: float = field(compare=False)
+
+
+def _tier_index(tier: str) -> int:
+    if tier not in TIER_ORDER:
+        raise ValueError(f"Unknown tier: {tier}")
+    return TIER_ORDER.index(tier)
+
+def _tier_at(idx: int) -> str:
+    if not (0 <= idx < len(TIER_ORDER)):
+        raise IndexError(f"Tier index {idx} out of range")
+    return TIER_ORDER[idx]
+
+def _size_mib(tier: str, n_elements: int) -> float:
+    """Размер тензора в MiB с учётом оверхеда GGUF."""
+    if n_elements <= 0:
+        return 0.0
+    return n_elements * TIER_SIZE_MULTIPLIER.get(tier, 0.0)
+
+@lru_cache(maxsize=64)
+def _mse_delta(cur_tier: str, next_tier: str) -> float:
+    return (2 ** (-2 * MSE_BPW[cur_tier])) - (2 ** (-2 * MSE_BPW[next_tier]))
+
+def _push_upgrade(group_id: int,
+                  group_registry: Dict[int, Tuple[List[str], int, int]],
+                  assignments: Dict[str, str],
+                  tensor_importance: Dict[str, float],
+                  upgrade_queue: List[UpgradeItem],
+                  importance_table: Dict[str, Any]):
+    
+    # Храним 3 элемента: имена, реальный размер, размер с padding
+    g_names, g_elements, g_elements_padded = group_registry[group_id]
+    rep_name = g_names[0]
+    cur_tier = assignments[rep_name]
+    cur_idx = _tier_index(cur_tier)
+
+    rep_info = importance_table.get(rep_name, {})
+    ttype = rep_info["type"] if "type" in rep_info else get_tensor_type(rep_name)
+    cls = get_tensor_class(ttype)
+    max_tier = CLASS_MAX_TIER.get(cls, "Q8_0")
+
+    if cur_idx >= _tier_index(max_tier) or cur_idx >= len(TIER_ORDER) - 1:
+        return
+
+    next_tier = _tier_at(cur_idx + 1)
+    
+    # Выбираем размер в зависимости от того, относится ли тир к K-quants
+    cur_size_g = g_elements_padded if cur_tier in K_QUANTS else g_elements
+    next_size_g = g_elements_padded if next_tier in K_QUANTS else g_elements
+    
+    cost_delta = _size_mib(next_tier, next_size_g) - _size_mib(cur_tier, cur_size_g)
+
+    quality_delta = _mse_delta(cur_tier, next_tier)
+    if quality_delta <= 0:
+        return
+
+    if cost_delta == 0:
+        utility_per_mb = float('inf')
+    elif cost_delta < 0:
+        return 
+    else:
+        total_g_imp = sum(tensor_importance.get(n, 0) for n in g_names)
+        utility_per_mb = (total_g_imp * quality_delta) / cost_delta
+
+    heapq.heappush(upgrade_queue, UpgradeItem(-utility_per_mb, group_id, next_tier, cost_delta))
+
+
+def compute_initial_assignments(non_mtp_names: Set[str], mtp_names: Set[str], 
+                                importance_table: Dict, allow_q3: bool, is_qat: bool = False) -> Dict[str, str]:
+    assignments = {name: MTP_DEPLOY_TIER for name in mtp_names}
+
+    for name in non_mtp_names:
+        rep_info = importance_table.get(name, {})
+        has_imatrix = "importance_mean" in rep_info
+        ttype = rep_info["type"] if "type" in rep_info else get_tensor_type(name)
+        cls = get_tensor_class(ttype)
+        
+        if cls in ("norms", "ssm_params"):
+            assignments[name] = "F16"
+        elif cls == "embd":
+            assignments[name] = "Q4_K" if is_qat else "Q5_K"
+        else:
+            base_floor = ("Q4_K" if cls == "attn_proj" else "IQ4_XS") if is_qat else "Q4_K"
+            if allow_q3 and cls in CAN_Q3 and has_imatrix:
+                assignments[name] = ALLOW_LOWER_FLOOR
+            else:
+                assignments[name] = base_floor
+            
+    return assignments
+
+
+def build_groups(tied_groups: List[List[str]], non_mtp_names: Set[str], 
+                 ne_map: Dict[str, int], padded_ne_map: Dict[str, int]) -> Dict[int, Tuple[List[str], int, int]]:
+    group_registry = {}
+    assigned_tensors = set()
+    
+    for group_idx, group in enumerate(tied_groups):
+        clean_group = [n for n in group if n in non_mtp_names]
+        if clean_group:
+            g_elements = sum(ne_map.get(n, 0) for n in clean_group)
+            g_elements_padded = sum(padded_ne_map.get(n, 0) for n in clean_group)
+            group_registry[group_idx] = (clean_group, g_elements, g_elements_padded)
+            assigned_tensors.update(clean_group)
+
+    unassigned_tensors = non_mtp_names - assigned_tensors
+    next_group_idx = len(group_registry)
+    
+    for name in unassigned_tensors:
+        group_registry[next_group_idx] = ([name], ne_map.get(name, 0), padded_ne_map.get(name, 0))
+        next_group_idx += 1
+            
+    return group_registry
+
+
+def optimal_classify(importance_table: dict, tied_groups: list, model: dict, 
+                     target_size_mib: float, allow_q3: bool = False) -> Tuple[dict, dict]:
+    if target_size_mib <= 0:
+        raise ValueError("target_size_mib must be positive")
+
+    features = model.get("features", {})
+    has_mtp = features.get("has_mtp", False)
+    n_layers = features.get("n_layers", 31)
+    is_qat = features.get("is_qat", False)
+    model_tensors = model.get("tensors", {})
+
+    ne_map = {k: v["n_elements"] for k, v in model_tensors.items()}
+    for tname, info in importance_table.items():
+        if tname not in ne_map:
+            ne_map[tname] = info["n_elements"]
+
+    # --- Вычисление MoE Padding (Целочисленное выравнивание) ---
+    moe_d_ff = features.get("moe_intermediate_size", 0)
+    padded_ne_map = dict(ne_map)
+    if moe_d_ff > 0 and moe_d_ff % 256 != 0:
+        aligned_d_ff = ((moe_d_ff + 255) // 256) * 256
+        for name, n_el in ne_map.items():
+            ttype = importance_table.get(name, {}).get("type", get_tensor_type(name))
+            if ttype in MOE_PAD_TYPES:
+                padded_ne_map[name] = (n_el // moe_d_ff) * aligned_d_ff
+    # -----------------------------------------------------------
+
+    all_names = set(ne_map.keys())
+    mtp_names = {n for n in all_names if is_mtp_tensor(n, n_layers)} if has_mtp else set()
+    non_mtp_names = all_names - mtp_names
+
+    tensor_importance = {}
+    for name in non_mtp_names:
+        raw_imp = importance_table.get(name, {}).get("importance_mean", 0.0)
+        tensor_importance[name] = raw_imp
+
+    assignments = compute_initial_assignments(non_mtp_names, mtp_names, importance_table, allow_q3, is_qat)
+    group_registry = build_groups(tied_groups, non_mtp_names, ne_map, padded_ne_map)
+
+    mtp_cost = sum(_size_mib(MTP_DEPLOY_TIER, ne_map.get(n, 0)) for n in mtp_names)
+    effective_target = target_size_mib - mtp_cost
+    
+    current_size = sum(
+        _size_mib(assignments[n], padded_ne_map.get(n, ne_map.get(n, 0)) if assignments[n] in K_QUANTS else ne_map.get(n, 0))
+        for n in non_mtp_names
+    )
+
+    if current_size > effective_target:
+        warnings.warn(f"Initial size {current_size:.1f} MiB already exceeds target {effective_target:.1f} MiB", RuntimeWarning)
+
+    upgrade_queue = []
+    for g_id in group_registry:
+        _push_upgrade(g_id, group_registry, assignments, tensor_importance, upgrade_queue, importance_table)
+
+    while upgrade_queue:
+        item = heapq.heappop(upgrade_queue)
+        g_id, next_tier, cost_delta = item.group_id, item.next_tier, item.cost_delta
+        
+        if cost_delta > 0 and current_size + cost_delta > effective_target:
+            continue
+            
+        for n in group_registry[g_id][0]:
+            assignments[n] = next_tier
+        current_size += cost_delta
+        
+        _push_upgrade(g_id, group_registry, assignments, tensor_importance, upgrade_queue, importance_table)
+
+    return assignments, padded_ne_map
+
+
+def compute_stats(assignments: dict, ne_map: dict = None, padded_ne_map: dict = None) -> dict:
+    """Собирает статистику по тирам с точным учётом K_QUANTS padding."""
+    stats = {"by_tier_count": {}, "by_tier_mib": {}, "total_mib": 0.0, "tensor_count": 0}
+    for name, tier in assignments.items():
+        if not isinstance(tier, str):
+            continue
+        stats["tensor_count"] += 1
+        stats["by_tier_count"][tier] = stats["by_tier_count"].get(tier, 0) + 1
+        
+        if ne_map:
+            if padded_ne_map and tier in K_QUANTS:
+                elements = padded_ne_map.get(name, ne_map.get(name, 0))
+            else:
+                elements = ne_map.get(name, 0)
+                
+            size = _size_mib(tier, elements)
+            stats["by_tier_mib"][tier] = stats["by_tier_mib"].get(tier, 0.0) + size
+            stats["total_mib"] += size
+            
+    return stats

config_generator.py

@@ -0,0 +1,155 @@
+import re
+from constants import QUANT_RANK
+
+
+def get_regex_priority(regex: str) -> int:
+    """Higher = more specific = should come first (first-match-wins)."""
+    score = 0
+
+    if "nextn" in regex:
+        score += 200
+    if re.search(r"(blk|BLK)\\.3[0-2]\\.", regex):
+        score += 100
+    if re.search(r"(blk|BLK)\\.0\\.", regex):
+        score += 90
+    if re.search(r"(blk|BLK)\\.31\\.", regex):
+        score += 80
+    # Layer-range group: (blk|BLK)\.( — more specific than \d+
+    if r"(blk|BLK)\.(" in regex:
+        score += 50
+    elif r"(blk|BLK)\.\d" in regex:
+        score += 30
+    if regex.startswith(".*"):
+        score -= 50
+    if regex.endswith(r"\.weight"):
+        score += 10
+
+    return score
+
+
+def _is_contiguous(lst, low, high):
+    if not lst:
+        return False
+    return len(lst) == (high - low + 1)
+
+
+def _group_ranges(lst):
+    if not lst:
+        return
+    start = lst[0]
+    end = lst[0]
+    for i in range(1, len(lst)):
+        if lst[i] == end + 1:
+            end = lst[i]
+        else:
+            yield (start, end)
+            start = end = lst[i]
+    yield (start, end)
+
+
+def _range_to_regex(start: int, end: int) -> str:
+    """Convert a range of layer numbers [start, end] to a valid regex."""
+    if start == end:
+        return str(start)
+    # For single-digit ranges, use character class
+    if end <= 9:
+        return f"[{start}-{end}]"
+    # Enumerate all numbers as pipe alternatives
+    alt = "|".join(str(i) for i in range(start, end + 1))
+    return f"(?:{alt})"
+
+
+def generate_flags(
+    assignments: dict,
+    model: dict,
+    base_type: str,
+    target_size_mib: float = None,
+) -> dict:
+    is_qat = model.get("features", {}).get("is_qat", False)
+    output_type = "Q5_K"
+    token_embd_type = "Q4_K" if is_qat else "Q5_K"
+
+    max_layer = model.get("features", {}).get("n_layers", 31)
+
+    rules = []
+
+    # Group blk tensors by (ttype, tier)
+    type_tier_layers = {}
+    for tname, tier in assignments.items():
+        parts = tname.split(".")
+        if len(parts) >= 3 and parts[0] in ("blk", "BLK"):
+            try:
+                layer = int(parts[1])
+            except ValueError:
+                continue
+            ttype = parts[2]
+            key = (ttype, tier)
+            if key not in type_tier_layers:
+                type_tier_layers[key] = []
+            type_tier_layers[key].append(layer)
+
+    # Generate rules for blk tensor groups
+    for (ttype, tier), layers in sorted(
+        type_tier_layers.items(),
+        key=lambda x: -QUANT_RANK.get(x[0][1], 0),
+    ):
+        layers = sorted(set(layers))
+
+        if len(layers) >= 8 and _is_contiguous(layers, 0, max_layer):
+            pattern = f"(blk|BLK)\\.\\d+\\.{ttype}={tier}"
+        else:
+            parts = []
+            for start, end in _group_ranges(layers):
+                if start == end:
+                    parts.append(str(start))
+                else:
+                    parts.append(_range_to_regex(start, end))
+            desc = "|".join(parts)
+            pattern = f"(blk|BLK)\\.({desc})\\.{ttype}={tier}"
+
+        prio = get_regex_priority(pattern) + (
+            10 if tier == "Q8_0" else 5 if tier == "Q6_K" else 0
+        ) + (5 if len(layers) == 1 else 0) + (3 if "ffn_down" in ttype else 0)
+
+        rules.append((pattern, prio))
+
+    # Generate rules for global tensors (non-blk)
+    prefix = model.get("features", {}).get("prefix", "blk")
+    for tname, tier in assignments.items():
+        parts = tname.split(".")
+        if len(parts) >= 2 and parts[0].lower() == prefix.lower():
+            continue
+        ttype = parts[0] if len(parts) >= 1 else tname
+        if ttype in ("token_embd", "output"):
+            continue
+        if ttype == tname and "." in tname:
+            # e.g. "nextn.eh_proj" without blk prefix
+            pass
+        # Check that this global tensor wasn't already handled as a blk tensor
+        pattern = f".*{re.escape(ttype)}.*={tier}"
+        prio = get_regex_priority(pattern) + (5 if tier == "Q8_0" else 0)
+        # Deduplicate (same pattern may appear from different names)
+        if not any(p == pattern for p, _ in rules):
+            rules.append((pattern, prio))
+
+    rules.sort(key=lambda x: -x[1])
+
+    flags = {
+        "imatrix": None,
+        "output_tensor_type": output_type,
+        "token_embedding_type": token_embd_type,
+        "tensor_type_rules": [f'--tensor-type "{r[0]}"' for r in rules],
+        "base_type": base_type,
+        "target_size_mib": target_size_mib,
+    }
+
+    return flags
+
+
+def format_flags(flags: dict) -> str:
+    lines = []
+    lines.append("  --output-tensor-type " + flags["output_tensor_type"])
+    lines.append("  --token-embedding-type " + flags["token_embedding_type"])
+    for rule in flags["tensor_type_rules"]:
+        lines.append("  " + rule)
+    return "\n".join(lines)

constants.py

@@ -0,0 +1,222 @@
+import re
+
+GGUF_TYPE_NAMES = {
+    0: "F32",
+    1: "F16",
+    2: "Q4_0", 3: "Q4_1", 6: "Q5_0", 7: "Q5_1",
+    8: "Q8_0",
+    10: "Q4_K", 11: "Q5_K", 12: "Q6_K",
+    13: "Q5_K_M", 14: "Q4_K_M",
+    15: "IQ4_XS", 16: "IQ4_NL",
+    20: "IQ3_XXS",
+    24: "IQ2_XXS",
+    30: "IQ1_S",
+}
+
+GGUF_TYPE_NAMES_INV = {v: k for k, v in GGUF_TYPE_NAMES.items()}
+
+# Ordered from worst to best quality
+TIER_ORDER = [
+    "IQ1_S", "IQ2_XXS", "IQ2_XS", "IQ2_S",
+    "IQ3_XXS", "Q3_K", "IQ3_S",
+    "IQ4_XS", "IQ4_NL", "Q4_K", "Q5_K", "Q6_K", "Q8_0", "F16",
+]
+
+# Exact bits per weight from ggml block structs (ggml_type_sizef * 8)
+# Does NOT include GGUF overhead — GGUF_OVERHEAD_FACTOR is applied separately
+TIER_BPW = {
+    "IQ1_S": 1.5625,
+    "IQ2_XXS": 2.0625,
+    "IQ2_XS": 2.3125,
+    "IQ2_S": 2.5,
+    "IQ3_XXS": 3.0625,
+    "Q3_K": 3.4375,
+    "IQ3_S": 3.44,
+    "IQ4_XS": 4.25,
+    "IQ4_NL": 4.5,
+    "Q4_K": 4.5,
+    "Q5_K": 5.5,
+    "Q6_K": 6.5625,
+    "Q8_0": 8.5,
+    "F16": 16.0,
+}
+GGUF_OVERHEAD_FACTOR = 1.0
+
+# Quality retention per tier (1.0 = F16, no loss). Used for non-linear utility.
+QUALITY_WEIGHTS = {
+    "F16": 1.000,
+    "Q8_0": 0.995,
+    "Q6_K": 0.990,
+    "Q5_K": 0.978,
+    "IQ4_XS": 0.940,
+    "IQ4_NL": 0.945,
+    "Q4_K": 0.960,
+    "Q3_K": 0.900,
+}
+
+# Quant quality rank: higher = better (same order as TIER_ORDER)
+QUANT_RANK = {tier: i for i, tier in enumerate(TIER_ORDER)}
+
+# Per-class hard floor — NEVER go below this without --allow-q3-or-lower
+# Matches hand-tuned v2 patterns: gate=Q6_K, attn_proj=Q6_K, ffn=IQ4_XS
+CLASS_HARD_FLOORS = {
+    "gate": "Q8_0",
+    "attn_proj": "Q8_0",
+    "ffn_gate_up": "IQ4_XS",
+    "ffn_down": "Q6_K",
+    "norms": "F16",
+    "ssm_params": "F16",
+    "mtp": "IQ4_XS",
+    "embd": "Q5_K",
+}
+
+# Per-class start tier — where greedy begins (Q5_K for all, like OptA base type)
+# Greedy upgrades from here toward CLASS_MAX_TIER
+CLASS_START_TIER = {
+    "gate": "Q8_0",
+    "attn_proj": "Q8_0",
+    "ffn_gate_up": "Q5_K",
+    "ffn_down": "Q6_K",
+    "norms": "F16",
+    "ssm_params": "F16",
+    "mtp": "Q5_K",
+    "embd": "Q5_K",
+}
+
+# Per-class max tier — never exceed this (for greedy upgrades)
+# Deep layers can be upgraded to Q8_0 for important tensors
+CLASS_MAX_TIER = {
+    "gate": "Q8_0",
+    "attn_proj": "Q8_0",
+    "ffn_gate_up": "Q8_0",
+    "ffn_down": "Q8_0",
+    "norms": "F16",
+    "ssm_params": "F16",
+    "mtp": "Q8_0",
+    "embd": "Q5_K",
+}
+
+# Classes that can go to Q3_K when --allow-q3-or-lower is set
+CAN_Q3 = {"ffn_gate", "ffn_up", "ffn_down", "attn_output", "ssm_out"}
+ALLOW_LOWER_FLOOR = "IQ2_XXS"  # lowest starting tier for --allow-q3-or-lower
+
+# Default floor for unclassified tensors / unknown class
+DEFAULT_FLOOR = "Q4_K"
+
+# Tier for MTP head deployment
+MTP_DEPLOY_TIER = "Q8_0"
+
+# Preferred tier for extreme importance spikes (>50% of total importance)
+SPIKE_IMPORTANCE_RATIO = 0.50
+SPIKE_PREFERRED_TIER = "F16"
+
+TENSOR_CLASS = {
+    # Qwen 3.5 hybrid
+    "attn_gate": "gate",
+    "ssm_alpha": "gate",
+    "ssm_beta": "gate",
+    "attn_q": "attn_proj",
+    "attn_k": "attn_proj",
+    "attn_v": "attn_proj",
+    "attn_qkv": "attn_proj",
+    "attn_output": "attn_proj",
+    "ffn_gate": "ffn_gate_up",
+    "ffn_up": "ffn_gate_up",
+    "ffn_down": "ffn_down",
+    "ssm_out": "ffn_down",
+    "ssm_conv1d": "norms",
+    "router": "norms",
+    "ssm_dt": "ssm_params",
+    "ssm_a": "ssm_params",
+    "nextn": "mtp",
+    "ffn_gate_exps": "ffn_gate_up",
+    "ffn_up_exps": "ffn_gate_up",
+    "ffn_down_exps": "ffn_down",
+    "ffn_gate_inp": "norms",
+
+    # Standard llama.cpp tensor names
+    "q_proj": "attn_proj",
+    "k_proj": "attn_proj",
+    "v_proj": "attn_proj",
+    "o_proj": "attn_proj",
+    "gate_proj": "ffn_gate_up",
+    "up_proj": "ffn_gate_up",
+    "down_proj": "ffn_down",
+}
+
+ARCH_FEATURES = {
+    "qwen35": {
+        "has_qkv": True,
+        "has_ssm": True,
+        "has_mtp": True,
+        "has_moe": False,
+        "is_qat": False,
+        "prefix": "blk",
+        "n_layers": 32,
+    },
+    "mellum2": {
+        "has_qkv": False,
+        "has_ssm": False,
+        "has_mtp": False,
+        "has_moe": True,
+        "is_qat": False,
+        "prefix": "blk",
+        "n_layers": 28,
+    },
+    "gemma4": {
+        "has_qkv": False,
+        "has_ssm": False,
+        "has_mtp": False,
+        "has_moe": False,
+        "is_qat": True,
+        "prefix": "blk",
+        "n_layers": 48,
+    },
+}
+
+
+def strip_weight(name: str) -> str:
+    return name.lstrip(".").removesuffix(".weight").removesuffix(".bias")
+
+
+def get_tensor_type(name: str) -> str:
+    parts = strip_weight(name).split(".")
+    if len(parts) >= 2 and parts[0] in ("blk", "BLK"):
+        return parts[2] if len(parts) >= 3 else "unknown"
+    if "token_embd" in name:
+        return "token_embd"
+    if name.startswith("output") and "norm" not in name:
+        return "output"
+    return name
+
+
+def get_tensor_class(ttype: str) -> str:
+    if ttype in TENSOR_CLASS:
+        return TENSOR_CLASS[ttype]
+    if "norm" in ttype or "scale" in ttype:
+        return "norms"
+    if ttype.startswith("ssm_"):
+        return "ssm_params"
+    if ttype in ("token_embd", "output", "embed_tokens", "lm_head",
+                 "vision_embedder", "audio_embedder"):
+        return "embd"
+    return "unknown"
+
+
+def is_mtp_tensor(name: str, n_layers: int = 32) -> bool:
+    if "nextn" in name:
+        return True
+    if n_layers > 40:
+        return False  # Deep models (Gemma4, 48 layers) use separate drafter, not in-model MTP
+    layer = get_layer_number(name)
+    return layer is not None and layer >= n_layers
+
+
+def get_layer_number(name: str) -> int | None:
+    parts = strip_weight(name).split(".")
+    if len(parts) >= 2 and parts[0] in ("blk", "BLK"):
+        try:
+            return int(parts[1])
+        except ValueError:
+            return None
+    return None

imatrix_reader.py

@@ -0,0 +1,179 @@
+import gguf
+import numpy as np
+from typing import List
+
+
+def read_imatrix(path: str) -> dict:
+    """Parse imatrix GGUF, return per-tensor importance data."""
+    r = gguf.GGUFReader(path)
+    raw = {}
+    meta = {}
+
+    for k, v in r.fields.items():
+        try:
+            meta[k] = v.data
+        except:
+            meta[k] = str(v)
+
+    for t in r.tensors:
+        name = t.name
+        arr = np.array(t.data, dtype=np.float64)
+
+        if name.endswith(".in_sum2"):
+            base = name[:-8]
+            if base not in raw:
+                raw[base] = {}
+            raw[base]["in_sum2"] = arr
+        elif name.endswith(".counts"):
+            base = name[:-7]
+            if base not in raw:
+                raw[base] = {}
+            raw[base]["counts"] = float(np.mean(arr))
+
+    result = {}
+    for base, data in raw.items():
+        if "in_sum2" not in data:
+            continue
+        arr = data["in_sum2"]
+        result[base] = {
+            "importance_mean": float(np.mean(arr)),
+            "importance_sum": float(np.sum(arr)),
+            "importance_max": float(np.max(arr)),
+            "importance_min": float(np.min(arr)),
+            "n_elements": arr.size,
+            "in_sum2_raw": arr,
+        }
+
+    return {
+        "path": path,
+        "tensors": result,
+        "n_tensors": len(result),
+        "meta": meta,
+    }
+
+
+def combine_imatrix(imatrix_list: List[dict], method: str = "max") -> dict:
+    """Combine multiple imatrix into one by aggregating importance.
+    
+    Args:
+        imatrix_list: List of imatrix dicts from read_imatrix()
+        method: "max", "mean", or "weighted_mean"
+    """
+    if not imatrix_list:
+        return {}
+    if len(imatrix_list) == 1:
+        return imatrix_list[0]
+    
+    # Get all tensor names across all imatrix
+    all_names = set()
+    for im in imatrix_list:
+        all_names.update(im["tensors"].keys())
+    
+    combined_tensors = {}
+    for name in all_names:
+        vals = []
+        in_sum2_raw = None
+        n_elements = 0
+        
+        for im in imatrix_list:
+            if name in im["tensors"]:
+                t = im["tensors"][name]
+                vals.append(t["importance_mean"])
+                if in_sum2_raw is None and "in_sum2_raw" in t:
+                    in_sum2_raw = t["in_sum2_raw"]
+                n_elements = max(n_elements, t["n_elements"])
+        
+        if not vals:
+            continue
+        
+        if method == "max":
+            imp_mean = max(vals)
+        elif method == "mean":
+            imp_mean = sum(vals) / len(vals)
+        elif method == "weighted_mean":
+            # Could weight by n_elements or dataset size
+            imp_mean = sum(vals) / len(vals)
+        else:
+            imp_mean = max(vals)
+        
+        combined_tensors[name] = {
+            "importance_mean": imp_mean,
+            "importance_sum": imp_mean * n_elements,
+            "importance_max": max(v.get("importance_max", 0) for im in imatrix_list if name in im["tensors"] for v in [im["tensors"][name]]),
+            "importance_min": min(v.get("importance_min", float('inf')) for im in imatrix_list if name in im["tensors"] for v in [im["tensors"][name]]),
+            "n_elements": n_elements,
+            "in_sum2_raw": in_sum2_raw,
+        }
+    
+    # Merge metadata
+    combined_meta = {}
+    for im in imatrix_list:
+        for k, v in im["meta"].items():
+            if k not in combined_meta:
+                combined_meta[k] = v
+    
+    return {
+        "path": "+".join(im["path"] for im in imatrix_list),
+        "tensors": combined_tensors,
+        "n_tensors": len(combined_tensors),
+        "meta": combined_meta,
+    }
+
+
+def detect_tied_groups(imatrix: dict, atol: float = 1e-5) -> list:
+    """Find tied tensor groups (identical importance arrays)."""
+    names = sorted(imatrix["tensors"].keys())
+    tied_groups = []
+    visited = set()
+
+    for i, n1 in enumerate(names):
+        if n1 in visited:
+            continue
+        group = [n1]
+        arr1 = imatrix["tensors"][n1].get("in_sum2_raw")
+        if arr1 is None:
+            tied_groups.append(group)
+            visited.add(n1)
+            continue
+        for j in range(i + 1, len(names)):
+            n2 = names[j]
+            if n2 in visited:
+                continue
+            arr2 = imatrix["tensors"][n2].get("in_sum2_raw")
+            if arr2 is None:
+                continue
+            if arr1.shape == arr2.shape and np.allclose(arr1, arr2, atol=atol):
+                group.append(n2)
+                visited.add(n2)
+        tied_groups.append(group)
+        visited.add(n1)
+
+    return tied_groups
+
+
+def build_importance_table(imatrix: dict, model: dict) -> dict:
+    """Build unified importance table, merging imatrix with model tensor info."""
+    table = {}
+    for tname, info in imatrix["tensors"].items():
+        ttype = _imatrix_type(tname)
+        table[tname] = {
+            "importance_mean": info["importance_mean"],
+            "importance_sum": info["importance_sum"],
+            "importance_max": info["importance_max"],
+            "importance_min": info["importance_min"],
+            "n_elements": info["n_elements"],
+            "type": ttype,
+        }
+    # Also index by name without trailing dot (safety for old code)
+    for tname, info in list(table.items()):
+        if tname.endswith("."):
+            alt = tname.rstrip(".")
+            table[alt] = info
+    return table
+
+
+def _imatrix_type(name: str) -> str:
+    parts = name.split(".")
+    if len(parts) >= 3 and parts[0] == "blk":
+        return parts[2]
+    return name

main.py

@@ -0,0 +1,185 @@
+#!/usr/bin/env python3
+
+import argparse
+import json
+import os
+import sys
+
+from model_reader import read_model
+from imatrix_reader import read_imatrix, detect_tied_groups, build_importance_table
+from classifier import optimal_classify, compute_stats
+from config_generator import generate_flags, format_flags
+from quantizer import run_dry_run, run_quantization
+from constants import CLASS_HARD_FLOORS
+
+
+def _get_base_type(model: dict) -> str:
+    is_qat = model.get("features", {}).get("is_qat", False)
+    return "IQ4_XS" if is_qat else "Q5_K_M"
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="SHQ-program: imatrix-driven hybrid quantization"
+    )
+    parser.add_argument("--model", help="BF16 GGUF model path")
+    parser.add_argument("--imatrix", action="append", default=[],
+                        help="Imatrix GGUF path (can be specified multiple times)")
+    parser.add_argument("--imatrix-method", choices=["max", "mean"], default="max",
+                        help="How to combine multiple imatrix: max (conservative) or mean (default: max)")
+    parser.add_argument("--size", type=float, default=6800,
+                        help="Target file size in MiB (default: 6800 = ~6.6 GB)")
+    parser.add_argument("--output", default=None, help="Output GGUF path")
+    parser.add_argument("--run", action="store_true", help="Execute quantization")
+    parser.add_argument("--show-config", action="store_true", help="Print config and exit")
+    parser.add_argument("--verbose", action="store_true", help="Detailed output")
+    parser.add_argument("--allow-q3-or-lower", action="store_true",
+                        help="Allow Q3_K for low-importance tensors (risk of quality loss)")
+    parser.add_argument("--aggro", type=float, default=None,
+                        help="[deprecated] Use --size instead")
+    parser.add_argument("--show-floors", action="store_true",
+                        help="Print class hard floors and exit")
+
+    args = parser.parse_args()
+
+    if args.show_floors:
+        _show_floors()
+        return
+
+    if not args.model or not args.imatrix:
+        parser.print_usage()
+        print("main.py: error: --model and --imatrix are required")
+        sys.exit(1)
+
+    target_mib = args.size
+
+    print("=== SHQ-program ===")
+    print(f"Model:   {args.model}")
+    if len(args.imatrix) == 1:
+        print(f"Imatrix: {args.imatrix[0]}")
+    else:
+        print(f"Imatrix: {len(args.imatrix)} files ({args.imatrix_method})")
+        for p in args.imatrix:
+            print(f"  - {p}")
+    print(f"Target:  {target_mib:.0f} MiB ({target_mib / 1024:.2f} GB)")
+    if args.allow_q3_or_lower:
+        print("  --allow-q3-or-lower: low-importance tensors may go to Q3_K")
+    print()
+
+    print("[1/4] Reading model...")
+    model = read_model(args.model)
+    print(f"  Architecture: {model['architecture']}")
+    print(f"  Tensors: {model['n_tensors']}")
+    print(f"  Features: {json.dumps(model['features'], indent=2)}")
+
+    print("\n[2/4] Reading imatrix...")
+    imatrix_list = [read_imatrix(p) for p in args.imatrix]
+    for im in imatrix_list:
+        print(f"  {im['path']}: {im['n_tensors']} tensors, datasets={im['meta'].get('imatrix.datasets', '?')}")
+
+    from imatrix_reader import combine_imatrix
+    imatrix = combine_imatrix(imatrix_list, method=args.imatrix_method)
+    print(f"  Combined: {imatrix['n_tensors']} tensors")
+
+    print("\n[3/4] Detecting tied groups...")
+    tied_groups = detect_tied_groups(imatrix)
+    print(f"  Found {len(tied_groups)} tied groups:")
+    for g in tied_groups:
+        if len(g) > 1:
+            print(f"    TIED ({len(g)}): {g[0].replace('.weight', '')}  =  "
+                  f"{g[1].replace('.weight', '')}")
+
+    imp_table = build_importance_table(imatrix, model)
+
+    print("\n[4/4] Classifying tensors (greedy imatrix-driven)...")
+
+    # Получаем и маппинг тиров, и точную карту паддингов напрямую из классификатора
+    assignments, padded_ne_map = optimal_classify(
+        imp_table, tied_groups, model,
+        target_size_mib=target_mib,
+        allow_q3=args.allow_q3_or_lower,
+    )
+    
+    ne_map = {k: v["n_elements"] for k, v in model.get("tensors", {}).items()}
+    for tname, info in imp_table.items():
+        if tname not in ne_map:
+            ne_map[tname] = info["n_elements"]
+
+    # Передаем padded_ne_map для корректного вывода логов на экран
+    _show_tier_summary(assignments, imp_table, ne_map, padded_ne_map)
+
+    base_type = _get_base_type(model)
+    flags = generate_flags(assignments, model, base_type, target_mib)
+    flags["imatrix"] = args.imatrix
+
+    print(f"\nConfig (base={flags['base_type']}):")
+    print(format_flags(flags))
+
+    if args.show_config:
+        return
+
+    print("\n--- Dry Run ---")
+    dry_size = run_dry_run(flags, args.model)
+    _show_size_result(dry_size, target_mib)
+
+    if not args.run:
+        print("\nDry run only. Use --run to execute quantization.")
+        return
+
+    if not args.output:
+        base = os.path.splitext(os.path.basename(args.model))[0]
+        args.output = base + "-SHQ.gguf"
+
+    print(f"\n--- Running quantization: {args.output} ---")
+    success = run_quantization(flags, args.model, args.output)
+    if success:
+        print("Done!")
+    else:
+        print("Failed!")
+        sys.exit(1)
+
+
+def _show_tier_summary(assignments, imp_table, ne_map, padded_ne_map=None):
+    stats = compute_stats(assignments, ne_map, padded_ne_map)
+    
+    print("\n  Tier distribution:")
+    for tier in sorted(stats["by_tier_count"].keys()):
+        count = stats["by_tier_count"][tier]
+        mib = stats["by_tier_mib"].get(tier, 0.0)
+        print(f"    {tier}: {count} tensors ({mib:.1f} MiB)")
+    print(f"  Total estimated size: {stats['total_mib']:.1f} MiB")
+
+    ranked = sorted(
+        [(n, v) for n, v in imp_table.items()],
+        key=lambda x: -x[1]["importance_mean"],
+    )
+    print("\n  Top 10 by importance:")
+    for n, v in ranked[:10]:
+        tier = assignments.get(n, "base")
+        display = n.replace(".weight", "").replace(".bias", "")
+        print(f"    {display[:52]:52s} imp={v['importance_mean']:10.0f} tier={tier}")
+
+
+def _show_size_result(dry_size, target_mib):
+    if dry_size:
+        print(f"  Estimated size: {dry_size:.0f} MiB ({dry_size / 1024:.2f} GB)")
+        diff = dry_size - target_mib
+        if diff > 0:
+            print(f"  ⚠  Over target by {diff:.0f} MiB")
+        else:
+            print(f"  ✓ Under target by {-diff:.0f} MiB")
+    else:
+        print("  ⚠  Could not parse size from dry-run output")
+
+
+def _show_floors():
+    print("  Class hard floors (never below without --allow-q3-or-lower):\n")
+    max_n = max(len(c) for c in CLASS_HARD_FLOORS)
+    for cls, floor in sorted(CLASS_HARD_FLOORS.items()):
+        print(f"    {cls:<{max_n}}  →  {floor}")
+    print(f"\n  Default floor (unknown class): Q4_K")
+    print(f"  --allow-q3-or-lower enables Q3_K for: ffn_down, attn_output, ssm_out")
+
+
+if __name__ == "__main__":
+    main()

model_reader.py

@@ -0,0 +1,110 @@
+import gguf
+import numpy as np
+from constants import ARCH_FEATURES
+
+
+def detect_architecture(tensors: dict) -> str:
+    """Detect model architecture from tensor names."""
+    names = list(tensors.keys())
+
+    has_ssm = any("ssm_" in n for n in names)
+    has_qkv = any("attn_qkv" in n for n in names)
+    has_nextn = any("nextn" in n for n in names)
+    has_moe = any("exps" in n for n in names)
+    has_separate_qkv = any("attn_q.weight" in n for n in names)
+    has_gemma_specific = any(
+        t in n for t in ("layer_output_scale", "post_attention_norm", "post_ffw_norm")
+        for n in names
+    )
+
+    if has_ssm and has_qkv:
+        return "qwen35"
+    if has_moe:
+        return "mellum2"
+    # Исправление: заменяем неопределённую has_blk_attn на уже существующий признак
+    if has_separate_qkv or has_gemma_specific:
+        return "gemma4"
+    return "unknown"
+
+
+def _detect_prefix(tensors: dict) -> str:
+    for name in tensors:
+        if name.startswith("BLK."):
+            return "BLK"
+    return "blk"
+
+
+def _estimate_layers(tensors: dict) -> int:
+    max_layer = 0
+    for name in tensors:
+        parts = name.split(".")
+        if len(parts) >= 2 and parts[0] in ("blk", "BLK"):
+            try:
+                layer = int(parts[1])
+                if layer > max_layer:
+                    max_layer = layer
+            except ValueError:
+                pass
+    return max_layer + 1  # layers are 0-indexed
+
+
+def read_model(path: str) -> dict:
+    """Parse BF16 GGUF, return model info."""
+    r = gguf.GGUFReader(path)
+    tensors = {}
+    meta = {}
+
+    for k, v in r.fields.items():
+        # Безопасное извлечение данных (список/число, а не raw numpy)
+        try:
+            data = v.data
+            if isinstance(data, np.ndarray):
+                data = data.tolist()
+            elif isinstance(data, (np.generic,)):
+                data = data.item()
+            meta[k] = data
+        except Exception:
+            meta[k] = str(v)
+
+    for t in r.tensors:
+        shape = list(t.shape)
+        name = t.name
+        n_elements = int(np.prod(shape))
+        tensors[name] = {
+            "shape": shape,
+            "n_elements": n_elements,
+            "size_mib": n_elements * 2 / 1024 / 1024,
+        }
+
+    arch = detect_architecture(tensors)
+    arch_features = ARCH_FEATURES.get(arch, {}).copy()
+
+    prefix = _detect_prefix(tensors)
+    n_layers = _estimate_layers(tensors)
+
+    if arch == "mellum2" and arch_features.get("moe_intermediate_size", 0) == 0:
+        arch_features["moe_intermediate_size"] = 896
+
+    arch_features["prefix"] = prefix
+    if n_layers > 0:
+        arch_features["n_layers"] = n_layers
+
+    has_moe = any("exps" in n for n in tensors)
+    if has_moe:
+        arch_features["has_moe"] = True
+
+    has_nextn = any("nextn" in n for n in tensors)
+    # blk.32 is MTP only in ~32-layer models (Qwen). Skip for deeper models (Gemma4, 48 layers).
+    has_blk32 = n_layers <= 33 and any(
+        n.startswith("blk.32.") or n.startswith("BLK.32.") for n in tensors
+    )
+    arch_features["has_mtp"] = has_nextn or has_blk32
+
+    return {
+        "path": path,
+        "architecture": arch,
+        "features": arch_features,
+        "tensors": tensors,
+        "n_tensors": len(tensors),
+        "meta": meta,
+    }

quantizer.py

@@ -0,0 +1,75 @@
+import subprocess
+import re
+import os
+from utils import parse_size_line, parse_quant_size
+
+
+QUANTIZER_PATH = "/home/maxyag27/llm-tools/llama.cpp/build/bin/llama-quantize"
+
+
+def _build_cmd(flags: dict, model_in: str, model_out: str, dry_run: bool = False) -> list:
+    """Build llama-quantize command. Options before positional args."""
+    cmd = [QUANTIZER_PATH]
+
+    if dry_run:
+        cmd.append("--dry-run")
+
+    if flags.get("imatrix"):
+        imatrix = flags["imatrix"]
+        if isinstance(imatrix, list):
+            imatrix = imatrix[0]  # llama-quantize accepts one imatrix; we combined in importance table
+        cmd.extend(["--imatrix", imatrix])
+    cmd.extend(["--output-tensor-type", flags["output_tensor_type"]])
+    cmd.extend(["--token-embedding-type", flags["token_embedding_type"]])
+
+    for rule in flags["tensor_type_rules"]:
+        parts = rule.split(" ", 1)
+        if len(parts) == 2:
+            cmd.extend(["--tensor-type", parts[1].strip('"')])
+
+    # Positional args: model_in model_out type
+    cmd.append(model_in)
+    cmd.append(model_out)
+    cmd.append(flags["base_type"])
+
+    return cmd
+
+
+def run_dry_run(flags: dict, model_in: str) -> float | None:
+    """Run llama-quantize --dry-run and return quant size in MiB."""
+    cmd = _build_cmd(flags, model_in, "/dev/null", dry_run=True)
+
+    result = subprocess.run(
+        cmd,
+        capture_output=True,
+        text=True,
+        timeout=300,
+    )
+
+    output = (result.stdout or "") + (result.stderr or "")
+    size = parse_quant_size(output)
+    if size is not None:
+        return size
+
+    print("STDERR:", result.stderr[:2000])
+    return None
+
+
+def run_quantization(flags: dict, model_in: str, model_out: str, dry_run: bool = False) -> bool:
+    """Run actual quantization or dry run."""
+    cmd = _build_cmd(flags, model_in, model_out, dry_run=dry_run)
+
+    print("Running:", " ".join(cmd[:6]) + " ...")
+    result = subprocess.run(cmd, capture_output=True, text=True, timeout=3600)
+
+    if dry_run:
+        output = (result.stdout or "") + (result.stderr or "")
+        return parse_quant_size(output)
+
+    print(result.stderr[:500])
+    success = result.returncode == 0
+    if success:
+        import os
+        size_mib = os.path.getsize(model_out) / 1024 / 1024
+        print(f"Done: {model_out} ({size_mib:.0f} MiB)")
+    return success

requirements.txt

@@ -0,0 +1,2 @@
+gguf>=0.10.0
+numpy>=1.24.0

utils.py

@@ -0,0 +1,53 @@
+import numpy as np
+from constants import GGUF_TYPE_NAMES, GGUF_TYPE_NAMES_INV
+
+
+def type_name(type_id: int) -> str:
+    return GGUF_TYPE_NAMES.get(type_id, f"UNKNOWN({type_id})")
+
+
+def type_id(name: str) -> int:
+    return GGUF_TYPE_NAMES_INV.get(name, -1)
+
+
+def parse_size_line(line: str) -> float | None:
+    import re
+    m = re.search(r"quant size\s*=\s*([0-9.]+)\s*MiB", line)
+    if m:
+        return float(m.group(1))
+    m = re.search(r"model size\s*=\s*([0-9.]+)\s*MiB", line)
+    if m:
+        return float(m.group(1))
+    return None
+
+
+def parse_quant_size(output: str) -> float | None:
+    """Extract quant size from full output (prefers quant over model size)."""
+    import re
+    # Try quant size first across all lines
+    m = re.search(r"quant size\s*=\s*([0-9.]+)\s*MiB", output)
+    if m:
+        return float(m.group(1))
+    # Fall back to model size
+    m = re.search(r"model size\s*=\s*([0-9.]+)\s*MiB", output)
+    if m:
+        return float(m.group(1))
+    return None
+
+
+def parse_fallback_warnings(output: str) -> int:
+    import re
+    return len(re.findall(r"converting to\s+(q[0-9]_[0-9KMS]|iq[0-9])", output))
+
+
+def format_size(mib: float) -> str:
+    if mib >= 1024:
+        return f"{mib/1024:.2f} GB"
+    return f"{mib:.0f} MiB"
+
+
+def get_tensor_type(tensor_name: str) -> str:
+    parts = tensor_name.split(".")
+    if len(parts) >= 2 and parts[0] == "blk":
+        return parts[2] if len(parts) >= 3 else "unknown"
+    return "global"