Update code comments
Browse files- hexstate_requantize.py +17 -5
hexstate_requantize.py
CHANGED
|
@@ -1,15 +1,27 @@
|
|
| 1 |
#!/usr/bin/env python3
|
| 2 |
"""
|
| 3 |
-
|
| 4 |
|
| 5 |
Reads a source GGUF (F16/BF16/F32), copies all metadata verbatim,
|
| 6 |
-
and re-quantizes eligible weight tensors
|
|
|
|
| 7 |
|
| 8 |
-
|
| 9 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
|
| 11 |
Usage:
|
| 12 |
-
python3 hexstate_requantize.py input.gguf output.gguf
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
"""
|
| 14 |
|
| 15 |
import struct
|
|
|
|
| 1 |
#!/usr/bin/env python3
|
| 2 |
"""
|
| 3 |
+
HexState GGUF Re-Quantizer — GGUF-to-GGUF HPC quantization.
|
| 4 |
|
| 5 |
Reads a source GGUF (F16/BF16/F32), copies all metadata verbatim,
|
| 6 |
+
and re-quantizes eligible weight tensors using the HExState HPC engine
|
| 7 |
+
(Shor-optimized Griffiths-Niu measurement via libhexstate_q2k.so).
|
| 8 |
|
| 9 |
+
Quantization tiers:
|
| 10 |
+
- Attention Q/K/V/O + DeltaNet SSM projections → Q4_0 (HPC-optimized)
|
| 11 |
+
- FFN / MLP weight matrices → Q2_K (HPC-optimized)
|
| 12 |
+
- Embeddings, norms, biases, LM head → kept at source precision
|
| 13 |
+
|
| 14 |
+
Falls back to a pure numpy Q2_K implementation if the C library is not built.
|
| 15 |
|
| 16 |
Usage:
|
| 17 |
+
python3 hexstate_requantize.py input.gguf output.gguf [options]
|
| 18 |
+
|
| 19 |
+
Options:
|
| 20 |
+
--config <file> Load HuggingFace config.json for arch detection
|
| 21 |
+
--imatrix <file> Importance matrix for calibrated quantization
|
| 22 |
+
--keep-metadata Preserve all GGUF metadata as-is
|
| 23 |
+
--q2all Force all eligible tensors to Q2_K
|
| 24 |
+
--quantize-none Skip quantization (passthrough)
|
| 25 |
"""
|
| 26 |
|
| 27 |
import struct
|