File size: 2,637 Bytes
23a8c4a 8ee1b39 61496d7 e199ff4 8ee1b39 acc7fca 8ee1b39 acc7fca 22baa7c acc7fca 8ee1b39 e857d03 acc7fca e199ff4 acc7fca 8ee1b39 acc7fca 8ee1b39 acc7fca 8ee1b39 acc7fca 8ee1b39 acc7fca 8ee1b39 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ---
license: mit
---
# editor
model editor plus tensor level quantization engine - all in one
## install via pip/pip3
```bash
pip install gguf-editor
```
building the bundled quantizer requires a C/C++ toolchain and CMake ≥ 3.15
(on Windows: MSVC Build Tools). GPU accelerators are opt-in at build time:
```bash
CMAKE_ARGS="-DQUANTIZER_CUDA=ON" pip install gguf-editor # NVIDIA (CUDA)
CMAKE_ARGS="-DQUANTIZER_HIP=ON" pip install gguf-editor # AMD (ROCm/HIP)
CMAKE_ARGS="-DQUANTIZER_METAL=ON" pip install gguf-editor # Apple (macOS)
```
*if you build it from your machine, it will automatically render your CUDA driver (assuming cuda), hence, don't need to copy the bulky .DLLs from CUDA kit to the package directory; and the model.gguf in this repo is generated just for testing purposes (see examples below)
## usage
```bash
gguf-editor # launch the editor GUI in the browser
gguf-editor model.gguf # …opening a file right away
```
editor features (as in the desktop editor / chrome extension):
- inspect and edit metadata (all value types incl. arrays), add/delete keys
- rename, delete, reorder (drag), merge tensors; add zero-filled tensors;
import tensors from another GGUF
- find & replace across tensor names (literal or regex)
- per-tensor precision changes and/or a batch weight type — on save the file
is rebuilt with your edits and then converted by the quantizer into a
single output file
- streams tensor data disk-to-disk on save with live progress
CLI quantizer (mirrors the standalone `quantizer` binary):
```bash
gguf-editor quantize -m model-f16.gguf -o model-q4_k.gguf --type q4_k
gguf-editor quantize -m model.safetensors -o model-q8_0.gguf --type q8_0
gguf-editor quantize -m model.gguf -o out.gguf \
--tensor-type-rules "attention.*weight=q4_k" --device auto
gguf_editor devices
```
recently supported types: `f32 f16 bf16 q4_0 q4_1 q5_0 q5_1 q8_0 q1_0 q2_k q3_k q4_k
q5_k q6_k iq1_s iq1_m iq2_xxs iq2_xs iq2_s iq3_xxs iq3_s iq4_nl iq4_xs tq1_0
tq2_0 mxfp4 nvfp4`. Inputs may be GGUF or safetensors (auto-detected;
multi-part safetensors are merged automatically).
## Python API
```python
from gguf_editor import gguf, quantizer
parsed = gguf.parse_file("model.gguf") # header-only parse
print(parsed.version, len(parsed.tensor_infos))
quantizer.quantize("model-f16.gguf", "model-q4_k.gguf", default_type="q4_k")
```
or run it with `gguf-connector`
```
ggc et
```
|