| --- |
| license: mit |
| --- |
| # editor |
| model editor plus tensor level quantization engine - all in one |
|
|
|  |
|
|
| ## install via pip/pip3 |
|
|
| ```bash |
| pip install gguf-editor |
| ``` |
|
|
| building the bundled quantizer requires a C/C++ toolchain and CMake ≥ 3.15 |
| (on Windows: MSVC Build Tools). GPU accelerators are opt-in at build time: |
|
|
| ```bash |
| CMAKE_ARGS="-DQUANTIZER_CUDA=ON" pip install gguf-editor # NVIDIA (CUDA) |
| CMAKE_ARGS="-DQUANTIZER_HIP=ON" pip install gguf-editor # AMD (ROCm/HIP) |
| CMAKE_ARGS="-DQUANTIZER_METAL=ON" pip install gguf-editor # Apple (macOS) |
| ``` |
| *if you build it from your machine, it will automatically render your CUDA driver (assuming cuda), hence, don't need to copy the bulky .DLLs from CUDA kit to the package directory; and the model.gguf in this repo is generated just for testing purposes (see examples below) |
| |
| ## usage |
| |
| ```bash |
| gguf-editor # launch the editor GUI in the browser |
| gguf-editor model.gguf # …opening a file right away |
| ``` |
| |
| editor features (as in the desktop editor / chrome extension): |
| |
| - inspect and edit metadata (all value types incl. arrays), add/delete keys |
| - rename, delete, reorder (drag), merge tensors; add zero-filled tensors; |
| import tensors from another GGUF |
| - find & replace across tensor names (literal or regex) |
| - per-tensor precision changes and/or a batch weight type — on save the file |
| is rebuilt with your edits and then converted by the quantizer into a |
| single output file |
| - streams tensor data disk-to-disk on save with live progress |
| |
| CLI quantizer (mirrors the standalone `quantizer` binary): |
| |
| ```bash |
| gguf-editor quantize -m model-f16.gguf -o model-q4_k.gguf --type q4_k |
| gguf-editor quantize -m model.safetensors -o model-q8_0.gguf --type q8_0 |
| gguf-editor quantize -m model.gguf -o out.gguf \ |
| --tensor-type-rules "attention.*weight=q4_k" --device auto |
| gguf_editor devices |
| ``` |
| |
| recently supported types: `f32 f16 bf16 q4_0 q4_1 q5_0 q5_1 q8_0 q1_0 q2_k q3_k q4_k |
| q5_k q6_k iq1_s iq1_m iq2_xxs iq2_xs iq2_s iq3_xxs iq3_s iq4_nl iq4_xs tq1_0 |
| tq2_0 mxfp4 nvfp4`. Inputs may be GGUF or safetensors (auto-detected; |
| multi-part safetensors are merged automatically). |
| |
| ## Python API |
| |
| ```python |
| from gguf_editor import gguf, quantizer |
| |
| parsed = gguf.parse_file("model.gguf") # header-only parse |
| print(parsed.version, len(parsed.tensor_infos)) |
| |
| quantizer.quantize("model-f16.gguf", "model-q4_k.gguf", default_type="q4_k") |
| ``` |
| |
| or run it with `gguf-connector` |
| ``` |
| ggc et |
| ``` |
| |
|  |
| |
