GGUF
calcuis commited on
Commit
acc7fca
·
verified ·
1 Parent(s): 8ee1b39

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -15
README.md CHANGED
@@ -4,31 +4,68 @@ license: mit
4
  # editor
5
  model editor plus tensor level quantization engine - all in one
6
 
7
- clone this repo
8
- ```
9
- git clone https://github.com/gguf-org/editor
10
- ```
11
 
12
- enter the directory
13
- ```
14
- cd editor
15
  ```
16
 
17
- install the dependencies required
 
 
 
 
 
 
18
  ```
19
- npm i
 
 
 
 
 
20
  ```
21
 
22
- build the typescript frontend
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
  ```
24
- npm run build
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  ```
26
 
27
- run/test the app in developer mode
28
  ```
29
- npm run tauri:dev
30
  ```
31
 
32
- *if you are not prepared to build it yourself (though it's recommended), find the OS specific installer from [releases](https://github.com/mochiyaki/editor/releases)
33
-
34
  ![screenshot](https://raw.githubusercontent.com/gguf-org/gguf-desktop/master/pizza.jpg)
 
4
  # editor
5
  model editor plus tensor level quantization engine - all in one
6
 
7
+ ## Install
 
 
 
8
 
9
+ ```bash
10
+ pip install gguf-editor
 
11
  ```
12
 
13
+ building the bundled quantizer requires a C/C++ toolchain and CMake ≥ 3.15
14
+ (on Windows: MSVC Build Tools). GPU accelerators are opt-in at build time:
15
+
16
+ ```bash
17
+ CMAKE_ARGS="-DQUANTIZER_CUDA=ON" pip install gguf-editor # NVIDIA (CUDA toolkit)
18
+ CMAKE_ARGS="-DQUANTIZER_HIP=ON" pip install gguf-editor # AMD (ROCm/HIP)
19
+ CMAKE_ARGS="-DQUANTIZER_METAL=ON" pip install gguf-editor # Apple (macOS)
20
  ```
21
+
22
+ ## Usage
23
+
24
+ ```bash
25
+ gguf-editor # launch the editor GUI in the browser
26
+ gguf-editor model.gguf # …opening a file right away
27
  ```
28
 
29
+ editor features (as in the desktop editor / chrome extension):
30
+
31
+ - inspect and edit metadata (all value types incl. arrays), add/delete keys
32
+ - rename, delete, reorder (drag), merge tensors; add zero-filled tensors;
33
+ import tensors from another GGUF
34
+ - find & replace across tensor names (literal or regex)
35
+ - per-tensor precision changes and/or a batch weight type — on save the file
36
+ is rebuilt with your edits and then converted by the quantizer into a
37
+ single output file
38
+ - streams tensor data disk-to-disk on save with live progress
39
+
40
+ CLI quantizer (mirrors the standalone `quantizer` binary):
41
+
42
+ ```bash
43
+ gguf-editor quantize -m model-f16.gguf -o model-q4_k.gguf --type q4_k
44
+ gguf-editor quantize -m model.safetensors -o model-q8_0.gguf --type q8_0
45
+ gguf-editor quantize -m model.gguf -o out.gguf \
46
+ --tensor-type-rules "attention.*weight=q4_k" --device auto
47
+ gguf_editor devices
48
  ```
49
+
50
+ recently supported types: `f32 f16 bf16 q4_0 q4_1 q5_0 q5_1 q8_0 q1_0 q2_k q3_k q4_k
51
+ q5_k q6_k iq1_s iq1_m iq2_xxs iq2_xs iq2_s iq3_xxs iq3_s iq4_nl iq4_xs tq1_0
52
+ tq2_0 mxfp4 nvfp4`. Inputs may be GGUF or safetensors (auto-detected;
53
+ multi-part safetensors are merged automatically).
54
+
55
+ ## Python API
56
+
57
+ ```python
58
+ from gguf_editor import gguf, quantizer
59
+
60
+ parsed = gguf.parse_file("model.gguf") # header-only parse
61
+ print(parsed.version, len(parsed.tensor_infos))
62
+
63
+ quantizer.quantize("model-f16.gguf", "model-q4_k.gguf", default_type="q4_k")
64
  ```
65
 
66
+ or run it with `gguf-connector`
67
  ```
68
+ ggc et
69
  ```
70
 
 
 
71
  ![screenshot](https://raw.githubusercontent.com/gguf-org/gguf-desktop/master/pizza.jpg)