Append optimization guide to README

README.md

@@ -61,3 +61,96 @@ Note: Gemma-3 is dense (no MoE), so `*.shared_experts.*` is not applicable.
 - Base model: [VIDraft/Gemma-3-R1984-27B](https://huggingface.co/VIDraft/Gemma-3-R1984-27B)
 - Quantization: [exllamav3](https://github.com/turboderp-org/exllamav3) v0.0.34
 - Optimization method: [ArtusDev](https://huggingface.co/ArtusDev)
+
+---
+
+# EXL3 Optimization Guide
+
+## Targets
+`2.5bpw_H6 3.0bpw_H6 3.5bpw_H6 4.0bpw_H6 4.5bpw_H6 5.0bpw_H6 6.0bpw_H6`
+
+| Target | Action |
+|---|---|
+| 2.5bpw_H6 | optimized |
+| 3.0bpw_H6 | direct convert |
+| 3.5bpw_H6 | optimized |
+| 4.0bpw_H6 | optimized |
+| 4.5bpw_H6 | optimized |
+| 5.0bpw_H6 | direct convert |
+| 6.0bpw_H6 | direct convert |
+
+## Overview
+Dynamic EXL3 quants mix tensor precision, similar to mixed-precision GGUFs. There are two frameworks:
+
+- **Optimization**
+- **Recompilation**
+
+Usually, optimization and recompilation are used together: create a mixed quant through optimization, then run recompilation on top of it.
+
+## Optimization
+1. Start with two quants at different bpw, for example 2bpw and 3bpw.
+2. `measure.py` measures KLD differences by replacing layer groups in the lower-bpw quant with groups from the higher-bpw quant; standard EXL3 calibration data is used.
+3. The resulting `measurement.json` can be reused. You only have to create it once, no matter how many mixed quants you make.
+4. `optimize.py` uses that `measurement.json` to create a third quant from two source quants, replacing the tensors that matter most with higher-bpw tensors.
+
+Measurement takes about 20min to an hour for big models. Optimization takes about 30s-1m.
+
+```bash
+python util/measure.py -i /path/to/model-2bpw /path/to/model-3bpw -r /path/to/hf-model -o measurement.json -cr 10 -cc 1024 -d 0
+python util/optimize.py -i /path/to/model-2bpw /path/to/model-3bpw -m measurement.json -o /path/to/model-optimized -b 2.5 -ss 8192
+```
+
+Alternative measure form with `-ms`:
+```bash
+python util/measure.py -r /workspace/models/original-model -ms 128 -i /workspace/models/quant-2.5bpw /workspace/models/quant-3.5bpw -o /workspace/measurement.json
+```
+
+Optimize example:
+```bash
+python util/optimize.py -i /workspace/models/quant-2.5bpw /workspace/models/quant-3.0bpw -o /workspace/models/new-quant-2.75bpw -m /workspace/measurement.json -b 2.75
+```
+
+## Recompilation
+`override.yaml` replaces tensors in one quant with tensors from another quant. It is manual optimization. The source notes that attention and shared expert tensors were replaced with 8bpw tensors for all optimized quants. Recompilation takes about 30s-1m, and the actual new bpw is known after recompilation is done.
+
+### Artus multi-source example
+```yaml
+sources:
+  - id: 6
+    model_dir: /path/to/6bpw
+  - id: 8
+    model_dir: /path/to/8bpw
+overrides:
+  - key: "*.self_attn.*"
+    source: 6
+  - key: "*.shared_experts.*"
+    source: 8
+```
+
+### GLM-Air example
+The GLM-Air example replaces attention and shared experts with 8bpw tensors, and layers 2, 43, 1, 29 with 5bpw tensors because `measurement.json` showed those layers had the worst KLD.
+
+```yaml
+sources:
+  - id: 8
+    model_dir: /workspace/models/quants-8.0bpw
+  - id: 5
+    model_dir: /workspace/models/quants-5.0bpw
+overrides:
+  - key: "*.self_attn.*"
+    source: 8
+  - key: "*.shared_experts.*"
+    source: 8
+  - key: "model.layers.2.*"
+    source: 5
+  - key: "model.layers.43.*"
+    source: 5
+  - key: "model.layers.1.*"
+    source: 5
+  - key: "model.layers.29.*"
+    source: 5
+```
+
+```bash
+python util/recompile.py -i /workspace/models/quant-2.75bpw -o /workspace/models/quant-recompiled -or override.yaml
+```