Document automatic native kernel provisioning (orbitquant 0.6.0)
Browse files
README.md
CHANGED
|
@@ -19,9 +19,20 @@ OrbitQuant is a calibration-free post-training quantization method for image and
|
|
| 19 |
Install OrbitQuant and the Hugging Face runtime dependencies:
|
| 20 |
|
| 21 |
```bash
|
| 22 |
-
pip install "orbitquant[hf,kernels]>=0.
|
| 23 |
```
|
| 24 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
Download this model repository as an OrbitQuant artifact, then load the source Diffusers pipeline with the quantized component patched in:
|
| 26 |
|
| 27 |
```python
|
|
|
|
| 19 |
Install OrbitQuant and the Hugging Face runtime dependencies:
|
| 20 |
|
| 21 |
```bash
|
| 22 |
+
pip install "orbitquant[hf,kernels]>=0.6.0"
|
| 23 |
```
|
| 24 |
|
| 25 |
+
The optimized native kernel package is provisioned automatically at first
|
| 26 |
+
model load: OrbitQuant derives the runtime variant (torch minor and CUDA
|
| 27 |
+
version for CUDA, the torch stable ABI for CPU, plus OS and architecture),
|
| 28 |
+
downloads the matching prebuilt wheel from the
|
| 29 |
+
[OrbitQuant kernels release](https://github.com/iamwavecut/OrbitQuant/releases/tag/kernels-v1)
|
| 30 |
+
with checksum verification, and caches it under `~/.cache/orbitquant/kernels`.
|
| 31 |
+
Without a matching variant, CUDA inference falls back to the Triton packed
|
| 32 |
+
path. Run `orbitquant kernels-install` to provision explicitly. For ComfyUI,
|
| 33 |
+
the [ComfyUI-OrbitQuant](https://github.com/iamwavecut/ComfyUI-OrbitQuant)
|
| 34 |
+
node pack provisions the same kernels from its install hook.
|
| 35 |
+
|
| 36 |
Download this model repository as an OrbitQuant artifact, then load the source Diffusers pipeline with the quantized component patched in:
|
| 37 |
|
| 38 |
```python
|