Kernels:

---

ggml-org

/

ggml-quantization

Like 5

Follow

ggml-org 3.25k

Trusted publisher

This is the repository card of ggml-org/ggml-quantization that has been pushed on the Hub. It was built to be used with the kernels library. This card was automatically generated.

How to use

# make sure `kernels` is installed: `pip install -U kernels`
from kernels import get_kernel

kernel_module = get_kernel("ggml-org/ggml-quantization", version=1)
GEMV_TYPES = kernel_module.GEMV_TYPES

GEMV_TYPES(...)

Available functions

• GEMV_TYPES
• MAX_GEMV_ROWS
• dequantize
• get_rows
• mul_mat_id
• mul_mat_vec

Benchmarks

No benchmark available yet.