ArGrigorov's picture
Upload folder using huggingface_hub
e9c8366 verified
Raw
History Blame Contribute Delete
1.11 kB
"""NF4 (NormalFloat 4-bit) quantization primitives β€” QLoRA paper format.
Pure primitive subpackage: tensor-level quantization functions only. No
nn.Module wrapper classes β€” the unified QuantizedModule (base.py) is the
single place for quantized layer wrappers.
Exports:
NF4_LUT β€” 16 fixed non-uniform levels (const tensor)
quantize_nf4 β€” quantize weight tensor to NF4 indices + per-group scale
dequantize_nf4 β€” reconstruct weight from indices + scale
pack_nf4 / unpack_nf4 β€” pack two 4-bit indices per byte
double_quantize_scales_2d β€” quantize scales to int8 + per-block fp32 scale
dequantize_scales_2d β€” reconstruct scales from int8 + block_scale
"""
from agiws_neural_quant.nf4.nf4 import (
NF4_LUT, quantize_nf4, dequantize_nf4, pack_nf4, unpack_nf4,
)
from agiws_neural_quant.nf4.double_quant import (
double_quantize_scales_2d, dequantize_scales_2d,
)
__all__ = [
"NF4_LUT",
"quantize_nf4",
"dequantize_nf4",
"pack_nf4",
"unpack_nf4",
"double_quantize_scales_2d",
"dequantize_scales_2d",
]