metadata
license: apache-2.0
tags:
- OmniQuant
- Quantization
- Weight-Quantization
This repo contains all the required Learnable Weight Clipping for Omniquant https://arxiv.org/abs/2308.13137.
How to use it?
To use them please first run:
!mkdir OmniQuant_LWC
!git lfs install
!git clone https://huggingface.co/Tfloow/OmniQuant
!cp Omniquant/* OmniQuant_LWC/ # To avoid conflict in names
!git clone https://github.com/OpenGVLab/OmniQuant/tree/main
Then you can run OmniQuant as usual with the flag --resume:
CUDA_VISIBLE_DEVICES=0 python OmniQuant/main.py \
--model NAME_OF_MODEL \
--epochs 0 --output_dir ./log/test \
--eval_ppl --wbits 4 --abits 16 --group_size 128 --lwc \
--resume OmniQuant_LWC/NAME_OF_MODEL-w4a16g128.pth
Methodology
The weights were run using a fork of OmniQuant available at calibration.ipynb