| --- |
| license: apache-2.0 |
| tags: |
| - OmniQuant |
| - Quantization |
| - Weight-Quantization |
| --- |
| |
| This repo contains all the required Learnable Weight Clipping for Omniquant https://arxiv.org/abs/2308.13137. |
|
|
| # How to use it? |
|
|
| To use them please first run: |
|
|
| ``` |
| !mkdir OmniQuant_LWC |
| !git lfs install |
| !git clone https://huggingface.co/Tfloow/OmniQuant |
| !cp Omniquant/* OmniQuant_LWC/ # To avoid conflict in names |
| !git clone https://github.com/OpenGVLab/OmniQuant/tree/main |
| ``` |
|
|
| Then you can run OmniQuant as usual with the flag `--resume`: |
|
|
| ``` |
| CUDA_VISIBLE_DEVICES=0 python OmniQuant/main.py \ |
| --model NAME_OF_MODEL \ |
| --epochs 0 --output_dir ./log/test \ |
| --eval_ppl --wbits 4 --abits 16 --group_size 128 --lwc \ |
| --resume OmniQuant_LWC/NAME_OF_MODEL-w4a16g128.pth |
| ``` |
|
|
| # Methodology |
|
|
| The weights were run using a fork of OmniQuant available at [calibration.ipynb](https://github.com/Tfloow/AndaQuant/blob/main/calibration.ipynb) |
