File size: 914 Bytes
2f4514e 65910b3 92bccd9 65910b3 92bccd9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | ---
license: apache-2.0
tags:
- OmniQuant
- Quantization
- Weight-Quantization
---
This repo contains all the required Learnable Weight Clipping for Omniquant https://arxiv.org/abs/2308.13137.
# How to use it?
To use them please first run:
```
!mkdir OmniQuant_LWC
!git lfs install
!git clone https://huggingface.co/Tfloow/OmniQuant
!cp Omniquant/* OmniQuant_LWC/ # To avoid conflict in names
!git clone https://github.com/OpenGVLab/OmniQuant/tree/main
```
Then you can run OmniQuant as usual with the flag `--resume`:
```
CUDA_VISIBLE_DEVICES=0 python OmniQuant/main.py \
--model NAME_OF_MODEL \
--epochs 0 --output_dir ./log/test \
--eval_ppl --wbits 4 --abits 16 --group_size 128 --lwc \
--resume OmniQuant_LWC/NAME_OF_MODEL-w4a16g128.pth
```
# Methodology
The weights were run using a fork of OmniQuant available at [calibration.ipynb](https://github.com/Tfloow/AndaQuant/blob/main/calibration.ipynb) |