Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -15,6 +15,12 @@ Credits to Ajay Jaiswal, Jinhao Duan, Zhenyu Zhang, Zhangheng Li, Lu Yin, Shiwei
|
|
| 15 |
|
| 16 |
License: [MIT License](https://opensource.org/license/mit/)
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
Setup environment
|
| 19 |
```shell
|
| 20 |
pip install torch==2.0.0+cu117 torchvision==0.15.1+cu117 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu117
|
|
@@ -23,6 +29,8 @@ pip install accelerate
|
|
| 23 |
pip install auto-gptq # for gptq
|
| 24 |
```
|
| 25 |
|
|
|
|
|
|
|
| 26 |
How to use pruned models
|
| 27 |
```python
|
| 28 |
import torch
|
|
|
|
| 15 |
|
| 16 |
License: [MIT License](https://opensource.org/license/mit/)
|
| 17 |
|
| 18 |
+
Simplified lists:
|
| 19 |
+
* Models: Llama-2 13b, Llama-2 chat 13b, Vicuna 13b v1.3
|
| 20 |
+
* Compression methods:
|
| 21 |
+
- Pruning: Magnitude-based, Wanda, SparseGPT (2:4 semi-structured)
|
| 22 |
+
- Quantization: AWQ, GPTQ (3,4,8 bits)
|
| 23 |
+
|
| 24 |
Setup environment
|
| 25 |
```shell
|
| 26 |
pip install torch==2.0.0+cu117 torchvision==0.15.1+cu117 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu117
|
|
|
|
| 29 |
pip install auto-gptq # for gptq
|
| 30 |
```
|
| 31 |
|
| 32 |
+
## How to use models
|
| 33 |
+
|
| 34 |
How to use pruned models
|
| 35 |
```python
|
| 36 |
import torch
|