Update README.md
Browse files
README.md
CHANGED
|
@@ -29,7 +29,7 @@ AMD-Quark has been installed from source code inside the Docker image `rocm/vllm
|
|
| 29 |
|
| 30 |
**Quantization scripts:**
|
| 31 |
|
| 32 |
-
Step1
|
| 33 |
```
|
| 34 |
import runpy
|
| 35 |
from quark.torch import LLMTemplate
|
|
@@ -50,7 +50,7 @@ quantize_script = "/app/Quark/examples/torch/language_modeling/llm_ptq/quantize_
|
|
| 50 |
|
| 51 |
runpy.run_path(quantize_script, run_name="__main__")
|
| 52 |
```
|
| 53 |
-
|
| 54 |
```
|
| 55 |
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
| 56 |
export MODEL_DIR=zai-org/GLM-4.7
|
|
|
|
| 29 |
|
| 30 |
**Quantization scripts:**
|
| 31 |
|
| 32 |
+
- **Step1:** Creat the quantize_glm.py
|
| 33 |
```
|
| 34 |
import runpy
|
| 35 |
from quark.torch import LLMTemplate
|
|
|
|
| 50 |
|
| 51 |
runpy.run_path(quantize_script, run_name="__main__")
|
| 52 |
```
|
| 53 |
+
- **Step2:** Quantize with the quantize_glm.py
|
| 54 |
```
|
| 55 |
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
| 56 |
export MODEL_DIR=zai-org/GLM-4.7
|