metadata
license: apache-2.0
Quantization Details
- Quantization Method: AWQ
- Bits: 4
- Group Size: 128
- Quantization Tool: ms-swift
Memory Usage
| Type | GUI-Owl-7B | GUI-Owl-7B-AWQ-4bit |
|---|---|---|
| Memory Size | 16G GB | 6.5 GB |
Inference
Run with:
vllm serve JessieWei/GUI-Owl-7B-AWQ-4bit
Detailed parameter tuning refer to GUI-Owl-7B.