buthainaaa commited on
Commit
b262c1c
·
verified ·
1 Parent(s): aacd1cd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -12
README.md CHANGED
@@ -14,23 +14,13 @@ base_model: QCRI/Fanar-1-9B-Instruct
14
 
15
  # Fanar-1-9B-Instruct-AWQ
16
 
17
- AWQ 4-bit quantized version of [QCRI/Fanar-1-9B-Instruct](https://huggingface.co/QCRI/Fanar-1-9B-Instruct).
18
 
19
- ## Quick Start
20
 
21
- ```python
22
- from vllm import LLM
23
-
24
- llm = LLM(
25
- model="buthainaaa/Fanar-1-9B-Instruct-GPTQ",
26
- quantization="awq",
27
- dtype="half"
28
- )
29
- ```
30
 
31
  ## Details
32
 
33
- - **Quantization:** AWQ 4-bit (w4a16)
34
  - **Size:** ~5GB (vs ~18GB original)
35
  - **Memory:** 75% reduction
36
  - **Quality:** 95%+ retention
 
14
 
15
  # Fanar-1-9B-Instruct-AWQ
16
 
17
+ GPTQ 4-bit quantized version of [QCRI/Fanar-1-9B-Instruct](https://huggingface.co/QCRI/Fanar-1-9B-Instruct).
18
 
 
19
 
 
 
 
 
 
 
 
 
 
20
 
21
  ## Details
22
 
23
+ - **Quantization:** GPTQ 4-bit (w4a16)
24
  - **Size:** ~5GB (vs ~18GB original)
25
  - **Memory:** 75% reduction
26
  - **Quality:** 95%+ retention