Update README.md
Browse files
README.md
CHANGED
|
@@ -6,7 +6,7 @@ base_model:
|
|
| 6 |
|
| 7 |
This is a fine-tuned and quark quantized version of Qwen/Qwen2.5-Coder-7B-Instruct using the 'iat-05-1' adapter.
|
| 8 |
|
| 9 |
-
Model Details
|
| 10 |
Base Model: Qwen/Qwen2.5-Coder-7B-Instruct
|
| 11 |
Adapter: iat-05-1
|
| 12 |
Quantization: Quark / UINT4 / AWQ / BFLOAT16
|
|
@@ -14,7 +14,9 @@ Format: SafeTensors
|
|
| 14 |
Perplexity Score: 10.953088
|
| 15 |
Dataset: wikitext-2-raw-v1
|
| 16 |
|
| 17 |
-
|
|
|
|
|
|
|
| 18 |
Config(
|
| 19 |
global_quant_config=QuantizationConfig(
|
| 20 |
input_tensors=None,
|
|
@@ -57,9 +59,4 @@ Config(
|
|
| 57 |
log_severity_level=1,
|
| 58 |
version="0.10",
|
| 59 |
)
|
| 60 |
-
|
| 61 |
-
[QUARK-INFO]: In-place OPs replacement start.
|
| 62 |
-
|
| 63 |
-
[QUARK-INFO]: Module exclusion from quantization summary:
|
| 64 |
-
| Exclude pattern | Number of modules excluded |
|
| 65 |
-
| [] | 0 |
|
|
|
|
| 6 |
|
| 7 |
This is a fine-tuned and quark quantized version of Qwen/Qwen2.5-Coder-7B-Instruct using the 'iat-05-1' adapter.
|
| 8 |
|
| 9 |
+
### Model Details
|
| 10 |
Base Model: Qwen/Qwen2.5-Coder-7B-Instruct
|
| 11 |
Adapter: iat-05-1
|
| 12 |
Quantization: Quark / UINT4 / AWQ / BFLOAT16
|
|
|
|
| 14 |
Perplexity Score: 10.953088
|
| 15 |
Dataset: wikitext-2-raw-v1
|
| 16 |
|
| 17 |
+
### Quark Info
|
| 18 |
+
Quantizing with the quantization configuration:
|
| 19 |
+
```bash
|
| 20 |
Config(
|
| 21 |
global_quant_config=QuantizationConfig(
|
| 22 |
input_tensors=None,
|
|
|
|
| 59 |
log_severity_level=1,
|
| 60 |
version="0.10",
|
| 61 |
)
|
| 62 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|