Update README.md
Browse files
README.md
CHANGED
|
@@ -21,7 +21,7 @@ quantized_by: NightForger
|
|
| 21 |
It is just fast gptq 4q version of [this model](https://huggingface.co/IlyaGusev/saiga_nemo_12b).
|
| 22 |
|
| 23 |
# Quantize config:
|
| 24 |
-
```
|
| 25 |
{
|
| 26 |
"bits": 4,
|
| 27 |
"group_size": 128,
|
|
@@ -41,7 +41,7 @@ It is just fast gptq 4q version of [this model](https://huggingface.co/IlyaGusev
|
|
| 41 |
1024 examples from [SFT set](https://huggingface.co/datasets/IlyaGusev/saiga_scored).
|
| 42 |
|
| 43 |
# Code example (roleplay):
|
| 44 |
-
```
|
| 45 |
# Please don`t use this code (try vllm or exllama)
|
| 46 |
import torch
|
| 47 |
from transformers import AutoTokenizer, AutoConfig, GenerationConfig
|
|
|
|
| 21 |
It is just fast gptq 4q version of [this model](https://huggingface.co/IlyaGusev/saiga_nemo_12b).
|
| 22 |
|
| 23 |
# Quantize config:
|
| 24 |
+
```json
|
| 25 |
{
|
| 26 |
"bits": 4,
|
| 27 |
"group_size": 128,
|
|
|
|
| 41 |
1024 examples from [SFT set](https://huggingface.co/datasets/IlyaGusev/saiga_scored).
|
| 42 |
|
| 43 |
# Code example (roleplay):
|
| 44 |
+
```python
|
| 45 |
# Please don`t use this code (try vllm or exllama)
|
| 46 |
import torch
|
| 47 |
from transformers import AutoTokenizer, AutoConfig, GenerationConfig
|