stan-hua
/

Mistral-7B-Instruct-v0.3-AWQ-4bit

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions

Mistral-7B-Instruct-v0.3-AWQ-4bit

4.17 GB

Ctrl+K

Ctrl+K

1 contributor

History: 2 commits

stan-hua's picture

AWQ model for mistralai/Mistral-7B-Instruct-v0.3: {'w_bit': 4, 'zero_point': True, 'q_group_size': 128, 'version': 'GEMM'}

daba97d verified almost 2 years ago

.gitattributes

1.52 kB
initial commit almost 2 years ago
config.json

927 Bytes
AWQ model for mistralai/Mistral-7B-Instruct-v0.3: {'w_bit': 4, 'zero_point': True, 'q_group_size': 128, 'version': 'GEMM'} almost 2 years ago
generation_config.json

132 Bytes
AWQ model for mistralai/Mistral-7B-Instruct-v0.3: {'w_bit': 4, 'zero_point': True, 'q_group_size': 128, 'version': 'GEMM'} almost 2 years ago
model.safetensors

4.16 GB
xet

AWQ model for mistralai/Mistral-7B-Instruct-v0.3: {'w_bit': 4, 'zero_point': True, 'q_group_size': 128, 'version': 'GEMM'} almost 2 years ago
quant_config.json

90 Bytes
AWQ model for mistralai/Mistral-7B-Instruct-v0.3: {'w_bit': 4, 'zero_point': True, 'q_group_size': 128, 'version': 'GEMM'} almost 2 years ago
special_tokens_map.json

414 Bytes
AWQ model for mistralai/Mistral-7B-Instruct-v0.3: {'w_bit': 4, 'zero_point': True, 'q_group_size': 128, 'version': 'GEMM'} almost 2 years ago
tokenizer.json

1.96 MB
AWQ model for mistralai/Mistral-7B-Instruct-v0.3: {'w_bit': 4, 'zero_point': True, 'q_group_size': 128, 'version': 'GEMM'} almost 2 years ago
tokenizer.model

587 kB
xet

AWQ model for mistralai/Mistral-7B-Instruct-v0.3: {'w_bit': 4, 'zero_point': True, 'q_group_size': 128, 'version': 'GEMM'} almost 2 years ago
tokenizer_config.json

137 kB
AWQ model for mistralai/Mistral-7B-Instruct-v0.3: {'w_bit': 4, 'zero_point': True, 'q_group_size': 128, 'version': 'GEMM'} almost 2 years ago