Hugging Face's logo

Sinensis
/
Anubis-70B-v1.1-AWQ

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
4-bit precision
awq
Model card Files
xet
Community

AWQ quantization of TheDrummer/Anubis-70B-v1.1

quantization_config:
bits: 4
group_size: 128
quant_method: awq
version: gemm
zero_point: true

Downloads last month
20
Safetensors
Model size
71B params
Tensor type
I32
·
BF16
·
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Sinensis/Anubis-70B-v1.1-AWQ

Base model

meta-llama/Llama-3.1-70B
Finetuned
meta-llama/Llama-3.3-70B-Instruct
Finetuned
TheDrummer/Anubis-70B-v1.1
Quantized
(15)
this model