Astheral GGUF

GGUF quantizations of SEE7/Astheral, generated with llama.cpp 5676dfbe01e7ebe0387913680eee0ed39bf4a363.

Provided Quants

Measured decimal GB, sorted by file size. Each linked file has passed GGUF structure and upload hash checks.

Link Type Size/GB
GGUF imatrix 0.010
GGUF i1-IQ1_S 2.020
GGUF i1-IQ1_M 2.162
GGUF i1-IQ2_XXS 2.399
GGUF i1-IQ2_XS 2.606
GGUF i1-IQ2_S 2.758
GGUF i1-IQ2_M 2.948
GGUF i1-Q2_K_S 2.989
GGUF i1-Q2_K 3.179
GGUF i1-IQ3_XXS 3.275
GGUF i1-IQ3_XS 3.519
GGUF i1-Q3_K_S 3.664
GGUF i1-IQ3_S 3.682
GGUF i1-IQ3_M 3.785
GGUF i1-Q3_K_M 4.019
GGUF i1-Q3_K_L 4.322
GGUF i1-IQ4_XS 4.448
GGUF i1-Q4_0 4.676
GGUF i1-IQ4_NL 4.678
GGUF i1-Q4_K_S 4.693
GGUF i1-Q4_K_M 4.921
GGUF i1-Q4_1 5.130
GGUF i1-Q5_K_S 5.599
GGUF i1-Q5_K_M 5.733
GGUF i1-Q6_K 6.596

Calibration

Importance data is collected from real forward passes over a deterministic multilingual and instruction sample of AstheralDataset. Corpus hash, chunk settings, exact tensor formats, and file SHA-256 hashes are recorded in manifest.json.

Built with Llama. See LICENSE, UPSTREAM_LLAMA3_LICENSE, and NOTICE.

Downloads last month
4,131
GGUF
Model size
8B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

1-bit

2-bit

3-bit

4-bit

5-bit

6-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for SEE7/Astheral-GGUF

Finetuned
SEE7/Astheral
Quantized
(3)
this model